Getting OutofMemory error [java/lang/OutOfMemoryError] while deploying a large application [sterling ear file - smcfs.ear] in WebSphere Application Server

We are finding the following exception thrown in the SystemOut.log file when we deploy a large application (ex. sterling ear file - smcfs.ear) in WebSphere Application Server

Error in SystemOut.log file

[10/30/17 14:53:21:745 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] Starting workbench.

[10/30/17 14:53:21:751 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy]   @build@
[10/30/17 14:53:21:842 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] EJB Deploy configuration directory: /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/ejbdeploy/configuration/
[10/30/17 14:53:21:846 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] framework search path: /opt/IBM/WebSphere/AppServer_STER/deploytool/itp/plugins
[10/30/17 14:53:21:863 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] build:RAD851-I20121103_1921
[10/30/17 14:53:33:186 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] Creating the project.
[10/30/17 14:54:06:259 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2017/10/30 14:54:06 - please wait.

[10/30/17 14:54:06:912 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested System dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/core.20171030.145406.62101.0001.dmp' in response to an event
[10/30/17 14:54:10:663 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.62373.
[10/30/17 14:54:10:735 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I System dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/core.20171030.145406.62101.0001.dmp
[10/30/17 14:54:11:591 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Heap dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/heapdump.20171030.145406.62101.0002.phd' in response to an event
[10/30/17 14:54:12:563 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Heap dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/heapdump.20171030.145406.62101.0002.phd
[10/30/17 14:54:12:565 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Java dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/javacore.20171030.145406.62101.0003.txt' in response to an event
[10/30/17 14:54:12:654 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Java dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/javacore.20171030.145406.62101.0003.txt
[10/30/17 14:54:12:656 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Snap dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/Snap.20171030.145406.62101.0004.trc' in response to an event
[10/30/17 14:54:12:658 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Snap dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/Snap.20171030.145406.62101.0004.trc
[10/30/17 14:54:12:660 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
[10/30/17 14:54:13:000 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2017/10/30 14:54:13 - please wait.
[10/30/17 14:54:13:291 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Heap dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/heapdump.20171030.145413.62101.0005.phd' in response to an event
[10/30/17 14:54:14:281 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Heap dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/heapdump.20171030.145413.62101.0005.phd
[10/30/17 14:54:14:589 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Java dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/javacore.20171030.145413.62101.0006.txt' in response to an event
[10/30/17 14:54:14:613 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Java dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/javacore.20171030.145413.62101.0006.txt
[10/30/17 14:54:14:615 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP032I JVM requested Snap dump using '/opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/Snap.20171030.145413.62101.0007.trc' in response to an event
[10/30/17 14:54:14:616 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP010I Snap dump written to /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/Snap.20171030.145413.62101.0007.trc
[10/30/17 14:54:14:617 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
[10/30/17 14:54:14:800 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] Exception in thread "State Saver" java.lang.OutOfMemoryError: Java heap space
[10/30/17 14:54:14:805 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.util.jar.Manifest.read(Manifest.java:247)
[10/30/17 14:54:14:807 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.util.jar.Manifest.<init>(Manifest.java:81)
[10/30/17 14:54:14:808 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.util.jar.JarFile.getManifestFromReference(JarFile.java:198)
[10/30/17 14:54:14:813 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.util.jar.JarFile.getManifest(JarFile.java:180)
[10/30/17 14:54:14:815 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:1008)
[10/30/17 14:54:14:817 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.net.URLClassLoader.defineClass(URLClassLoader.java:654)
[10/30/17 14:54:14:819 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.net.URLClassLoader.access$400(URLClassLoader.java:92)
[10/30/17 14:54:14:821 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1159)
[10/30/17 14:54:14:823 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.security.AccessController.doPrivileged(AccessController.java:314)
[10/30/17 14:54:14:825 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.net.URLClassLoader.findClass(URLClassLoader.java:594)
[10/30/17 14:54:14:826 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:743)
[10/30/17 14:54:14:829 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.ClassLoader.loadClass(ClassLoader.java:711)
[10/30/17 14:54:14:833 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.ClassLoader.loadClass(ClassLoader.java:690)
[10/30/17 14:54:14:835 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.osgi.storagemanager.StorageManager.getOutputStream(StorageManager.java:777)
[10/30/17 14:54:14:837 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.osgi.internal.baseadaptor.BaseStorage.saveBundleDatas(BaseStorage.java:558)
[10/30/17 14:54:14:838 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.osgi.internal.baseadaptor.BaseStorage.saveAllData(BaseStorage.java:447)
[10/30/17 14:54:14:840 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.osgi.internal.baseadaptor.BaseStorage.access$1(BaseStorage.java:435)
[10/30/17 14:54:14:842 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.osgi.internal.baseadaptor.BaseStorage$StateSaver.run(BaseStorage.java:1227)
[10/30/17 14:54:14:844 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.Thread.run(Thread.java:780)
[10/30/17 14:54:15:249 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] Archive opener did not close archive: /tmp/smcfsshared.jar1403045142001111053.tmp
[10/30/17 14:54:15:251 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] Archive was opened here:
[10/30/17 14:54:15:252 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] org.eclipse.jst.jee.archive.internal.FailedToCloseException
[10/30/17 14:54:15:254 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.jee.archive.internal.ArchiveImpl.<init>(ArchiveImpl.java:135)
[10/30/17 14:54:15:255 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.jee.archive.internal.ArchiveFactoryImpl.openArchive(ArchiveFactoryImpl.java:53)
[10/30/17 14:54:15:256 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.JavaEEArchiveUtilities.openArchive(JavaEEArchiveUtilities.java:364)
[10/30/17 14:54:15:260 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.jee.archive.internal.ArchiveImpl.getNestedArchive(ArchiveImpl.java:294)
[10/30/17 14:54:15:262 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.ArchiveWrapper.getEarModules(ArchiveWrapper.java:191)
[10/30/17 14:54:15:263 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.application.internal.operations.EARComponentImportDataModelProvider.getModuleModels(EARComponentImportDataModelProvider.java:483)
[10/30/17 14:54:15:266 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.application.internal.operations.EARComponentImportDataModelProvider.propertySet(EARComponentImportDataModelProvider.java:164)
[10/30/17 14:54:15:272 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.wst.common.frameworks.internal.datamodel.DataModelImpl.internalSetProperty(DataModelImpl.java:182)
[10/30/17 14:54:15:276 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.wst.common.frameworks.internal.datamodel.DataModelImpl.setProperty(DataModelImpl.java:172)
[10/30/17 14:54:15:279 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.importEar(BatchExtension.java:1109)
[10/30/17 14:54:15:280 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.runEarDeploy(BatchExtension.java:379)
[10/30/17 14:54:15:282 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension$RunDeploy.run(BatchExtension.java:356)
[10/30/17 14:54:15:284 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1975)
[10/30/17 14:54:15:285 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1957)
[10/30/17 14:54:15:294 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension$RunDeploy.execute(BatchExtension.java:321)
[10/30/17 14:54:15:295 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.run(BatchExtension.java:284)
[10/30/17 14:54:15:296 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[10/30/17 14:54:15:298 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
[10/30/17 14:54:15:300 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[10/30/17 14:54:15:302 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.reflect.Method.invoke(Method.java:613)
[10/30/17 14:54:15:305 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
[10/30/17 14:54:15:308 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
[10/30/17 14:54:15:309 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
[10/30/17 14:54:15:311 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
[10/30/17 14:54:15:313 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
[10/30/17 14:54:15:314 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[10/30/17 14:54:15:316 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
[10/30/17 14:54:15:317 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[10/30/17 14:54:15:319 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.reflect.Method.invoke(Method.java:613)
[10/30/17 14:54:15:320 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.impl.BootLoaderLoader.run(BootLoaderLoader.java:494)
[10/30/17 14:54:15:323 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.impl.BatchDeploy.execute(BatchDeploy.java:114)
[10/30/17 14:54:15:325 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.EJBDeploy.execute(EJBDeploy.java:106)
[10/30/17 14:54:15:327 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.EJBDeploy.deploy(EJBDeploy.java:348)
[10/30/17 14:54:15:329 IST] 00000092 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.EJBDeploy.main(EJBDeploy.java:309)
[10/30/17 14:54:18:086 IST] 00000093 DeployEJBTask E   ADMA0086E: [EJBDeploy] An unexpected exception was thrown.  Halting execution.
[10/30/17 14:54:18:089 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] Shutting down workbench.
[10/30/17 14:54:18:768 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] Error executing deployment: java.lang.OutOfMemoryError. Error is Java heap space.
[10/30/17 14:54:18:771 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] java.lang.OutOfMemoryError: Java heap space
[10/30/17 14:54:18:779 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.Throwable.fillInStackTrace(Native Method)
[10/30/17 14:54:18:780 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.Throwable.<init>(Throwable.java:67)
[10/30/17 14:54:18:787 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.Throwable.<init>(Throwable.java:78)
[10/30/17 14:54:18:788 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.Error.<init>(Error.java:81)
[10/30/17 14:54:18:789 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:63)
[10/30/17 14:54:18:790 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
[10/30/17 14:54:18:792 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jdt.internal.core.util.ConstantPool.decodeEntry(ConstantPool.java:36)
[10/30/17 14:54:18:794 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jdt.internal.core.util.ClassFileAttribute.<init>(ClassFileAttribute.java:31)
[10/30/17 14:54:18:796 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jdt.internal.core.util.SignatureAttribute.<init>(SignatureAttribute.java:28)
[10/30/17 14:54:18:798 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jdt.internal.core.util.ClassFileReader.<init>(ClassFileReader.java:259)
[10/30/17 14:54:18:800 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jdt.core.ToolFactory.createDefaultClassFileReader(ToolFactory.java:269)
[10/30/17 14:54:18:802 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.JavaEEArchiveUtilities.isEJBArchive(JavaEEArchiveUtilities.java:900)
[10/30/17 14:54:18:804 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.JavaEEArchiveUtilities.refineForJavaEE(JavaEEArchiveUtilities.java:606)
[10/30/17 14:54:18:805 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.JavaEEArchiveUtilities.openArchive(JavaEEArchiveUtilities.java:370)
[10/30/17 14:54:18:807 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.jee.archive.internal.ArchiveImpl.getNestedArchive(ArchiveImpl.java:294)
[10/30/17 14:54:18:812 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.internal.archive.ArchiveWrapper.getEarModules(ArchiveWrapper.java:191)
[10/30/17 14:54:18:889 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.application.internal.operations.EARComponentImportDataModelProvider.getModuleModels(EARComponentImportDataModelProvider.java:483)
[10/30/17 14:54:18:896 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.jst.j2ee.application.internal.operations.EARComponentImportDataModelProvider.propertySet(EARComponentImportDataModelProvider.java:164)
[10/30/17 14:54:18:898 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.wst.common.frameworks.internal.datamodel.DataModelImpl.internalSetProperty(DataModelImpl.java:182)
[10/30/17 14:54:18:901 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.wst.common.frameworks.internal.datamodel.DataModelImpl.setProperty(DataModelImpl.java:172)
[10/30/17 14:54:18:915 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.importEar(BatchExtension.java:1109)
[10/30/17 14:54:18:920 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.runEarDeploy(BatchExtension.java:379)
[10/30/17 14:54:18:923 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension$RunDeploy.run(BatchExtension.java:356)
[10/30/17 14:54:18:928 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1975)
[10/30/17 14:54:18:931 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1957)
[10/30/17 14:54:18:940 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension$RunDeploy.execute(BatchExtension.java:321)
[10/30/17 14:54:18:951 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at com.ibm.etools.ejbdeploy.batch.plugin.BatchExtension.run(BatchExtension.java:284)
[10/30/17 14:54:18:957 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[10/30/17 14:54:18:960 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
[10/30/17 14:54:18:963 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[10/30/17 14:54:18:968 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at java.lang.reflect.Method.invoke(Method.java:613)
[10/30/17 14:54:18:970 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
[10/30/17 14:54:18:973 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
[10/30/17 14:54:18:975 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
[10/30/17 14:54:18:984 IST] 00000093 DeployEJBTask I   ADMA0158I: [EJBDeploy] EJBDeploy level: @build@
[10/30/17 14:54:19:829 IST] 00000090 SystemOut     O com.ibm.etools.ejbdeploy.EJBDeploymentException: Error executing EJBDeploy
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.etools.ejbdeploy.EJBDeploy.execute(EJBDeploy.java:251)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ejs.util.deployment.deploywrapper.DeployUtil$1.run(DeployUtil.java:356)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ejs.util.deployment.deploywrapper.DeployUtil.deployModule(DeployUtil.java:351)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ejs.util.deployment.deploywrapper.DeployUtil.deploy(DeployUtil.java:94)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ws.management.application.task.DeployEJBTask.ejbDeploy(DeployEJBTask.java:445)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ws.management.application.task.DeployEJBTask.install(DeployEJBTask.java:246)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ws.management.application.task.DeployEJBTask.performTask(DeployEJBTask.java:140)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at com.ibm.ws.management.application.SchedulerImpl.run(SchedulerImpl.java:315)
[10/30/17 14:54:19:830 IST] 00000090 SystemOut     O at java.lang.Thread.run(Thread.java:780)
[10/30/17 14:54:19:762 IST] 00000090 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/logs/ffdc/server1_e7466de5_17.10.30_14.54.19.6394410144076148983809.txt com.ibm.ws.management.application.task.DeployEJBTask.ejbDeploy 452
[10/30/17 14:54:19:783 IST] 00000090 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/logs/ffdc/server1_e7466de5_17.10.30_14.54.19.7682992154335726128636.txt com.ibm.ws.management.application.task.DeployEJBTask.ejbDeploy 459
[10/30/17 14:54:19:819 IST] 00000090 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer_STER/profiles/AppSrv01/logs/ffdc/server1_e7466de5_17.10.30_14.54.19.7845695867609147245583.txt com.ibm.ws.management.application.task.DeployEJBTask.performTask 167
[10/30/17 14:54:19:929 IST] 00000090 InstallSchedu I   ADMA5014E: The installation of application SterlingApplications failed.

Solution for above Error:

In Deploying process - Ejbdeploy handles its own memory settings in the file /opt/IBM/WebSphere/AppServer_STER/deploytool/itp/ejbdeploy.sh

In this file you will find the java memory variable 'EJBDEPLOY_JVM_HEAP' that defaults to 256M.

To increase the ejbdeploy java memory settings you can either set it file or in OS level.

1) Set it in /opt/IBM/WebSphere/AppServer_STER/deploytool/itp/ejbdeploy.sh file

EJBDEPLOY_JVM_HEAP="-Xms1024 -Xmx1024" at the beginning of the ejbdeploy.sh file.

2) Set it in operating System Environment.

Set EJBDEPLOY_JVM_HEAP= '-Xms1024 -Xmx1024' in OS environment variable.

EXPORT EJBDEPLOY_JVM_HEAP= '-Xms1024 -Xmx1024'

all the best...!
Reference Links: https://www.ibm.com/developerworks/community/forums/html/topic?id=bbe6064c-447f-4944-b868-b7b9d5b2a471
https://developer.ibm.com/answers/questions/192693/getting-outofmemory-condition-while-deploying-a-la.html