Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > java.lang.OutOfMemoryError

Reply
Thread Tools

java.lang.OutOfMemoryError

 
 
Rizwan
Guest
Posts: n/a
 
      12-13-2004
I have written a java program (an EJB) to read a file and write it to
database w.r.t. some rules. The program works ok but when the file has
around 6000 rows it fails and give an error message :

2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
Unexpected Error:
java.lang.OutOfMemoryError

My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000 on
Windows 2000/XP


Any reason? Thanks


 
Reply With Quote
 
 
 
 
Stefan Schulz
Guest
Posts: n/a
 
      12-13-2004
On Mon, 13 Dec 2004 14:38:50 -0500, Rizwan <(E-Mail Removed)> wrote:

> I have written a java program (an EJB) to read a file and write it to
> database w.r.t. some rules. The program works ok but when the file has
> around 6000 rows it fails and give an error message :
>
> 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
> Unexpected Error:
> java.lang.OutOfMemoryError
>
> My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000
> on
> Windows 2000/XP


Try making parts of the data eligable for Garbage Collection as soon as you
can, you are (obviously) running out of memory.

--

Whom the gods wish to destroy they first call promising.
 
Reply With Quote
 
 
 
 
srh
Guest
Posts: n/a
 
      12-13-2004
Hi,

I have find out where exactly it is failing but dont know how to fix
it. My program read the first row then based on some rules it generates
a SQL statement for this row in a string. I then add it to the batch
using addBatch method of Statement. When I finish reading the file I
executes the batch using executeBatch metohd of Statement. Thats where
it is failing. Any reason? Thanks

Statement stmt = cx.createStatement();
FileReader fr = new FileReader( file );
BufferedReader br = new BufferedReader( fr );
String lineData = null;
String insertStatementSQL = "";
while ( (lineData = br.readLine()) != null ) {
.....
stmt.addBatch(insertStatementSQL);
}

int returnValueIntArray[] = stmt.executeBatch();

Stefan Schulz wrote:
> On Mon, 13 Dec 2004 14:38:50 -0500, Rizwan <(E-Mail Removed)>

wrote:
>
> > I have written a java program (an EJB) to read a file and write it

to
> > database w.r.t. some rules. The program works ok but when the file

has
> > around 6000 rows it fails and give an error message :
> >
> > 2004-12-13 14:08:47,467 ERROR

[org.jboss.ejb.plugins.LogInterceptor]
> > Unexpected Error:
> > java.lang.OutOfMemoryError
> >
> > My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever

2000
> > on
> > Windows 2000/XP

>
> Try making parts of the data eligable for Garbage Collection as soon

as you
> can, you are (obviously) running out of memory.
>
> --
>
> Whom the gods wish to destroy they first call promising.


 
Reply With Quote
 
John C. Bollinger
Guest
Posts: n/a
 
      12-13-2004
Rizwan wrote:

> I have written a java program (an EJB) to read a file and write it to
> database w.r.t. some rules. The program works ok but when the file has
> around 6000 rows it fails and give an error message :
>
> 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
> Unexpected Error:
> java.lang.OutOfMemoryError
>
> My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000 on
> Windows 2000/XP
>
>
> Any reason? Thanks


Yes. You are running out of memory.

This probably means you are creating huge objects or huge numbers of
objects, retaining references to objects you no longer need, or both. I
cannot possibly tell you which of those or where, for the dry cleaner
lost my magic turban and no one seems to want to make me a new one.


John Bollinger
http://www.velocityreviews.com/forums/(E-Mail Removed)
 
Reply With Quote
 
Tilman Bohn
Guest
Posts: n/a
 
      12-13-2004
On Mon, 13 Dec 2004 12:51:30 -0800, srh wrote:

> Hi,
>
> I have find out where exactly it is failing but dont know how to fix
> it. My program read the first row then based on some rules it generates
> a SQL statement for this row in a string. I then add it to the batch
> using addBatch method of Statement. When I finish reading the file I
> executes the batch using executeBatch metohd of Statement. Thats where
> it is failing. Any reason?


Yes. Isn't this obvious?

Either try dispatching smaller batches at a time or give your VM
more memory to work with. The latter option only moves the threshold
at which your app will fail, so the former is preferred.

Cheers, Tilman

--
`Boy, life takes a long time to live...' -- Steven Wright

 
Reply With Quote
 
srh
Guest
Posts: n/a
 
      12-13-2004
But I want all the SQL statements to be part of one transaction; if any
one SQL statement fails, I want to rollback the whole transaction.
Correct me please if I am wron, but do you want me to do it this way:

int recordCount = 0;

while ( (lineData = br.readLine()) != null ) {
recordCount ++;
.....
if (recordCount == 2000) {
stmt.executeBatch();
recordCount = 1;
}
stmt.addBatch(insertStatementSQL);
}

if (recordCount > 0) {
stmt.executeBatch();
}

Thanks

 
Reply With Quote
 
srh
Guest
Posts: n/a
 
      12-13-2004
oops... forget to add one line stmt.clearBatch(); in the (recordCount
== 2000) block.

It still failed. On the second 2000 records. Any input?

16:33:47,929 INFO [STDOUT] importTimeCardFile: recordCount = 2000
16:35:03,070 INFO [STDOUT] importTimeCardFile: stmt.executeBatch()
successful
16:35:03,070 INFO [STDOUT] importTimeCardFile: stmt.clearBatch()
successful
16:35:03,632 INFO [STDOUT] importTimeCardFile: recordCount = 2000
16:36:14,210 INFO [STDOUT] importTimeCardFile: SQLException = SQL
Attempt to pr
oduce a ResultSet from executeBatch
16:36:14,210 ERROR [STDERR] sun.jdbc.odbc.JdbcOdbcBatchUpdateException:
SQL Atte
mpt to produce a ResultSet from executeBatch
16:36:14,226 ERROR [STDERR] at
sun.jdbc.odbc.JdbcOdbcStatement.executeBatchU
pdate(JdbcOdbcStatement.java:101
16:36:14,226 ERROR [STDERR] at
sun.jdbc.odbc.JdbcOdbcStatement.executeBatch(
JdbcOdbcStatement.java:912)
16:36:14,226 ERROR [STDERR] at
org.jboss.resource.adapter.jdbc.WrappedStatem
ent.executeBatch(WrappedStatement.java:736)
16:36:14,226 ERROR [STDERR] at
com.pendylum.ejb.TimeCardImportEJBBean.import
TimeCardFile(TimeCardImportEJBBean.java:574)
16:36:14,226 ERROR [STDERR] at
com.pendylum.ejb.TimeCardImportEJBBean.proces
sTimeCardImport(TimeCardImportEJBBean.java:1943)
16:36:14,226 ERROR [STDERR] at
sun.reflect.NativeMethodAccessorImpl.invoke0(
Native Method)
16:36:14,226 ERROR [STDERR] at
sun.reflect.NativeMethodAccessorImpl.invoke(N
ativeMethodAccessorImpl.java:39)
16:36:14,226 ERROR [STDERR] at
sun.reflect.DelegatingMethodAccessorImpl.invo
ke(DelegatingMethodAccessorImpl.java:25)
16:36:14,226 ERROR [STDERR] at
java.lang.reflect.Method.invoke(Method.java:3
24)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.StatelessSessionContainer$Conta
inerInterceptor.invoke(StatelessSessionContainer.j ava:683)
16:36:14,226 ERROR [STDERR] at
org.jboss.resource.connectionmanager.CachedCo
nnectionInterceptor.invoke(CachedConnectionInterce ptor.java:185)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.StatelessSessionInstanc
eInterceptor.invoke(StatelessSessionInstanceInterc eptor.java:72)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.AbstractTxInterceptor.i
nvokeNext(AbstractTxInterceptor.java:84)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.TxInterceptorCMT.runWit
hTransactions(TxInterceptorCMT.java:267)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.TxInterceptorCMT.invoke
(TxInterceptorCMT.java:12
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.SecurityInterceptor.inv
oke(SecurityInterceptor.java:11
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.LogInterceptor.invoke(L
ogInterceptor.java:191)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.plugins.ProxyFactoryFinderInter
ceptor.invoke(ProxyFactoryFinderInterceptor.java:1 22)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.StatelessSessionContainer.inter
nalInvoke(StatelessSessionContainer.java:331)
16:36:14,226 ERROR [STDERR] at
org.jboss.ejb.Container.invoke(Container.java
:700)
16:36:14,226 ERROR [STDERR] at
sun.reflect.GeneratedMethodAccessor63.invoke(
Unknown Source)
16:36:14,226 ERROR [STDERR] at
sun.reflect.DelegatingMethodAccessorImpl.invo
ke(DelegatingMethodAccessorImpl.java:25)
16:36:14,226 ERROR [STDERR] at
java.lang.reflect.Method.invoke(Method.java:3
24)
16:36:14,226 ERROR [STDERR] at
org.jboss.mx.capability.ReflectedMBeanDispatc
her.invoke(ReflectedMBeanDispatcher.java:284)
16:36:14,226 ERROR [STDERR] at
org.jboss.mx.server.MBeanServerImpl.invoke(MB
eanServerImpl.java:546)
16:36:14,226 ERROR [STDERR] at
org.jboss.invocation.local.LocalInvoker.invok
e(LocalInvoker.java:101)
16:36:14,226 ERROR [STDERR] at
org.jboss.invocation.InvokerInterceptor.invok
e(InvokerInterceptor.java:90)
16:36:14,226 ERROR [STDERR] at
org.jboss.proxy.TransactionInterceptor.invoke
(TransactionInterceptor.java:46)
16:36:14,226 ERROR [STDERR] at
org.jboss.proxy.SecurityInterceptor.invoke(Se
curityInterceptor.java:45)
16:36:14,226 ERROR [STDERR] at
org.jboss.proxy.ejb.StatelessSessionIntercept
or.invoke(StatelessSessionInterceptor.java:100)
16:36:14,226 ERROR [STDERR] at
org.jboss.proxy.ClientContainer.invoke(Client
Container.java:85)
16:36:14,226 ERROR [STDERR] at
$Proxy74.processTimeCardImport(Unknown Source
)
16:36:14,226 ERROR [STDERR] at
org.apache.jsp.inputprocess_jsp._jspService(U
nknown Source)
16:36:14,226 ERROR [STDERR] at
org.apache.jasper.runtime.HttpJspBase.service
(HttpJspBase.java:137)
16:36:14,226 ERROR [STDERR] at
javax.servlet.http.HttpServlet.service(HttpSe
rvlet.java:853)
16:36:14,226 ERROR [STDERR] at
org.apache.jasper.servlet.JspServletWrapper.s
ervice(JspServletWrapper.java:210)
16:36:14,226 ERROR [STDERR] at
org.apache.jasper.servlet.JspServlet.serviceJ
spFile(JspServlet.java:295)
16:36:14,226 ERROR [STDERR] at
org.apache.jasper.servlet.JspServlet.service(
JspServlet.java:241)
16:36:14,226 ERROR [STDERR] at
javax.servlet.http.HttpServlet.service(HttpSe
rvlet.java:853)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.ApplicationFilterCha
in.internalDoFilter(ApplicationFilterChain.java:24 7)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.ApplicationFilterCha
in.doFilter(ApplicationFilterChain.java:193)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardWrapperValve
..invoke(StandardWrapperValve.java:256)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:643)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline.inv
oke(StandardPipeline.java:480)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:995)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardContextValve
..invoke(StandardContextValve.java:191)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:643)
16:36:14,226 ERROR [STDERR] at
org.jboss.web.tomcat.security.JBossSecurityMg
rRealm.invoke(JBossSecurityMgrRealm.java:220)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.valves.CertificatesValve.
invoke(CertificatesValve.java:246)
16:36:14,226 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,241 ERROR [STDERR] at
org.jboss.web.tomcat.tc4.statistics.Container
StatsValve.invoke(ContainerStatsValve.java:76)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,241 ERROR [STDERR] at
org.jboss.web.tomcat.session.ClusteredSession
Valve.invoke(ClusteredSessionValve.java:7
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline.inv
oke(StandardPipeline.java:480)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:995)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardContext.invo
ke(StandardContext.java:2417)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardHostValve.in
voke(StandardHostValve.java:180)
16:36:14,241 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:643)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.valves.ErrorDispatcherVal
ve.invoke(ErrorDispatcherValve.java:171)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.valves.ErrorReportValve.i
nvoke(ErrorReportValve.java:172)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,476 ERROR [STDERR] at
org.jboss.web.tomcat.security.SecurityAssocia
tionValve.invoke(SecurityAssociationValve.java:65)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.valves.AccessLogValve.inv
oke(AccessLogValve.java:577)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:641)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline.inv
oke(StandardPipeline.java:480)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:995)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardEngineValve.
invoke(StandardEngineValve.java:174)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline$Sta
ndardPipelineValveContext.invokeNext(StandardPipel ine.java:643)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.StandardPipeline.inv
oke(StandardPipeline.java:480)
16:36:14,476 ERROR [STDERR] at
org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:995)
16:36:14,476 ERROR [STDERR] at
org.apache.coyote.tomcat4.CoyoteAdapter.servi
ce(CoyoteAdapter.java:197)
16:36:14,476 ERROR [STDERR] at
org.apache.coyote.http11.Http11Processor.proc
ess(Http11Processor.java:781)
16:36:14,476 ERROR [STDERR] at
org.apache.coyote.http11.Http11Protocol$Http1
1ConnectionHandler.processConnection(Http11Protoco l.java:549)
16:36:14,476 ERROR [STDERR] at
org.apache.tomcat.util.net.TcpWorkerThread.ru
nIt(PoolTcpEndpoint.java:605)
16:36:14,476 ERROR [STDERR] at
org.apache.tomcat.util.threads.ThreadPool$Con
trolRunnable.run(ThreadPool.java:677)
16:36:14,476 ERROR [STDERR] at
java.lang.Thread.run(Thread.java:534)
16:36:14,476 INFO [STDOUT] processTimeCardImport: closing the
connection....


574 line is where the stmt.executeBatch(); is

 
Reply With Quote
 
Andy Hill
Guest
Posts: n/a
 
      12-13-2004
"srh" <(E-Mail Removed)> wrote:
>But I want all the SQL statements to be part of one transaction; if any
>one SQL statement fails, I want to rollback the whole transaction.
>

So why not make the Connection non-auto-committing, issue your SQL statements
one at a time (instead of in a batch), then do an explicit Connection.commit()
when you know you haven't screwed the pooch? That way you don't have to keep
more than the minimum amount of info around in memory.
 
Reply With Quote
 
booxplode@yahoo.com
Guest
Posts: n/a
 
      12-15-2004

Rizwan wrote:
> I have written a java program (an EJB) to read a file and write it to
> database w.r.t. some rules. The program works ok but when the file

has
> around 6000 rows it fails and give an error message :
>
> 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
> Unexpected Error:
> java.lang.OutOfMemoryError
>
> My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever

2000 on
> Windows 2000/XP
>
>
> Any reason? Thanks


If there's a leak, that's tricker. But I'm
betting in this case you might just need a
bigger heap. Try java -Xmx200m (or something
to that effect.)

--Joe

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments