Error Occuring::java.lang.OutOfMemoryError: Java heap space

Question

I am trying to read a file containing 158000 records(12MB) separted by newline and put that record in the mysql database but after inserting around 49000 records my java program throws following exception

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:315)
    at java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:310)
    at java.util.jar.Manifest.read(Manifest.java:178)
    at java.util.jar.Manifest.<init>(Manifest.java:52)
    at java.util.jar.JarFile.getManifestFromReference(JarFile.java:167)
    at java.util.jar.JarFile.getManifest(JarFile.java:148)
    at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:696) at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:696)

    at java.net.URLClassLoader.defineClass(URLClassLoader.java:228)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:430) at com.mysql.jdbc.Util.handleNewInstance(Util.java:430)

    at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:553)
    at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1378)
    at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4143)
    at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4042)
    at loadcsvdatabase.LoadCsvDatabase.fnLoadCsvIntoMemory(LoadCsvDatabase.java:52)
    at loadcsvdatabase.LoadCsvDatabase.main(LoadCsvDatabase.java:29)

Java Code of inserting the data

FileInputStream file;
        DataInputStream dataIn;
        BufferedReader bReader;
        Connection conn= openConnection();
        String strRecord=new String();
        String[]strSplittedRecord;
        file= new FileInputStream("D:\\Java\\CountryWhois.txt");
        dataIn= new DataInputStream(file);
        bReader= new BufferedReader(new InputStreamReader(dataIn));
        PreparedStatement insertStmt;
        String strQuery="insert into countrywhois values(?,?,?,?,?,?)";
        while((strRecord=bReader.readLine())!=null)
        {

            strSplittedRecord=strRecord.split(",");

         try {

             insertStmt=conn.prepareStatement(strQuery);
             insertStmt.setString(1,strSplittedRecord[0].substring(1, strSplittedRecord[0].length()-1));
             insertStmt.setString(2,strSplittedRecord[1].substring(1, strSplittedRecord[1].length()-1));
             insertStmt.setString(3,strSplittedRecord[2].substring(1, strSplittedRecord[2].length()-1));
             insertStmt.setString(4,strSplittedRecord[3].substring(1, strSplittedRecord[3].length()-1));
             insertStmt.setString(5,strSplittedRecord[4].substring(1, strSplittedRecord[4].length()-1));
             insertStmt.setString(6,strSplittedRecord[5].substring(1, strSplittedRecord[5].length()-1));
             int nResultInsert=insertStmt.executeUpdate();
             if(nResultInsert!=0)
             {
                System.out.println("Inserted 1");


             }
             else
             {
                if(conn!=null)
                {
                    conn.close();
                    System.out.println("Disconnected from database");
                }
             }
         }
         catch (Exception e) 
         {
             conn.close();
             e.printStackTrace(System.out);        
         }

        }

it Throws Exception at Line insertStmt=conn.prepareStatement(strQuery);

Please suggest me why my program is running out of memory exception..

Torben · Accepted Answer · 2012-01-12 06:30:59Z

7

You are creating a new PreparedStatement every time you insert a record. That is not how PreparedStatements are supposed to be used. Also, you never close the created PreparedStatements, which probably causes the OutOfMemoryException, but that is not the root problem.

You must create only one PreparedStatement and reuse that. Then, after inserting all records, close the PreparedStatement and the Connection.

answered Jan 12, 2012 at 6:30

Torben

3,91731 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Christian Kuetbach Over a year ago

That is the point additionally addBatch may improve the performance.

Augustus Thoo Over a year ago

I second ckuetbach. But first Nishit have to fix the OOM problem first before optimizing. Also, due to the large size of records, he would need to limit addBatch calls to 100 (IMO) for every 1 executeBatch() call.

Nishit Jain Over a year ago

Thanks guys!! That was the mistake i was doing by placing the prepared statement inside while loop

mohdajami · Accepted Answer · 2012-01-12 06:35:08Z

0

insertStmt=conn.prepareStatement(strQuery);

Should be outside the while loop. Put it exactly after initializing strQuery

answered Jan 12, 2012 at 6:35

mohdajami

9,7103 gold badges35 silver badges54 bronze badges

Comments

Augustus Thoo · Accepted Answer · 2012-01-12 06:59:41Z

You should revise your code to follow this way:

...

try {
  Connection conn = openConnection();
  try {

    String strQuery="insert into countrywhois values(?,?,?,?,?,?)";
    PreparedStatement insertStmt=conn.prepareStatement(strQuery);
    while (...) {
        try {
            ....
            int nResultInsert=insertStmt.executeUpdate();

            // handle your result here (e.g. print/log)
            // DO NOT close() here, continue with loop

        } catch (Exception e) {
            // handle your exception here (e.g. print/log)
            // DO NOT close() here, continue with loop, OR break loop
        }
    }

  } finally {
      // closing SQL Connection no matter what happens to your while-loop
      conn.close();
  }
} catch (Exception e) {
    // handle your exception here (e.g. print/log)
}

Collectives™ on Stack Overflow

Error Occuring::java.lang.OutOfMemoryError: Java heap space

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related