1

I have ask similar question before : Java String Memory Leak

But I was not sure what to ask:

Here is another piece of code I wrote:

import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.InputStreamReader;


public class TestString {
    public static int  readLineXX(BufferedReader br) throws Exception {
            String s = br.readLine();
            if ( s == null ) return 0;
            return 1;
    }
    public static void main(String args[]) {
        while (true) {
        try {
            FileInputStream fstream = new FileInputStream("bigfile.txt");
            DataInputStream in = new DataInputStream(fstream);
            BufferedReader  br = new BufferedReader(new InputStreamReader(in));
            while ( readLineXX (br)!= 0) {
                //System.out.print(".");
            }
            br.close();
            in.close();
            fstream.close();
        } catch (Exception e) {
        }       
    }
    }
}

Since the String in Java are Immutable, will the String s be garbage collected. I ran this code for long time using -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails options, It is running fine.

This is small snippet code of my main application where I think is leak and causing OOM. bigfile.txt is around 1GB.

Improve this question
2
  • 4
    If you get an OutOfMemoryError in your main application, but you don't get one when you run this, what makes you think it's this part of your app that causes the error? You'll diagnose the problem much more easily if you can reproduce it in a small program. Since you're not reproducing the error, I suspect there's a crucial difference between this code and your app. Commented May 20, 2011 at 21:26
  • I'd make your return from readLineXX read "return s == null ? 0 : 1;". To answer the question though, if this isn't leaking then your problem is elsewhere. We can't theorize what is possibly wrong with your code. We'd write something the size of Core Java.... Commented May 20, 2011 at 22:51

4 Answers 4

Reset to default
4

Yes, s will be garbage collected. Being immutable does not prevent it from being garbage collected, it just prevents it from being modified.

Note that br.readlLine() will potentially use a large amount of memory if the line is very large. For example, if your text file is 1gb without any line breaks, then it is quite possible for br.readLine() to consume 2gb, 1gb for the data that is read in, and then 1gb for the created string.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

The Strings in your code can't cause memory leak, as they aren't referenced anywhere after returning from readLineXX, so all will get garbage collected. My suspects are rather the streams you are using. I can well imagine that for a 1GB input file the input streams can get pretty big.

One way to test this would be to cut your input file into 2 or more parts, and read each of them in turn. If the culprits are the streams, reading several smaller input files one by one should not cause a memory leak.

Improve this answer

4 Comments

In my main application I am creating a list<String> after reading a file line by line, and then doing some operation, I guess I have to look there. I thought s get never deleted.
@Avinash - so, you are adding 1Gb of Strings to a List and running out of memory?
If you REALLY need to make a list from a 1 GB file you probably need to rethink your design. Sure, you can increase the size of your java heap size with a -X flag, but then you'll get a 16 GB file and be back in the same boat. Anyhow, try increasing the heap size. hausheer.osola.com/docs/5
Here are the common errors with using -X to increase the heap size: javahowto.blogspot.com/2006/06/…
1

instead of 

        FileInputStream fstream = new FileInputStream("bigfile.txt");
        DataInputStream in = new DataInputStream(fstream);
        BufferedReader  br = new BufferedReader(new InputStreamReader(in));

you could use 

        BufferedReader  br = new BufferedReader(new FileReader("bigfile.txt"));

The only way to cause an OOME is having very long lines and limited memory. Are you using -Xmx options? Why are you using DataInputStream - is this binary or text data?

Improve this answer

3 Comments

True, although your code has a subtle bug in it: you are not specifying encoding to use, so you will get whatever arbitrary decoder platform has. So you should specify encoding as argument to FileReader (or if there isn't such constructor, use FileInputStream with InputStreamReader).
@StaxMan: correct, you should specify the encoding, but this would change the original code's behavior.
True. I assume original had the bug, so you are just pointing out other fixes. Fair enough.
0

There are no obvious memory leaks in the code.

Dump the heap with the -XX:+HeapDumpOnOutOfMemoryError JVM argument and analyse the results. (I'm assuming a Java 6 Hotspot JVM; if you're using another, there will likely be similar options.)

The DataInputStream is redundant; streams should be closed in finally blocks; but I don't see how such things could influence the errors you are reporting.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.