4

I am trying to run simple program available on this website https://stanfordnlp.github.io/CoreNLP/api.html
My Program

import java.io.BufferedReader;  
import java.io.BufferedWriter;  
import java.io.FileNotFoundException;  
import java.io.FileReader;  
import java.io.FileWriter;  
import java.io.IOException;  
import java.io.PrintWriter;  
import java.util.List;  
import java.util.Properties;  

import edu.stanford.nlp.ling.CoreAnnotations.NamedEntityTagAnnotation;  
import edu.stanford.nlp.ling.CoreAnnotations.PartOfSpeechAnnotation;  
import edu.stanford.nlp.ling.CoreAnnotations.SentencesAnnotation;  
import edu.stanford.nlp.ling.CoreAnnotations.TextAnnotation;  
import edu.stanford.nlp.ling.CoreAnnotations.TokensAnnotation;  
import edu.stanford.nlp.ling.CoreLabel;  
import edu.stanford.nlp.pipeline.Annotation;  
import edu.stanford.nlp.pipeline.StanfordCoreNLP;  
import edu.stanford.nlp.util.CoreMap;  

public class StanfordClass {

    public static void main(String[] args) throws Exception {
     Properties props = new Properties();
      props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse");

        StanfordCoreNLP pipeline = new StanfordCoreNLP(props);

        String text = "What is the Weather in Mumbai right now?";
         Annotation document = new Annotation(text);
          pipeline.annotate(document);

        List<CoreMap> sentences = document.get(SentencesAnnotation.class);

       for(CoreMap sentence: sentences) {
          // traversing the words in the current sentence
          // a CoreLabel is a CoreMap with additional token-specific methods
          for (CoreLabel token: sentence.get(TokensAnnotation.class)) {
            // this is the text of the token
            String word = token.get(TextAnnotation.class);
            // this is the POS tag of the token
            String pos = token.get(PartOfSpeechAnnotation.class);
            // this is the NER label of the token
            String ne = token.get(NamedEntityTagAnnotation.class);

            System.out.println(String.format("Print: word: [%s] pos: [%s] ne: [%s]",word, pos, ne));
          }
        }
    }
}  

But getting Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

What I tried
1. if I remove ner (named entity recognizer) property from above code i.e. props.setProperty("annotators", "tokenize, ssplit, pos, lemma, parse");
then the code runs fine.
2.but I required ner(named entity recognizer) hence I increase heap size in eclipse.ini file up to 1g and sure that this much size is far enough for this program and also sure that heap size is not the problem in this case. I think something is missing but not getting that.

5
  • did you check the methods which are taking more memory? Commented Dec 26, 2017 at 6:19
  • how should I check? Commented Dec 26, 2017 at 6:25
  • this applicable to up to Java 5 but I am using java 8 and heap size up to 1gb is too much for this simple program Commented Dec 26, 2017 at 6:31
  • Oh I am sorry, i forgot use this khelekore.org/jmp/tijmp Commented Dec 26, 2017 at 6:34
  • I have used eclipse memory analyzer and there is no problem related to memory Commented Dec 26, 2017 at 6:38

1 Answer 1

1

After lots of searches gets answer here Using Stanford CoreNLP

Use following answer:-
1.Windows -> Preferences
2.Java -> Installed JREs
3.Select the JRE and click on Edit
4.On the default VM arguments field, type in "-Xmx1024M". (or your memory preference, for 1GB of ram it is 1024)
5.Click on finish or OK.

Sign up to request clarification or add additional context in comments.

1 Comment

I had to set the heap space to 4 GB before it worked. i.e. "-Xmx4096M" But then, it did! :) Watching Eclipse in Task Manager (Windows 10), the memory footprint went up to 5.9 GB just before the demo program finished. No surprise, Natural Language Processing requires a lot of RAM.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.