OutOfMemoryError in StringBuilder.append() even with more than enough memory

Question

I get an OutOfMemoryError from java.lang.StringBuilder.append even when I set the memory size to 32 GB and reading only a 500 MB file into the builder.

java.lang.OutOfMemoryError: null
    at java.base/java.lang.AbstractStringBuilder.hugeCapacity(AbstractStringBuilder.java:214) ~[na:na]
    at java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:206) ~[na:na]
    at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:173) ~[na:na]
    at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538) ~[na:na]
    at java.base/java.lang.StringBuilder.append(StringBuilder.java:174) ~[na:na]
    at com.github.loa.indexer.service.index.base64.Base64Encoder.encode(Base64Encoder.java:31) ~[classes/:na]

It would be great do add code snippet to get more context as to what is happening. — Albatross
– Albatross, Commented May 30, 2019 at 19:02
My use case is quite complex. Trying to stream a file into byte arrays in 3072-byte long chunks, then encode the chunks with java.util.Base64, append the result of the encoding to a StringBuilder. Sadly the StringBuilder dies quickly. I need to get the whole base64 string into the memory to send it to Elasticsearch (streaming is not an option). The code is available here: github.com/bottomless-archive-project/library-of-alexandria/… — Lakatos Gyula
– Lakatos Gyula, Commented May 30, 2019 at 19:16
@Andreas Java 11. I think it specific to Java 9+ but not sure. — Lakatos Gyula
– Lakatos Gyula, Commented May 30, 2019 at 20:07
Kinda sloppy that they throw an OOM. Now I need to catch it somewhere in my code and I know, catching OOMs are quite bad. — Lakatos Gyula
– Lakatos Gyula, Commented May 30, 2019 at 20:09

Lakatos Gyula · Accepted Answer · 2019-05-30 18:57:41Z

5

This happens because the byte array that the StringBuilder want to allocate would require a larger array size than allowed by the JVM (more than 2147483647 aka Integer.MAX_VALUE elements).

answered May 30, 2019 at 18:57

Lakatos Gyula

4,1907 gold badges38 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Andreas Over a year ago

Said another way, it's a limit of Java that strings cannot exceed 2 GB in size, which generally means that it cannot exceed 1 billion characters (latin1-only strings can be 2 billion characters in later versions of Java).

Collectives™ on Stack Overflow

OutOfMemoryError in StringBuilder.append() even with more than enough memory

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related