Why does a large array constructor call break the Scala compiler?

Question

I'm doing some image processing for a job and was scripting things with Im4Java. In order to write some unit cases I decided to just store a local image as a byte array in the code. I grabbed a simple test image

And then converted it to the byte array like so:

import java.nio.file.{Paths, Files}
import java.nio.charset.StandardCharsets
val stuff =scala.io.Source.fromFile(fileName)(scala.io.Codec.ISO8859).map(_.toByte).toArray
Files.write(Paths.get("tmp.txt"), stuff.mkString(",").getBytes(StandardCharsets.UTF_8))

And then put those bytes into an Array[Byte] apply constructor:

class testCase() {
    val smallFile = Array[Byte](-1,-40,-1, …) // 9816 arguments to the constructor passed here
}

When I tried to compile this (see the full scala file in this gist) I was surprised to see it break the compiler's typer! I get the following stacktrace (I've truncated it a bit):

[error] uncaught exception during compilation: java.lang.StackOverflowError
java.lang.StackOverflowError
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1241)
at scala.reflect.internal.Symbols$Symbol.baseTypeSeqLength$1(Symbols.scala:1628)
at scala.reflect.internal.Symbols$Symbol.isLess(Symbols.scala:1631)
at scala.reflect.internal.Types$Type.baseTypeIndex(Types.scala:992)
at scala.reflect.internal.Types$CompoundType.baseType(Types.scala:1655)
at scala.reflect.internal.Types$ClassTypeRef$class.baseType(Types.scala:2186)
at scala.reflect.internal.Types$TypeRef$$anon$6.baseType(Types.scala:2544)
at scala.reflect.internal.Types$class.firstTry$1(Types.scala:6064)
at scala.reflect.internal.Types$class.isSubType2(Types.scala:6228)
at scala.reflect.internal.Types$class.isSubType(Types.scala:5837)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:13)
at scala.reflect.internal.Types$class.fourthTry$1(Types.scala:6223)
at scala.reflect.internal.Types$class.thirdTryRef$1(Types.scala:6123)
at scala.reflect.internal.Types$class.thirdTry$1(Types.scala:6145)
at scala.reflect.internal.Types$class.secondTry$1(Types.scala:6109)
at scala.reflect.internal.Types$class.firstTry$1(Types.scala:6070)
at scala.reflect.internal.Types$class.isSubType2(Types.scala:6228)
at scala.reflect.internal.Types$class.isSubType(Types.scala:5837)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:13)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:872)
at scala.tools.nsc.typechecker.Typers$Typer.adapt(Typers.scala:1091)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5660)
at scala.tools.nsc.typechecker.Typers$Typer.typedArg(Typers.scala:3042)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3069)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3071)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3071)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3071)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3071)
at scala.tools.nsc.typechecker.Typers$Typer.loop$1(Typers.scala:3071)
(etc as this is where the stackoverflow happens)

I'm using the Scala 2.10.5 compiler and java jdk1.7.0_79

I'm probably going to just use a smaller image or something to fix my issue, but I would like to know why the compiler does this and if it can be fixed?

Normally, the image itself can't be the problem, since the compiler won't know about it at compile-time. — Clashsoft
– Clashsoft, Commented Jul 22, 2015 at 19:10
I doubt it has something to do with the actual image since error is thrown during compile time. Can you provide full version of code (it's unclear how fileName, Files and Paths are defined)? — om-nom-nom
– om-nom-nom, Commented Jul 22, 2015 at 19:11
@Clashsoft there is no actual problem with the image. I just provided the code so that someone could reproduce the byte array I was using in the gist file if they wanted to in the same way. — EdgeCaseBerg
– EdgeCaseBerg, Commented Jul 22, 2015 at 19:12
Maybe you hit the "64K bytecode per method" limit the JVM has? Or, well, you're passing a bit over 9000 arguments to that constructor; some sources seem to say there's a limit at 255. — lynn
– lynn, Commented Jul 22, 2015 at 19:12

Community · Accepted Answer · 2017-05-23 12:34:19Z

7

Well I tried to reproduce your problem and newer scalac (2.11.7) has a better error message (hope it clears up the problem):

» scalac Arrays.scala
Arrays.scala:1: error: Could not write class testCase because it exceeds JVM code size limits. Method <init>'s code too large!
class testCase()  {
      ^
one error found

So it looks like, just like @Marius noted in comments, you're hitting "64K bytecode per method" limit imposed by JVM.

edited May 23, 2017 at 12:34

CommunityBot

11 silver badge

answered Jul 22, 2015 at 19:15

om-nom-nom

62.9k13 gold badges186 silver badges231 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

EdgeCaseBerg Over a year ago

So it's looking like an answer to how big anything can be depends on those JVM settings? Which look like some of the limits are reference in this question: stackoverflow.com/questions/17422480/…

EdgeCaseBerg Over a year ago

Thanks @om-nom-nom this answers the question, but for clarity of an answer for anyone else who runs into this, could you edit your answer to explicitly state that it's a JVM code size limit as well as showing the error from the compiler?

Collectives™ on Stack Overflow

Why does a large array constructor call break the Scala compiler?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related