The problem

I am implementing an out-of-process C# REPL for a Unity game. I tried an in-process REPL at first, but couldn't get it to work because of some weird VTable related exceptions when loading Roslyn assemblies. So, the game sends the code to execute to a backend process written with .NET 9 which compiles it and sends back an assembly which the game then loads and executes. I decided to use the Microsoft.CodeAnalysis.CSharp.Scripting library at the back-end. Here is the simplified code:

(EmitResult result, byte[]? assemblyBytes) Compile(string code, string[] assemblyPaths)
{
    var options = ScriptOptions.Default
        .AddReferences(assemblyPaths)
        .AddImports("System");
    var script = CSharpScript.Create(code, options);
    var compilation = script.GetCompilation();
    var assemblyStream = new MemoryStream();
    var emitResult = compilation.Emit(assemblyStream);
    return (emitResult, emitResult.Success ? assemblyStream.ToArray() : null);
}

This method successfully compiles a line of code send from the game, but it produces an assembly which depends on .NET 9, so the game fails to load it. How do I specify that it must be compiled against .NET Standard 2.1?

What I tried

1. Replace the default assemblies with game assemblies

    var options = ScriptOptions.Default
        .WithReferences(assemblyPaths);

Initially I added a filtered set of assemblies to the default set, so there are no duplicates. In this attempt I replaced the default set with all game assemblies. This produced a lot of errors like Predefined type 'System.Object' is not defined or imported.

I found that compilation.References still contains System.Private.CoreLib and tried to remove that:

    compilation = compilation.RemoveReferences(compilation.References.Where(reference => reference.Display?.Contains("System.Private.CoreLib") ?? false));

Then it produced the error Cannot implicitly convert type 'int' to 'System.Object' when the code is "2+2".

2. Replace the default assemblies with ones defined by the Basic.Reference.Assemblies.NetStandard21 library.

    var options = ScriptOptions.Default
        .WithReferences(NetStandard21.References.All)
        .AddReferences(assemblyPaths)
        .AddImports("System");

I thought that the problem of (1) was that I compiled against runtime assemblies rather than reference assemblies, so I tried to give the reference assemblies to the compiler.

By description of the library it is exactly what I need, but it produced an exception during compilation: Index was outside the bounds of the array.

Either the library is buggy (I created an issue) or I am using it wrong way.

3. Use the Microsoft.Extensions.DependencyModel library to determine reference assemblies

Basically the same as (2), but more involved and does not throw an exception. I followed these steps to configure a .NET Standard 2.1 class library project to output the reference assemblies and copied the produced deps.json file and refs directory to my back-end directory, then the following code successfully resolved the reference assemblies:

var options = ScriptOptions.Default
    .WithReferences(DependencyContext.Load(typeof(ClassFromNetStandardLibrary).Assembly)!.CompileLibraries.SelectMany(cl => cl.ResolveReferencePaths()))

But the emitResult contained the same errors as in (1). I tried to remove the reference to System.Private.CoreLib as well, but still had the other error from (1).

Further investigation

I inspected the compilation.ScriptClass with the debugger and found that its methods return type references System.Object from System.Private.CoreLib rather than netstandard, hence the type conversion errors. Is there a way to replace that with proper object type?