Creating byte array without initializing with 0s [duplicate]

Question

For a network related framework I need a lot of byte[] buffers to read and write data. When creating a new byte array, the CLR will initialize all values with 0. For buffers used with streams, this seems to be unnecessary overhead:

var buffer = new byte[65536];

var read = await stream.ReadAsync(buffer, 0, buffer.Length);

Is there a way to create a byte[] array without initializing all values with 0 in C#? Probably by invoking a malloc style method? I'm sure this question has been answered, but I didn't find any clues to start with.

No there isn't. What you could do is use a pool of buffers, so the overhead only occurs once. You will then need to ensure that your logic handles the fact that the array may have 'old' data in it. — mjwills
– mjwills, Commented Apr 1, 2019 at 7:43
While reusing arrays is a good start here, I suspect you may also want to look into "pipelines", which would give you all the advantages of ArrayPool<byte>, but also lots more, including a better consumption model, and the ability to use non-contiguous buffers; pipelines is the API that underpins the impressive Kestrel performance. You say "network related framework" - in that case, Pipelines.Sockets.Unofficial (nuget) has a Socket<=>Pipe bridge for direct consumption, and a Stream<=>Pipe bridge if you need intermediaries (for example, SslStream) — Marc Gravell
– Marc Gravell, Commented Apr 1, 2019 at 8:08
@MarcGravell: Thank you very much! Especially the PositionOf and Advance of the API seem to be the dream when it comes to tokenizing / parsing requests. I'll have a look at pipelines when implementing HTTP/2. — Gene
– Gene, Commented Apr 1, 2019 at 8:29
@Gene note: HTTP/2 (client and server) is being added as part of .NET Core 3; if you want more details / examples of using pipelines, I have blogged about it extensively — Marc Gravell
– Marc Gravell, Commented Apr 1, 2019 at 9:50
@MarcGravell: Well, the idea of the project is to implement the HTTP protocol myself, mainly for educational reasons. But I will certainly look at the interfaces there. It's still a long road till 7 million requests per second :) — Gene
– Gene, Commented Apr 1, 2019 at 10:23

Gene · Accepted Answer · 2019-04-01 07:57:34Z

2

Thanks to mjwills link, I stumbled upon the ArrayPool<T> of System.Buffers:

static void Main(string[] args)
{
    var pool = ArrayPool<byte>.Create();

    var watch = new Stopwatch();
    watch.Start();

    Parallel.For(0, 1000000, (i) =>
    {
        //DoSomethingWithBuffers();
        DoSomethingWithPooledBuffers(pool);
    });

    Console.WriteLine(watch.ElapsedMilliseconds);
}

private static int DoSomethingWithBuffers()
{
    var buffer = new byte[65536];
    return buffer.Length;
}

private static int DoSomethingWithPooledBuffers(ArrayPool<byte> pool)
{
    var buffer = pool.Rent(65536);

    var length = buffer.Length;

    pool.Return(buffer);

    return length;
}

Which makes quite a difference (release mode):

DoSomethingWithBuffers: 3264ms
DoSomethingWithPooledBuffers: 470ms

answered Apr 1, 2019 at 7:57

Gene

4,2425 gold badges35 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Creating byte array without initializing with 0s [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related