As far as I know, the BinaryReader loop performance is poor. Another method I can think of is to first ReadAllBytes and then Buffer.BlockCopy into int[], but that would result in an additional copy. Is it possible to read a huge binary file directly into int[] efficiently?
2 Answers
You can use MemoryMarshal.AsBytes to read all data:
using var stream = new FileStream(...);
var target = new int[stream.Length / 4];
stream.Read(MemoryMarshal.AsBytes(target.AsSpan()));
No BinaryReader is used in that case. Be aware of endianness of int representation. This code above might cause problems if the file doesn't match to your hardware.
2 Comments
埃博拉酱
This is perfect as long as the stream supports the Length property - and things become difficult if it does not …
Matthew Watson
@埃博拉酱 If the stream does not support
Length then at some point the code will have to loop. And it won't be possible to pre-size the result buffer, which means that some data copying will be required.If you want to read the array as another type, you can use MemoryMarshal.Cast.
using System;
using System.Runtime.InteropServices;
class Program
{
public static void Main(string[] args)
{
byte[] arrayByte = { 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88 };
Span<int> spanInt = MemoryMarshal.Cast<byte, int>(arrayByte);
Console.WriteLine("0x{0:X8}", spanInt[0]); // For little endian it will be 0x44332211.
Console.WriteLine("0x{0:X8}", spanInt[1]); // For little endian it will be 0x88776655.
}
}
Another alternative is Unsafe.As. However, there are some problems, such as Length not reflecting the converted type value. I recommend using the MemoryMarshal class.
using System;
using System.Runtime.CompilerServices;
class Program
{
public static void Main(string[] args)
{
byte[] arrayByte = { 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88 };
int[] arrayInt = Unsafe.As<byte[], int[]>(ref arrayByte);
Console.WriteLine("Length={0}", arrayInt.Length); // 8!? omg...
Console.WriteLine("0x{0:X8}", arrayInt[0]); // For little endian it will be 0x44332211.
Console.WriteLine("0x{0:X8}", arrayInt[1]); // For little endian it will be 0x88776655.
}
}
4 Comments
Mario Vernari
...as far the endianness in the file matches those of the machine. In my old code still have a int composition made by OR-ing the bytes.
radian
OK, Added endian comment.
埃博拉酱
MemoryMarshal.Cast returns a Span, which doesn't implement IEnumerable, which often leads to me having to ToArray it again, resulting in an extra copy. Instead, after Unsafe.As, you can append a Take to correct the array length error, and it returns IEnumerable, and no copy occurs.
埃博拉酱
Another problem is that Unsafe.As seems not working on Android. It's simply converting the bytes into ints.
ReadAllBytesand either use bytes or useEnumerable.Castto convert to int?