I have a *ushort that points to an array of dimensions x * y of 16 bit values, I would like to be able to write that to disk without copying to managed space, is that possible?
8
-
You need to give more detail. Like how/why you have an unsafe data structure. If you wrote the original code. What you've already tried etc.Jan Bannister– Jan Bannister2010-01-21 15:33:54 +00:00Commented Jan 21, 2010 at 15:33
-
This is an image array of unsigned 16 bit ints. It is created from an C++ dll which I am interoping with. There is a function that will return an array of pointers to each frame, I need to be able to write out a frame as a file. Often this needs to happen very quickly, several times a second so speed is important. This is my first time here so forgive the lack of detail, I will get better over time.GMAN– GMAN2010-01-21 15:37:56 +00:00Commented Jan 21, 2010 at 15:37
-
Are you sure copying from unmanaged to managed memory would be the bottleneck of your application? Hard disk drives are still magnitudes slower than RAM, i.e. my guess is your application will spend more time waiting for the hardware than on copying some bytes in memory.dtb– dtb2010-01-21 15:41:23 +00:00Commented Jan 21, 2010 at 15:41
-
I am not sure, I am still a novice at this complicated subject, and was hoping that I could do some research before trying the various options. These frames can be quite large, currently 3 Mb per frame, but can be as large as 18 Mb per frame. I was just thinking if was already in memory why copy it into the managed heap just to write it to disk.GMAN– GMAN2010-01-21 15:50:32 +00:00Commented Jan 21, 2010 at 15:50
-
Is the memory allocated by the C++ dll? If not, it might be even better to allocate managed memory and pass it to the dll, than to allocate unmanaged memory and copy it to managed memory or write it to disk using P/Invoke.dtb– dtb2010-01-21 15:52:14 +00:00Commented Jan 21, 2010 at 15:52
|
Show 3 more comments
2 Answers
Try this - I've created an extension method on Stream that will write a pointer to shorts. It has the downside of allocating and copying the memory and the major added downside of huge chunks of unsafe code. It does, however, work as you would expect. I set up a managed array and then got a pointer to it to simulate getting an ushort * from unmanaged code.
using System;
using System.IO;
using System.Runtime.InteropServices;
namespace UnsafeWriting
{
class Program
{
static void Main(string[] args)
{
ushort[] someArray = new ushort[20];
for (int i = 0; i < 20; i++) { someArray[i] = (ushort)i; }
using (FileStream stm = new FileStream("output.bin", FileMode.Create))
{
unsafe
{
fixed (ushort* dataPtr = &someArray[0])
{
stm.Write(dataPtr, someArray.Length);
}
}
}
}
}
public static class Extensions
{
public static unsafe void Write(this Stream stm, ushort* data, int count)
{
count *= 2;
byte[] arr = new byte[count];
Marshal.Copy(new IntPtr(data), arr, 0, count);
stm.Write(arr, 0, count);
}
}
}
If you don't like the allocate and copy, then you can do this instead:
public static unsafe void Write(this Stream stm, ushort* data, int count)
{
count *= 2;
byte* b = (byte*)data;
for (int i = 0; i < count; i++)
{
stm.WriteByte(*b++);
}
}
which also works as expected, except in this case you're adding the overhead of a method invocation for every byte. The underlying file system streams buffer, so it won't be too much overhead, but it's still there.
Don't ask me what would happen on a big endian processor (are there any that run .NET?) in either case.
EDIT - for grins I ran FileStream through Reflector to look at the underlying code and FileStream itself buffers both the Write() and WriteByte() calls, so I see no reason to use anything but the WriteByte version. The buffersize that is hard-coded into FileStream is 4K.
3 Comments
radbyx
I tryed your code sir. It's first time I try anything with unsafe. The error I get is, with the 'unsafe' keyword, saying "Cannot use unsafe construct in safe context" What should I do?
radbyx
Oh I found the solution. I had to check the "Allow unsafe code" in the property of the project
GMAN
this looks like the best solution, these can be very large arrays, 1024 x 1024 x ushort x 128 frames = too much to reallocate if it is not necessary. Thanks for the excellent solution
You could use WriteFile using P/Invoke, but why make the simple complicated?
A simple solution would be to copy the frames to managed memory using, for example, Marshal.Copy, and then write it to a file with a FileStream.
