ASP.Net Web API doesn't read all bytes from StreamContent

Question

I have an ASP.Net Web API set up on my website that is used to communicated with a WPF desktop application. I have an action setup on the API to receive binary files from the client application. However in some (seemingly random) cases when I get all the bytes from the request not all the bytes are read. Hopefully you can give me an idea of how to do this in a way that will work all of the time. Here's the code:

Client Side:

public static SubmitTurnResult SubmitTurn(int turnId, Stream fileStream)
{
    HttpClient client = CreateHttpClient();

    HttpContent content = new StreamContent(fileStream);
    content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
    content.Headers.ContentDisposition.FileName = "new-turn.Civ5Save";
    content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    content.Headers.ContentLength = fileStream.Length;

    HttpResponseMessage response = client.PostAsync(
        string.Format("SubmitTurn?authKey={0}&turnId={1}",
                        LocalSettings.Instance.AuthenticationKey,
                        turnId
                        ),
        content
    ).Result;

    response.EnsureSuccessStatusCode();

    return response.Content.ReadAsAsync<SubmitTurnResult>().Result;
}

SubmitTurnResult is an enum that defines the result on the server, turnId is the ID for the entity this file is attached to, and fileStream is an actual FileStream reading the bytes of disk.

Server Side:

[HttpGet, HttpPost]
public SubmitTurnResult SubmitTurn(string authKey, int turnId)
{

    try
    {
        bool worked = false;
        int gameId = 0;

        using (GmrEntities gmrDb = new GmrEntities())
        {
            var player = gmrDb.Users.FirstOrDefault(u => u.AuthKey == authKey);
            if (player != null)
            {
                var turn = player.Turns.FirstOrDefault(t => t.TurnID == turnId);
                if (turn != null)
                {
                    byte[] saveFileBytes = null;

                    using (MemoryStream tempStream = new MemoryStream())
                    {
                        var task = this.Request.Content.CopyToAsync(tempStream);
                        task.Wait();

                        saveFileBytes = tempStream.ToArray();
                        tempStream.Close();
                    }

                    if (saveFileBytes.Length != this.Request.Content.Headers.ContentLength.Value)
                    {
                        throw new Exception(string.Format("Byte array length ({0}) not equal to HTTP content-length header ({1}). This is not good!",
                                    saveFileBytes.Length, this.Request.Content.Headers.ContentLength.Value));
                    }

                    worked = GameManager.SubmitTurn(turn, saveFileBytes, gmrDb);

                    if (worked)
                    {
                        gameId = turn.Game.GameID;

                        gmrDb.SaveChanges();
                    }
                }
            }
        }


        return SubmitTurnResult.OK;
    }
    catch (Exception exc)
    {
        DebugLogger.WriteExceptionWithComments(exc, string.Format("Diplomacy: Sumbitting turn for turnId: {0}", turnId));

        return SubmitTurnResult.UnexpectedError;
    }
}

Was this on 32-bit Windows Server 2003 or Windows XP? We're running into this same behavior with StreamContent, but when streaming a response from a Windows Server 2003 Web API service. It doesn't repro on 2008. Also, we've found that this only repros with a FileStream. Converting the FileStream to a MemoryStream bypasses the issue (at the expense of memory of course). We've found that when the response stream terminates early, it's always on a 4096-byte boundary, and it hits a cap at around 3.5MB. — Oran Dennison
– Oran Dennison, Commented Nov 8, 2013 at 0:28

Oran Dennison · Accepted Answer · 2013-11-15 19:59:03Z

11

As noted in my previous comment, we ran into this same behavior with StreamContent, but when streaming a response from a Windows Server 2003 Web API service. ~~It doesn't repro on 2008.~~ Actually, it also repros on Windows Server 2008 if I configure the VM with a small amount of RAM (712 MB), but with 4 GB of RAM it doesn't repro. Also, we found that this only repros with a FileStream. Converting the FileStream to a MemoryStream bypasses the issue (at the expense of memory of course). We found that when the response stream terminates early, it's always on a 4096-byte boundary, and it hits a cap at around 3.5MB.

Here's the workaround that fixed things for me, tailored to your code example:

public static SubmitTurnResult SubmitTurn(int turnId, Stream fileStream)
{
    HttpClient client = CreateHttpClient();

    var memoryStream = new MemoryStream((int)fileStream.Length);
    fileStream.CopyTo(memoryStream);
    fileStream.Close();
    memoryStream.Seek(0, SeekOrigin.Begin);
    HttpContent content = new StreamContent(memoryStream);

If desired, you can conditionally do the MemoryStream copy only when Stream is a FileStream.

edited Nov 15, 2013 at 19:59

answered Nov 8, 2013 at 19:44

Oran Dennison

3,3271 gold badge31 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mel Green Over a year ago

Thanks for your suggestion, unfortunately it didn't resolve the issue for me. The client portion of the code is running on hundreds of machines from WinXP to Win8.1 and I have changed it to use a MemoryStream instead of a FileStream. The server portion of it is running on WinServer 2012R2. I still experience the issue now and then where the amount of bytes read from the content stream by the server doesn't match the amount of bytes sent by the client =\

Oran Dennison Over a year ago

That's really unfortunate, and doesn't bode well for my "solution" that seems to be working for me so far. The best that can be done in your case is to build client-side retry logic if the HTTP response isn't successful. Or file a bug report with Microsoft, ideally with a test case that can repro the issue, probably by running repeatedly in a loop. If you have something that repros it, you can call up Microsoft Product Support Services, give them your credit card, and they won't charge it if they confirm that it really is a bug on their end. It's time-consuming, but it works. Good luck!

Collectives™ on Stack Overflow

ASP.Net Web API doesn't read all bytes from StreamContent

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related