System.OutOfMemoryException was thrown at converting base64 string to byte array while processing multiple files

Question

I am trying to send batch of files along with various other form elements over asynchronous XMLHTTPRequest. Each batch can contain file size up to 15 MB. There can be multiple batches.

I am reading the files in a javascript and converting them into base64 string and then trying to receive the files in the controller (using for loop) and converting them into byte array.

string fileValue1 = form.GetValues("fileName1");
string fileValue2 = form.GetValues("fileName2");

The file value is of the format as below, depending upon the type of the attachment:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMgAAABkC...

Using the below substring to remove the content before ',' and doing some processing before converting it into byte array.

fileValue1 = fileValue1 .Substring(fileValue1 .IndexOf(',') + 1);

fileValue1 = fileValue1 .Trim().Replace(" ", "+");

fileValue1 = Convert.FromBase64String(fileValue1)

Since I am sending files through multiple batches (say if there are 10 batches), there will be total of around 150 to 200 MB of files trying to reach the controller through asynchronous AJAX calls.

While sending, I am getting the below error message:

System.OutOfMemoryException was thrown at converting base64 string to byte array - occuring at replace statement.

I have followed various posts for workaround, but nothing seems to be working for me. I tried to increase "httpRuntime maxRequestLength" and " maxAllowedContentLength" in web.config to 4 GB to allow huge size, but nothing seems to be working.

If I remove the line

 fileValue1 = fileValue1 .Substring(fileValue1 .IndexOf(',') + 1);

I am getting below error:

The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

Maximum size of file can reach upto 500 MB when sent through multiple batches.

I am not sure, how to read this string file by block to make sure memory is available before converting to byte array. Any help would be greatly appreciated.

probably you need change the configuration of your server to accept request bigger or change the time before expire the request because when you have several request the server want attend all and the request and the process is slow then the request expire and you dont receive all data — Emiliano
– Emiliano, Commented Jun 29, 2017 at 4:32
I believe I have the configuration setup to accept bigger request. As soon as it reaches the controller, it fails. It doesn't expire or takes longer time. — SRS
– SRS, Commented Jun 29, 2017 at 4:34
but you need say to the server that you going to process information for a while too then I add a second link in my answer — Emiliano
– Emiliano, Commented Jun 29, 2017 at 4:39
OutOfMemoryException - means your Server process has run out of memory ;-) So, look at your Server process - it is 32 bit ? If yes, can you make it 64 bit ? How is your Server side code hosted ? (IIS/ self-hosted/ etc) — Subbu
– Subbu, Commented Jun 29, 2017 at 4:41
@Emiliano - Thanks. I will check the links and update if I had any luck — SRS
– SRS, Commented Jun 29, 2017 at 4:43

Emiliano · Accepted Answer · 2017-06-29 04:35:37Z

0

probably you need change the configuration of your server to accept request bigger or change the time before expire the request because when you have several request the server want attend all and the request and the process is slow then the request expire and you don't receive all data

this link can be helpful or this other

answered Jun 29, 2017 at 4:35

Emiliano

7289 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

SRS Over a year ago

I have tried changing various parameters in Web.config file to make sure that server can accept bigger requests. Tried including maxQueryStringLength, executionTimeout values, changed maxRequestLength. But still getting "System.OutOfMemoryException".

Emiliano Over a year ago

yes, I believe you, but you don't change that I say, I say change the timeout of the script

Emiliano Over a year ago

check my two links please

SRS Over a year ago

Sorry, I was checking your links and then only making my changes in web.config. I forgot mention about this in my previous comment.

Collectives™ on Stack Overflow

System.OutOfMemoryException was thrown at converting base64 string to byte array while processing multiple files

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related