Javascript, uploading several files within an Array.reduce with Promises, how?

Question

Evolving from Javascript, spliced FileReader for large files with Promises, how?, which showed me how a Promise can also resolve a function, now I am stuck with the same but inside an Array.reduce function.

The goal is that I want to upload a file (which already does) within an array, where each array item (a file) is uploaded sequentially (i.e. controlled through promises).

Then, I understand that the answer is somehow in http://www.html5rocks.com/en/tutorials/es6/promises/?redirect_from_locale=es , but I cannot understand how to apply that to here. My array is not an array of promises, is an array of files. Well, the whole thing is still obfuscated to me.

This is my code, which would work if I could see the ein console.log message:

return myArray.reduce(function(previous, current) {
    var BYTES_PER_CHUNK = 100000;
    var start = 0;
    var temp_end = start + BYTES_PER_CHUNK;
    var end = parseInt(current.size);
    if (temp_end > end) temp_end = end;
    var content = ''; // to be filled by the content of the file
    var uploading_file = current;
    Promise.resolve().then(function() {
        return upload();
    })
    .then(function(content){
        // do stuff with the content
        Promise.resolve();
    });
},0)  // I add the 0 in case myArray has only 1 item
//},Promise.resolve()) goes here?

.then(function(){
    console.log('ein') // this I never see
});

function upload() {
  if (start < end) {
    return new Promise(function(resolve){
      var chunk = uploading_file.slice(start, temp_end);
      var reader = new FileReader();
      reader.readAsArrayBuffer(chunk);
      reader.onload = function(e) {
        if (e.target.readyState == 2) {
          content += new TextDecoder("utf-8").decode(e.target.result);
          start = temp_end;
          temp_end = start + BYTES_PER_CHUNK;
          if (temp_end > end) temp_end = end;
          resolve(upload());
        }
      }
    });
  } else {
    uploading_file = null;
    return Promise.resolve(content);
  }
}

updated after several comments, it seems that now it works ... not sure yet

var uploading_file, start, temp_end, end, content; var BYTES_PER_CHUNK = 100000;

myArray.reduce(function(previous, current) { return previous .then(function() { BYTES_PER_CHUNK = 100000; start = 0; temp_end = start + BYTES_PER_CHUNK; end = parseInt(current.size); if (temp_end > end) temp_end = end; content = ''; uploading_file = current;
```
upload()
.then(function(content){
    // do stuff with "content"
    console.log('here')
    return Promise.resolve();
});
```
}); },Promise.resolve()) .then(function(){ console.log('ein'); });

function upload() { if (start < end) { return new Promise(function(resolve){ var chunk = uploading_file.slice(start, temp_end); var reader = new FileReader(); reader.readAsArrayBuffer(chunk); reader.onload = function(e) { if (e.target.readyState == 2) { content += new TextDecoder("utf-8").decode(e.target.result); start = temp_end; temp_end = start + BYTES_PER_CHUNK; if (temp_end > end) temp_end = end; resolve(upload()); } } }); } else { uploading_file = null; return Promise.resolve(content); } }

improved code, seems to work, perhaps easier to read?

    var start, temp_end, end;
    var BYTES_PER_CHUNK = 100000;

    myArray.reduce(function(previous, current) {
        return previous
        .then(function() {
            start = 0;
            temp_end = start + BYTES_PER_CHUNK;
            end = parseInt(current.size);
            if (temp_end > end) temp_end = end;
            current.data = '';

            return upload(current)
            .then(function(){
                // do other stuff
                return Promise.resolve();
            });
        });
    },Promise.resolve())
    .then(function(){
      // do other stuff
    });

    function upload(current) {
        if (start < end) {
            return new Promise(function(resolve){
                var chunk = current.slice(start, temp_end);
                var reader = new FileReader();
                reader.readAsText(chunk);
                reader.onload = function(e) {
                    if (e.target.readyState == 2) {
                        current.data += e.target.result;
                        start = temp_end;
                        temp_end = start + BYTES_PER_CHUNK;
                        if (temp_end > end) temp_end = end;
                        resolve(upload(current));
                    }
                }
            });
        } else {
            return Promise.resolve();
        }
    }

What is the point of using Array reduce if you never even refer to the previous parameter - you're basically doing a forEach with bonus overhead ... — Jaromanda X
– Jaromanda X, Commented Aug 8, 2016 at 6:06
You slay me @T.J.Crowder - now I need to know what is the current best practice when chaining asynchronous tasks together based on an array ... no need for details, just a hint will do me :p — Jaromanda X
– Jaromanda X, Commented Aug 8, 2016 at 6:11
@T.J.Crowder, no offenses taken, but could you expand a little bit more? Apart from the Array.reduce, the rest of the code is wrong? I cannot find another way to do sequential file uploading when these have to be spliced in chunks. How would you do this? — GWorking
– GWorking, Commented Aug 8, 2016 at 6:14
@JaromandaX, you're right, I've got the impression that I had to use Array.reduce for the sequential uploading, maybe I do have to use it but I am doing it wrong here — GWorking
– GWorking, Commented Aug 8, 2016 at 6:14
@Gerard: A couple of things jump out: 1. Promise.resolve().then(function() { return upload(); }) is basically just upload(), as upload returns a promise. 2. Promise.resolve(); when you don't use the return value is a no-op. — T.J. Crowder
– T.J. Crowder, Commented Aug 8, 2016 at 6:17

Graham · Accepted Answer · 2017-09-23 02:05:48Z

4

You're very close! You need to use the previous value; it should be a promise. Set the initial value of the reduce to be Promise.resolve(). Then inside the reduce function, instead of Promise.resolve().then(...). you should have something like:

return previous
  .then(function() { return upload(current); })
  .then(function() { /* do stuff */ });

It's important that you return here. This will become previous next time the reduce function is called.

You have quite a few issues with the upload function. The biggest issue is that the way you are passing it variables makes it very hard to read :) (and error-prone!)

If you're only reading a text file, use readAsText instead. Note I've renamed it to readFile, because that's a more accurate name.

// returns a promise with the file contents
function readFile(file) {
    return new Promise(function (resolve) {
        var reader = new FileReader();
        reader.onload = function(e) {
            resolve(e.target.result);
        };
        reader.readAsText(file);
    };
}

Then your reduce is simply:

files.reduce(function(previous, file) {
    return previous
      .then(function() { return readFile(file); })
      .then(function(contents) {
          // do stuff
      });
}, Promise.resolve());

You have a big bug with the upload_file variable though. The variable is local to the scope of the reduce function, so it will but undefined inside upload. Pass that in as an argument instead:

function upload(upload_file) { ... }

Side note on var. This is why even though you set upload_file with var inside the reduce function, it would be whatever it was before that function was called for upload:

var a = 3;

function foo() {
  var a = 4;
  console.log(a); // 4
}

foo();
console.log(a); // 3

edited Sep 23, 2017 at 2:05

Graham

7,86020 gold badges67 silver badges92 bronze badges

answered Aug 8, 2016 at 6:16

cdrini

1,10613 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

T.J. Crowder Over a year ago

The link is useful, but it would be much more useful if you showed more clearly where this fits into the OP's code, and what parts of the OP's code you're removing.

cdrini Over a year ago

Mostly just wanted to paste the link, because I think it fits this case perfectly. Actually, the OPs use of reduce/promises is very close, but upload is quite broken, now that I look at it. That code will result in a lot of issues; posted this too fast.

GWorking Over a year ago

@cdrini Thanks a lot, working on an updated code, still not working though. With regard to the upload_file, this is defined before the Array.reduce so the scope should be ok (I've made the mistake while adapting the code to the question). However, I do think I should pass it through variable, but before that I'd like to fix/understand this promise barrier

cdrini Over a year ago

@Gerard your promises are very nearly perfect, it's upload that I'm still trying to wrap my head around :) I was assuming that upload returned a promise which resolved when the file was fully uploaded, but the use of variables both inside and outside makes the code difficult to read. If it's defined before the reduce, don't use var; this defines it only within the function.

GWorking Over a year ago

@cdrini I've changed the use of "global" variables for variables that I send to the functions (EDIT2), hopefully clearer to read now? But why don't use var? Shouldn't I always(?)

|

Collectives™ on Stack Overflow

Javascript, uploading several files within an Array.reduce with Promises, how?

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest