Node.js fs.readFile store read data in memory

Question

I'm building a node.js app which will process a lot of numerical data. How do I store in memory data which has been read by fs.readFile, Example:

var somedata = ''; 
fs.readFile('somefile', 'utf8', function(err,data){
 somedata = data; 
});

//fs.readFile is done by now.. 
console.log(somedata) // undefined, why is this not in memory?

I read file data periodically and currently I have to write what ive read elsewhere and read it again to use it in another function.

How do i just store it in memory? Is this just how javascript works??

Note: i know readFile returns a promise.. Even if there is a timeout the data read still is not saved to the variable. — gorilla bull
– gorilla bull, Commented Feb 11, 2021 at 19:42
Duplicate: Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference — user5734311
– user5734311, Commented Feb 11, 2021 at 19:43
Since you're using node, the short answer is to use readFileSync instead. — user5734311
– user5734311, Commented Feb 11, 2021 at 19:43
@ChrisG yea too bad all the other libraries out there dont have sync versions of their async methods... — gorilla bull
– gorilla bull, Commented Feb 11, 2021 at 20:49

Vitalii · Accepted Answer · 2021-02-11 20:02:30Z

1

With readFile() the function

function(err,data){
 somedata = data; 
}

gets executed at a later time (asynchronously), which is common JavaScript programming pattern.

JS in a browser and NodeJS is single-thread and uses event-loop for concurrency.

Using readFileSync() will stop execution and return the result of read operation (like you would expect with other languages).

var somedata = ''; 
somedata = fs.readFileSync('somefile', 'utf8');

//fs.readFileSync is done by now.. (which is important difference with readFile()
console.log(somedata)

References:

edited Feb 11, 2021 at 20:02

answered Feb 11, 2021 at 19:55

Vitalii

2,1611 gold badge7 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ortniel ADOUMASSE · Accepted Answer · 2024-12-08 18:21:19Z

-1

In the current version of nodeJs you got a Buffer, something like this <Buffer 66 69 72 73 74 6e 61 6d 65 2c 6c 61 73 74 6e 61 6d 65 2c 61 67 65 2c 66 69 65 6c 64 0a 4a 6f 68 61 6e 6e 2c 4b 65 72 62 72 6f 75 2c 33 30 2c 43 53 0a ... 192 more bytes> when you try to log data directly. Instead of that, you can use the toString() method on data and you will get the actual string read from the file. I suggest you to avoid readFileSync cause on big file it can block the event loop of nodeJS.

answered Dec 8, 2024 at 18:21

Ortniel ADOUMASSE

1

2 Comments

Sam R Dec 9, 2024 at 14:49

While readFileSync can block the main thread, and I agree that using it is not ideal in most situations, the issue here is that the console.log happens before the data has been read. This is an issue with asynchronicity and not a problem with decoding the buffer.

Ortniel ADOUMASSE Dec 10, 2024 at 16:22

You're absolutely right, I didn't pay attention at the time.

Collectives™ on Stack Overflow

Node.js fs.readFile store read data in memory

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related