Write to large file without loading the whole file into RAM

Question

I would like to read and write to a large file (stored on the user's file system) using browser Javascript without loading the whole file into RAM (it's too big to fit in RAM). I already have the code that lets me read a section of the file without loading the file into RAM, but I can't find how to write to the file.

class DiskAdapter {
    file;

    constructor(file) {
        this.file = file;
    }

    async read(address, len) {
        let diskFile = this.file;
        return new Promise((resolve) => {
            let reader = new FileReader();
            reader.onload = function() {
                resolve(reader.result);
            };
            let blob = diskFile.slice(address, address + len);
            reader.readAsArrayBuffer(blob);
        });
    }

    async write(address, len, data) {
        // what goes here?
    }
}

document.getElementById("diskImage").addEventListener('change', (changeEvent) => {
    let diskFile = changeEvent.target.files[0];
    let diskAdapter = new DiskAdapter(diskFile);
    console.log(await diskAdapter.read(0, 512)); // read MBR of disk image
});

My goal is to have a disk image (a .iso or .img file) that can be booted in a virtualized OS in a browser. The user can then perform tasks in the OS and the changes will automatically be saved to persistent storage. And I want the disk image to be easily swappable (the user can easily boot from a different iso or copy the iso to a different browser).

@Phil I want to write to the file which is stored on the user's file system. — Vexcess
– Vexcess, Commented Aug 21 at 6:54
Writing to the user's file system is not yet something that easily done in Javascript running in a browser. You could try the File System API, but it is not widely supported. It will also prompt the user to give permission. In general I would advice against using this, except for experiments. Using a normal file download seems the better option. See: sqlpey.com/javascript/javascript-file-download Note that downloadable files are normally created server side. — KIKO Software
– KIKO Software, Commented Aug 21 at 7:17
I read your edit and I am confused. I've never heard of a whole OS literally running in a browser using Javascript. Am I missing something here? Perhaps you mean that you want to use the browser to remotely access an OS? It's also possible you don't quite understand the purpose of a browser. — KIKO Software
– KIKO Software, Commented Aug 21 at 7:54
You can run a whole OS in a browser. copy.sh/v86 lets you run Windows 2000 and many more. The issue is that v86 is pretty slow because it's an emulator. My plan was to write + compile an OS to web assembly. Running in web assembly should be reasonably close to native performance at least compared to an emulator written in JavaScript. — Vexcess
– Vexcess, Commented Aug 21 at 8:24

Vexcess · Accepted Answer · 2025-08-30 17:10:20Z

The browser's traditional file API only allows reading files and downloading files. It does not allow writing to an existing file. However, there is the new File System API https://developer.mozilla.org/en-US/docs/Web/API/File_System_API . The File System API allows writing to files on the user's file system, however, this is only supported in Chromium based browsers. Fortunately, there is the Origin Private File System API https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system . The Origin Private File System allows writing to hidden files on the user's file system. And unlike the former, the Origin Private File System is fully supported across Chromium, Firefox, and Safari.

The OPFS API therefore fulfills the requirement of writing to a large file on the user's file system without reading the entire file into RAM while also supporting all major browsers. The one issue is that the file you write to will not be visible to the user. However, it is possible to copy files from the private file system to the user's visible file system. Needing to copy the file for the user to see it is an minor annoyance, but it's acceptable for my use case.

Edit: MDN claimed Origin Private File System is fully supported across Chromium, Firefox, and Safari, but it turns out Safari really only partially supports it (createWritable is only supported in a pre-release version of Safari). But Safari users are only 6.5% of users so the following still works on the overwhelming majority of browsers.

Here's the code for writing to arbitrary points of a file:

// get root directory
const opfsRoot = await navigator.storage.getDirectory();
let fileHandle;
try {
    // get file handle
    fileHandle = await opfsRoot.getFileHandle('myfile');
} catch (e) {
    // create file if we failed to access it
    fileHandle = await opfsRoot.getFileHandle('myfile', {create: true});
}

// write to the file at arbitrary points
const writable = await fileHandle.createWritable();
await writable.write({
    type: "write",
    position: 0,
    data: "00001111"
});
await writable.write({
    type: "write",
    position: 2,
    data: "2222"
});
await writable.close();

// read back the file to confirm we've successfully written to the file
const file = await fileHandle.getFile();
console.log(await file.text()); // 00222211

For more information: https://web.dev/articles/origin-private-file-system#copy_a_file_from_the_origin_private_file_system_to_the_user-visible_file_system

Collectives™ on Stack Overflow

Write to large file without loading the whole file into RAM

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related