Js binding for large rust object using wasm-bindgen

Question

I want to write a vscode extension that displays the content of a large binary file, written with bincode:

#[macro_use]
extern crate serde_derive;

use std::collections::HashMap;
use std::fs::File;
use std::io::{BufReader, BufWriter};

#[derive(Serialize, Deserialize)]
pub struct MyValue {
    pub name: String,
}

#[derive(Serialize, Deserialize)]
pub struct MyStruct {
    pub data: HashMap<String, MyValue>,
}

impl MyStruct {
    pub fn dump(&self, filename: &str) -> Result<(), String> {
        let file = File::create(filename).map_err(|msg| msg.to_string())?;
        let writer = BufWriter::new(file);
        bincode::serialize_into(writer, self).map_err(|msg| msg.to_string())
    }

    pub fn load(filename: &str) -> Result<Self, String> {
        let file = File::open(filename).map_err(|msg| msg.to_string())?;
        let reader = BufReader::new(file);
        bincode::deserialize_from::<BufReader<_>, Self>(reader).map_err(|msg| msg.to_string())
    }
}

Therefore there is a wasm binding:


#[wasm_bindgen]
#[derive(Clone)]
pub struct PyMyStruct {
    inner: Arc<MyStruct>,
}

#[wasm_bindgen]
impl PyMyStruct {
    pub fn new(filename: &str) -> Self {
        Self {
            inner: Arc::new(MyStruct::load(filename).unwrap()),
        }
    }

    pub fn header(self) -> Array {
        let keys = Array::new();
        for key in self.inner.data.keys() {
            keys.push(&JsValue::from_str(key));
        }
        keys
    }

    pub fn value(&self, name: &str) -> JsValue {
        if let Some(value) = self.inner.data.get(name) {
            JsValue::from_serde(value).unwrap_or(JsValue::NULL)
        } else {
            JsValue::NULL
        }
    }
}

which provides a simple interface to the JavaScript world in order to access the content of that file. Using Arc in order to prevent expensive unintended memory copy when handling on the JavaScript side. (It might look strange that keys is not marked as mutable but the rust compiler recomended that way)

When running the test code:

const {PyMyStruct} = require("./corejs.js");

let obj = new PyMyStruct("../../dump.spb")
console.log(obj.header())

you get the error message:

Error: null pointer passed to rust

Does someone know how to handle this use case?

Thank you!

Ibraheem Ahmed · Accepted Answer · 2020-12-11 12:38:26Z

1

The issue here is that you are using new PyMyStruct() instead of PyMyStruct.new(). In wasm-bindgen's debug mode you will get an error about this at runtime. Using .new() will fix your issue:

let obj = PyMyStruct.new("../../dump.spb")

If you add the #[wasm_bindgen(constructor)] annotation to the new method, then new PyMyStruct() will work as well:

#[wasm_bindgen]
impl PyMyStruct {
    #[wasm_bindgen(constructor)]
    pub fn new(filename: &str) -> Self {
        Self {
            inner: 1,
        }
    }
}

Now this is fine:

let obj = new PyMyStruct("../../dump.spb")

edited Dec 11, 2020 at 12:38

answered Dec 10, 2020 at 23:02

Ibraheem Ahmed

14k2 gold badges57 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Matthias Over a year ago

WIth your first suggestion I get: ``` let obj = PyMyStruct("../../dump.spb").new() ^ TypeError: Class constructor PyMyStruct cannot be invoked without 'new' ``` But with ` #[wasm_bindgen(constructor)]` it works.

Matthias Over a year ago

Unfortunately: There are still 2 problems remaining: 1. You can only call a method of the object once. The second time you get still Error: null pointer passed to rust 2. Rust is not able to open a local file when running in wasm, even when in Node.js.

Ibraheem Ahmed Over a year ago

@Matthias I updated my answer to fix your first error

Matthias · Accepted Answer · 2020-12-11 12:53:48Z

I solved that problem by using https://neon-bindings.com instead of compiling the API to web-assembly.

The binding here looks as follow:

use core;
use std::rc::Rc;
use neon::prelude::*;

#[derive(Clone)]
pub struct MyStruct {
    inner: Rc<core::MyStruct>,
}

declare_types! {
    pub class JsMyStruct for MyStruct {
        init(mut cx) {
            let filename = cx.argument::<JsString>(0)?.value();

            match core::MyStruct::load(&filename) {
                Ok(inner) => return Ok(MyStruct{
                    inner: Rc::new(inner)
                }),
                Err(msg) => {
                    panic!("{}", msg)
                }
            }
        }

        method header(mut cx) {
            let this = cx.this();
            let container = {
                let guard = cx.lock();
                let this = this.borrow(&guard);
                (*this).clone()
            };
            let keys = container.inner.data.keys().collect::<Vec<_>>();
            let js_array = JsArray::new(&mut cx, keys.len() as u32);
            for (i, obj) in keys.into_iter().enumerate() {
                let js_string = cx.string(obj);
                js_array.set(&mut cx, i as u32, js_string).unwrap();
            }
            Ok(js_array.upcast())
        }

        method value(mut cx) {
            let key = cx.argument::<JsString>(0)?.value();
            let this = cx.this();
            let container = {
                let guard = cx.lock();
                let this = this.borrow(&guard);
                (*this).clone()
            };
            if let Some(data) = container.inner.data.get(&key) {
                return Ok(neon_serde::to_value(&mut cx, data)?);
            } else {
                panic!("No value for key \"{}\" available", key);
            }
        }
    }
}

register_module!(mut m, {
    m.export_class::<JsMyStruct>("MyStruct")?;
    Ok(())
});

Collectives™ on Stack Overflow

Js binding for large rust object using wasm-bindgen

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related