How to bundle libtorch with my rust binary?

Question

I am developing an AI chat desktop application targeting Apple M chips. The app utilizes embedding models and reranker models, for which I chose Rust-Bert due to its capability to handle such models efficiently. Rust-Bert relies on tch, the Rust bindings for LibTorch.

To enhance the user experience, I want to bundle the LibTorch library, specifically for the MPS (Metal Performance Shaders) backend, with the application. This would prevent users from needing to install LibTorch separately, making the app more user-friendly.

However, I am having trouble locating precompiled binaries of LibTorch for the MPS backend that can be bundled directly into the application via the cargo build.rs file. I need help finding the appropriate binaries or an alternative solution to bundle the library with the app during the build process.

I have set the LD export DYLD_LIBRARY_PATH=/opt/homebrew/Cellar/pytorch/2.2.0_10/lib:$DYLD_LIBRARY_PATH but after compilation and running.

*[main][~/Programs/pyano/rust-backend]$ ./target/release/rust-backend

dyld[93114]: Symbol not found: __ZN2at4_ops10layer_norm4callERKNS_6TensorEN3c108ArrayRefINS5_6SymIntEEERKNS5_8optionalIS2_EESC_db
  Referenced from: <1138FCDA-AB1A-3CC6-B540-4DE31C13A6CF> /Users/saurav/Programs/pyano/rust-backend/target/release/rust-backend
  Expected in:     <8F36AFD3-91CB-3828-BD48-4381DB5297F2> /opt/homebrew/Cellar/pytorch/2.2.0_10/libexec/lib/python3.12/site-packages/torch/lib/libtorch_cpu.dylib

Build static binary for your Rust application with musl target. — Sibi
– Sibi, Commented Sep 25, 2024 at 5:33

skellig · Accepted Answer · 2024-10-18 09:53:01Z

1

Perhaps building with LIBTORCH_STATIC=1 may help, see crate tch. When downloaded, libtorch shared objects are stored under torch-sys in your project build folder

answered Oct 18, 2024 at 9:53

skellig

291 bronze badge

Sign up to request clarification or add additional context in comments.

1 Comment

Community Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

How to bundle libtorch with my rust binary?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related