31

I've been getting the slug size too large warning (Compiled slug size: 789.8M is too large (max is 500M)) from Heroku and I can't figure out why, as my model size (cnn.pth below) is fairly small and my file directory is only 1.1mb in total: screenshot of directory.

It seems like the size increase is caused by running pipenv install torch, as the slug size was 89.1mb before installing torch and 798.8mb after.

My Pipfile currently has these packages installed:

[packages]
flask = "*"
flask-sqlalchemy = "*"
psycopg2 = "*"
psycopg2-binary = "*"
requests = "*"
numpy = "*"
gunicorn = "*"
pillow = "*"
torch = "*"

Is there any workaround for this?

Edit: I'm running Mac OSX 10.10.5, using Flask and pipenv.

Improve this question

5 Answers 5

Reset to default
23

The pytorch package that you're installing comes with both cpu and gpu support, thus has a large size. It seems you're using the free version of heroku, and only require the cpu support. The solution is to install the pytorch package for cpu only i.e.

In requirements.txt, write the wheel file path corresponding to the version of pytorch (cpu) that you're interested in. You can find the list of wheel files, which can be installed with pip. For example, for PyTorch 1.3.1, torchvision 0.4.2, Python 3.7, Linux, you can write the following for pytorch and torchvision respectively:

https://download.pytorch.org/whl/cpu/torch-1.3.1%2Bcpu-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cpu/torchvision-0.4.2%2Bcpu-cp37-cp37m-linux_x86_64.whl

The above will download torch-1.3.1+cpu-cp37-cp37m-linux_x86_64.whl (107MB) torchvision-0.4.2+cpu-cp37-cp37m-linux_x86_64.whl (13MB) respectively.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Thanks for replying! I tried this but it didn't work. I realised I didn't mention my OS in my original question - I'm on Mac OSX 10.10.5. I've tried using the mac version pipenv install https://download.pytorch.org/whl/cpu/torch-1.3.1-cp37-none-macosx_10_7_x86_64.whl (from the link you shared) but it returned me this error: torch-1.3.1-cp37-none-macosx_10_7_x86_64.whl is not a supported wheel on this platform..
I wonder if the issue might be because I'm using pipenv install instead of pip install? I found this solution while searching, but found their answer rather confusing, and haven't been able to get it to work (I changed the name of the package in the Pipfile and Pipfile.lock).
Make sure you're using the correct python version and macosx version in wheel file as well. If it doesn't work, use the same pip/pipenv install torch to check the correct wheel (.whl) path which is printed while installation, and use the same path in your installation.
Unfortunately there doesn't seem to be a CPU only wheel file for OSX 10.10.5, the latest they support is presumably 10.7 (looking at the file name). I'm using Python 3.7.
I know it's late but that worked for me: https://download.pytorch.org/whl/cpu/torch-1.3.1%2Bcpu-cp36-cp36m-linux_x86_64.whl
|
18
  1. Go to PyTorch Get Started page. Choose Stable version, Linux, pip, python, cpu and you can see:
pip install torch==1.8.1+cpu torchvision==0.9.1+cpu torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
  1. Copy versions to your requirements.txt.
torch==1.8.1+cpu 
torchvision==0.9.1+cpu
  1. And add to the beginning of the requirements.txt this:
-f https://download.pytorch.org/whl/torch_stable.html

After that, all packages will be installed on Heroku.

For reference: torch==1.8.1+cpu (169.1 MB); torchvision==0.9.1+cpu (13.3 MB)

Improve this answer

2 Comments

Just a hint: I tried this with torch>=1.8.1+cpu and it didn't work; the slug size was still too large. It seems to be important to use ==, not >= in the requirements.
ERROR: Could not find a version that satisfies the requirement torch==1.8.1+cpu (from -r /tmp/build_9027c8a3/requirements.txt (line 1)) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0)
3

By today, March 28, 2022, these are the working versions:

torch==1.11.0+cpu
torchvision==0.12.0+cpu
Improve this answer

Comments

2

This works for me in requirements.txt (thanks to https://github.com/pytorch/pytorch/issues/47354#issuecomment-1003118847):

# Attempt to reduce slug size by discarding GPU-specific libraries
--find-links https://download.pytorch.org/whl/cpu/torch_stable.html
torch==2.0.1
--find-links https://download.pytorch.org/whl/cpu/torch_stable.html
torchvision==0.15.2

This reduces the wheel size from ~700MB to ~200MB

enter image description here

Improve this answer

Comments

1

(Aug, 2, 2022) the only solution I found was leaving the requirements.txt like this:

--find-links https://download.pytorch.org/whl/torch_stable.html
torch==1.11.0+cpu
--find-links https://download.pytorch.org/whl/torch_stable.html
torchvision==0.12.0+cpu
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.