1

We have a python script in our source code repository which I maintain. Let's imagine it is at location

scripts/python/make_salad/make_salad.py

It is checked into the repository as is. Users of the script want to just click on the script in windows explorer and off they go. They refuse to use the command line. However the script also depends on many external packages that I have to install. I've used the following trick to install any packages required the first time the user runs the script. It goes like

def install(package):
    # This is an evil little function
    # that installs packages via pip.
    # This means the script can install
    # it's own dependencies.
    try:
        __import__(package)
    except:
        import subprocess
        subprocess.call([sys.executable, "-m", "pip", "install", package])

install("colorama")
install("pathlib")
install("iterfzf")
install("prompt_toolkit")
install("munch")
install("appdirs")
install("art")
install("fire")

import os
import tkFileDialog
import getpass
import json
import shutil
import subprocess
import sys
import pprint
import art

# <snip> out all my business logic
print("Making Salad")

However I don't like this because it installs the packages to the global package repository. What I'd like is if all the packages were installed something like

scripts/python/make_salad/make_salad.py
                         /__packages__
                                      /colorama
                                      /pathlib
                                      /iterfzf
                                      ...
                                      /fire

and then this directory would be first on the search path when import is called. Is it possible to hack the above script so the above is possible?

Note the requirements

  1. Only the single script is stored in repository
  2. Users have to click on the script in windows explorer
  3. Require to pip install external packages from within the script
  4. Packages should not pollute the global packages
Improve this question
8
  • So you'd like to have things installed in a virtual environment of sorts? Commented Aug 21, 2019 at 13:48
  • Yeah but I can't have users messing around at the command line or they freak out. And I can't check any virtual environments into source control or prebuild anything. Commented Aug 21, 2019 at 13:52
  • Would any of the exe-builders help in this case (depending on the packages); in that it builds a single executable out of Python and all relevant packages. Commented Aug 21, 2019 at 13:55
  • No. I can't check in big executables into source Commented Aug 21, 2019 at 13:55
  • 1
    @vinzBad The requirement (one of them) is Python 2.7; venv has only been built-in since Python 3.3, as per your link. Commented Aug 21, 2019 at 14:02

2 Answers 2

Reset to default
4

It turns out not too hard to use virtualenv directly from the script to manage this problem. I wrote a small helper class.

import sys
import subprocess

class App:
    def __init__(self, virtual_dir):
        self.virtual_dir = virtual_dir
        self.virtual_python = os.path.join(self.virtual_dir, "Scripts", "python.exe")

    def install_virtual_env(self):
        self.pip_install("virtualenv")
        if not os.path.exists(self.virtual_python):
            import subprocess
            subprocess.call([sys.executable, "-m", "virtualenv", self.virtual_dir])
        else:
            print("found virtual python: " + self.virtual_python)

    def is_venv(self):
        return sys.prefix==self.virtual_dir

    def restart_under_venv(self):
        print("Restarting under virtual environment " + self.virtual_dir)
        subprocess.call([self.virtual_python, __file__] + sys.argv[1:])
        exit(0)

    def pip_install(self, package):
        try:
            __import__(package)
        except:
            subprocess.call([sys.executable, "-m", "pip", "install", package, "--upgrade"])

    def run(self):
        if not self.is_venv():
            self.install_virtual_env()
            self.restart_under_venv()
        else:
            print("Running under virtual environment")

And can use it from the top of the main script make_salad.py like so

pathToScriptDir = os.path.dirname(os.path.realpath(__file__))

app = App(os.path.join(pathToScriptDir, "make_salad_virtual_env"))

app.run()

app.install("colorama")
app.install("pathlib")
app.install("iterfzf")
app.install("prompt_toolkit")
app.install("munch")
app.install("appdirs")
app.install("art")
app.install("fire")
app.install("appdirs")

print "making salad"

The first time running it will install the virtual env and all the required packages. Second time it will just run the script under the virtual environment.

found virtual python: C:\workspace\make_salad_virtualenv\Scripts\python.exe
Restarting under virtual environment C:\workspace\make_salad_virtualenv\Scripts
Running under virtual environment
Making Salad
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

Perhaps you could use the --prefix option with pip, modify sys.path to include that prefix (+ lib/python2.7/site-packages/). I think --prefix will even work with a relative path:

def install(package):
    try:
        __import__(package)
    except:
        import subprocess
        subprocess.call([sys.executable, "-m", "pip", "install", package, "--prefix="packages"])
.
.
.
sys.path.append("packages/lib/python2.7/site-packages/")
.
.
.
import art
.
.
.

(untested)

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.