Is it possible to embed python without the standard library?

Question

I'm working with a cmake build for python 2.7.6 and I've got a basic embedded script running, like so:

#include <stdio.h>
#include <Python.h>

int main(int argc, char *argv[]) {

  /* Setup */
  Py_SetProgramName(argv[0]);
  Py_Initialize();

  /* Run the 'main' module */
  int rtn = Py_Main(argc, _argv);
  Py_Finalize();

  return rtn;
}

..but when I run it, I get:

ImportError: No module named site

If I setup a correct $PYTHONHOME, it works fine; but that's not what I'm trying to do. I'm trying to embed a copy of python in a stand alone application without the standard library.

I appreciate the use of it, but for this specific embedded environment I want something more like lua, (but with python syntax obviously), where only the specific libraries exposed by the parent application are available.

This has the added bonus of not caring about distributing (or building) the standard library with all it's cross linked dynamic libraries.

Is this possible as all? Or am I inevitably going to stumble into missing fundamental blocks of the language like sys.path, import, {}, [] or similar that are part of the standard library?

If it is possible, how would you go about doing it?

Why not provide your own minimal 'standard library'? Include just site.py, anything site.py needs to run, and anything your application requires? — Martijn Pieters
– Martijn Pieters, Commented Jan 6, 2014 at 14:09
But, yes, the CPython implementation does require certain modules to be present and importable. In Python 3, that includes the import machinery. sys is a built-in module, syntax is never imported but part of the language so {} and [] are not going to break. — Martijn Pieters
– Martijn Pieters, Commented Jan 6, 2014 at 14:19
I was thinking of this, but it's a little annoying to ship a binary with one .py file; I was thinking perhaps of hacking the import mechanism like kivy does for its kivy-ios project, and embedding the minimal required modules is raw text blobs, but I'm not exactly sure how to proceed down that path~ — Doug
– Doug, Commented Jan 6, 2014 at 14:26
You can include the python files as a .zip file if you prefer. — Martijn Pieters
– Martijn Pieters, Commented Jan 6, 2014 at 14:27

Doug · Accepted Answer · 2014-01-10 05:33:56Z

11

Simple answer is yes you can.

int main(int argc, char *argv[]) {

  /* Setup */
  Py_NoSiteFlag = 1; // <--- This
  Py_SetProgramName(argv[0]);
  Py_Initialize();

  /* Run the 'main' module */
  int rtn = Py_Main(argc, argv);
  Py_Finalize();

  return rtn;
}

As far as I can tell, nothing breaks and you can continue to use everything (including the ability to import modules~) perfectly fine. If you try to import anything from the standard library that isn't bundled, you'll get:

>>> import os;
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named os

I'll leave a copy of the code up here for anyone who's interested;

https://github.com/shadowmint/cmake-python-embed-nostdlib

edited Jan 10, 2014 at 5:33

answered Jan 7, 2014 at 8:53

Doug

35.6k40 gold badges189 silver badges231 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user395760 Over a year ago

Importing site.py should be avoidable without modifying the interpreter, I think. The main executable avoids it when given the -S flag. You'd have to check whether the relevant flag (Py_NoSiteFlag) is internal or private though. Alternatively, add a dummy site module to your embedded build. Both options seem better than maintaining a separate patch, even if it's so simple and unlikely to break.

Doug Over a year ago

@delnan you are completely correct! thanks! I've update my answer to reflect using the Py_NoSiteFlag, that's definitely a better solution.

P i Over a year ago

This doesn't seem to be sufficient for Python3: stackoverflow.com/questions/34724057/…

Paulo Scardine · Accepted Answer · 2014-01-06 14:28:16Z

1

AFAIK, you can't. Without the standard library, Python won't even start, since it tries to find os.py (in 3.x; string.py in 2.x). On startup, it imports a number of modules, in particular site.py. You need at least pythonxy.dll, and probably the contents of the lib folder, plus the extension modules, i.e. the contents of the DLLs folder.

You can ship it with a file pythonXX.zip (where XX is the version number like 27 or 33) which should be a zipped copy of the standard library. You are free to strip the library from stuff that you don't need; there are various tools that compute dependencies statically (see modulefinder).

I guess that is why Lua is more popular as an embedded scripting language.

answered Jan 6, 2014 at 14:28

Paulo Scardine

78.2k12 gold badges134 silver badges153 bronze badges

Collectives™ on Stack Overflow

Is it possible to embed python without the standard library?

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related