Can I pre-compile a python script?

Question

I have a python script. Lets say http://domain.com/hello.py, which only prints "Hello, World!".

Is it possible to precompile this Python file?

I get around 300 requests per second and the overhead of compiling is way to high. In Java the server can handle this easily but for calculations Python works much easier.

Python compiles .py files to .pyc on import, meaning it's only done once. What makes you think it's the "overhead of compiling" that's making your app slow? — univerio
– univerio, Commented May 9, 2015 at 0:12
@univerio, I don't import anything I just have one line "print 'hello world!" I tried testing the load with Jmeter and it could handle only about 2000 requests per minute. My Java servlet which does db queries calculations etc. can handle about 70k per minute. What else could be the cause of this? — user3605780
– user3605780, Commented May 9, 2015 at 0:16
this is more likely an artifact of you trying to serve python files as normal cgi .... try at least fcgi... but really you should be using nginx + gnuicorn or some kind of caching ... apache+mod_wsgi also would work... the problem with serving them as normal cgi (and maybe fcgi... Im not sure) is that its having to spool up a completely new interpreter environment for each request — Joran Beasley
– Joran Beasley, Commented May 9, 2015 at 0:20

Joran Beasley · Accepted Answer · 2015-05-09 00:30:56Z

11

the problem is not that you need to "precompile" python, the problem is that you are trying to execute python scripts using normal cgi script stuff...

the real answer is to use a better web backend than simple cgi to run your python

I would suggest the following in order of appearance

  1. nginx + gnunicorn
  2. apache2 + mod-wsgi
  3. something else
  4. anything else
  ...
n-1. fcgi
  n. cgi

I know this isnt really an answer and is entirely opinion based

answered May 9, 2015 at 0:30

Joran Beasley

114k13 gold badges167 silver badges187 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Will · Accepted Answer · 2015-05-09 00:17:00Z

2

Python code is compiled automatically the first time it is run, by the CPython (standard Python) interpreter. You can pre-compile it if you want to optimize the first request, but that usually isn't necessary. Aside from that, you'd need to convert your Python code into a Python C / cython Module. There are some tools to help you convert Python code into a Python Module if that's the route you want to go.

There's also a Python module called SciPy that's commonly used for Scientific Computing and Data Science applications, which provides a tool called Weave which allows you to inline C/C++ code into your Python code allowing certain performance-critical portions of the code to run using compiled C/C++ code.

answered May 9, 2015 at 0:17

Will

24.8k14 gold badges100 silver badges111 bronze badges

1 Comment

Joran Beasley Over a year ago

I dont think this really addresses OP problem (which is probably mis-stated in the title and OP ...) "precompiling" python is not the actual solution that will help the OP ... and i dont think you can really optimize print "hello world" with any amount of scipy or numpy

Stuart Cook · Accepted Answer · 2021-11-04 09:50:04Z

1

via the python interface where your python source file is abc.py:

import py_compile
py_compile.compile('abc.py')

answered Nov 4, 2021 at 9:50

Stuart Cook

111 bronze badge

Collectives™ on Stack Overflow

Can I pre-compile a python script?

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related