Parsing Python code from C++

Question

I can find a load of information on the reverse, not so much this way around :)

So, the summary is that I want to write some Python code-completion stuff in C++, but I can't figure out the best way of tokenizing the Python code.

Are there any libraries out there that will do this?

I'm leaning towards calling Python's tokenize.tokenize directly from C++... but whenever I look at calling Python code from C++ I go cross-eyed.

Python code completion support is very limited due to its dynamic nature. Add to that trying to do it in C++, you are setting yourself up for a world of pain. — Alex B
– Alex B, Commented Oct 16, 2012 at 11:55

Krumelur · Accepted Answer · 2012-10-16 11:06:28Z

3

Using regular parser-generators to generate parsers from the grammar is usually complicated with Python (for example due to its significant whitespace and difficult line-continuation rules).

I am not sure about your experience with Python, but my recommendation would be to parse the Python file from Python, and do as much of the processing as possible in Python, then return the result to the C++ code using well-defined data types (such as the stdc++ ones) and using Boost.python for the bindings.

answered Oct 16, 2012 at 11:06

Krumelur

32.8k10 gold badges82 silver badges123 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Parsing Python code from C++

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related