How do I dynamically make a function in python?

Question

I have a computer generated AST in python. I interpret this AST 2000 times, but this is slow. Moreover, this AST changes during runtime, and it's not known before compile time. I can codegen my AST to a valid python string, but I don't know how to execute it. To give a conceptual overview, this is what I have now:

def exec(AST):
   {local variables}
   {interpret AST over local variables 2000 times}

I would like:

def exec(AST):
  {local variables}
  new_fn = make_lambda(to_python_string(AST))
  {iterate new_fn over local variables 2000 times}

Note: different calls to exec will have totally new unknown AST that was constructed at runtime. My current bottleneck is in the AST interpretation, and my hope is that dynamic function generation will make it faster.

"new_fn = make_lambda(to_python_string(AST))" You answered your own question, writ e a function that returns a function and use this in the loop. — Dennis Beier
– Dennis Beier, Commented Jul 31, 2020 at 21:25

lee · Accepted Answer · 2020-08-01 00:30:22Z

1

So python3.8 does have an exec command that takes a string, and executes it. There are some caveats if your generated function calls local variables.

This works:

def exec(AST):
  {local variables}
  exec("def new_fn():\n" + to_python_string(AST), globals())
  {iterate new_fn over local variables 2000 times}

This is blazingly faster than interpreting the AST.

answered Aug 1, 2020 at 0:30

lee

7822 gold badges14 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dennis Beier · Accepted Answer · 2020-07-31 21:38:41Z

0

This works:

def fun1():
    return 5


def fun2():
    return 7


def make_lambda(x):
    if x == 1:
        return fun1
    else:
        return fun2

print(make_lambda(1)())
print(make_lambda(2)())

answered Jul 31, 2020 at 21:38

Dennis Beier

3733 silver badges15 bronze badges

1 Comment

lee Over a year ago

This isn't what I'm looking for. Here fun1 and fun2 are known at compile time. In my case, I have the AST for fun1/fun2, which can be codegened into a valid string of python code. More over, this AST changes during execution. In my python file, I don't know what fun1 and fun2 look like, it's constructed during runtime, and want to execute it. Right now I interpret the AST which is slow.

Collectives™ on Stack Overflow

How do I dynamically make a function in python?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related