Function parameter semantics (with nested functions)

Question

I'm learning python and I am stumbling over trying to understand how Python handles closures. I generally understand this to be a free-variable is stored in memory to be accessed long after the callee scope has expired. I am confused as to how this kind of setup is resolved.

def do_something(x):
    return lambda y: y + x;
f = do_something(4);
f(4)

Here f(4) seems to use the lambda function signature instead of the do_something signature. It returns 8, but what internally causes python to use y instead of x as the input parameter? Should I think of do_something similarly to a class definition, where f = do_something(4) is a constructor and f(4) is the actual action to take?

See this for where I first encountered this.

There are two separate functions here, one of which returns the other. do_something has x as a parameter while the lambda has y. — interjay
– interjay, Commented Nov 23, 2015 at 17:40
Now it's even worse. Which one do you think is the constructor, which one do you think is the action? — Barmar
– Barmar, Commented Nov 23, 2015 at 17:42
I think the f = do_something(x) is the constructor, as f(4) seems to be using the signature from the lambda function. — d4r3llo5
– d4r3llo5, Commented Nov 23, 2015 at 17:43

Barmar · Accepted Answer · 2015-11-23 17:55:07Z

1

The expression lambda y: y + x creates a new function, and this function is returned by do_something to its caller. In this function, x is a free variable, and the closure saves a reference to the parameter to the do_something function that was used to create it. So when you call

f = do_something(4)

it's equivalent to:

f = lambda y: y + 4

which is equivalent to:

def f(y):
    return y + 4

Then when you call f(4) it calls that function, binding y to the f parameter.

You're correct that this is similar to the way class constructors work; in some programming languages, closures are used as the internal implementation of class instances. Classes add things like inheritance, and also typically provide easier syntax to declare the class properties and methods.

edited Nov 23, 2015 at 17:55

answered Nov 23, 2015 at 17:47

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

d4r3llo5 Over a year ago

So basically the function do_something is replaced by the lambda function in all future calls of variable 'f'. In your example, f = lambda y: y + 4, it would be assumed that y is initialized to 0 then, right? Is the semantics for having an outer method do_something just for the functionality of the closure?

Barmar Over a year ago

do_something isn't replaced by anything. In general x = func(y) means to call func, and assign whatever it returns to x. do_something returns a function, and that function is assigned to f.

Barmar Over a year ago

y is initialized to the argument of f.

mgilson · Accepted Answer · 2015-11-23 17:55:29Z

Function objects have an attribute that stores the values in a "cell". You can access the closed-over variables from the __closure__ attribute:

>>> def do_something(x):
...     return lambda y: y + x;
... 
>>> f = do_something(4)
>>> f.__closure__[0]
<cell at 0x105d86600: int object at 0x7f905ac12bd0>

This doesn't seem all that helpful. Luckily, it's easy to get the cell's contents too:

>>> f.__closure__[0].cell_contents
4

Ok, that's only the first part of the puzzle (how does python store the closed over data). Lets look at the disassembled bytecode for f to see what python is doing with this data:

>>> import dis
>>> dis.dis(f)
  2           0 LOAD_FAST                0 (y)
              3 LOAD_DEREF               0 (x)
              6 BINARY_ADD          
              7 RETURN_VALUE

The first code LOAD_FAST is what is used to load a local variable (stored as an array of pointers to objects in C -- the 0 indicates that this is the first object in the array). In this case y is a local variable since it's in the function's signature. The next code is [LOAD_DEREF](enter link description here) which actually looks in the cell and to load the value from the closure. And that completes the picture for this problem at a low level ...

At a higher level, you don't need to worry about any of this. All you need to know is that a python function is able to look up the values of any scope that encloses it.

Collectives™ on Stack Overflow

Function parameter semantics (with nested functions)

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related