Is there a wrapper for numpy / ndarrays that returns the array for each of the mutation methods?

Question

The decision by python and apparently numpy to not return an object after its mutation is a source of frequent inconvenience. This question will be based on a different approach: to use a builder pattern so that we can do this:

x = np.random.randint(0,10+1,50).sort()[:5]

and end up with the least five values from a random set of ten numbers. Instead the above results in a runtime error due to subscripting None after the sort. I am wondering if there were either a library that provides that builder pattern atop numpy or some way to coerce all mutation numpy methods to return self instead of None.

For a single mutation the numpy method might suffice:

x = np.sort(np.random.randint(0,10+1,50))[:5]

But that approach does not scale when a series of methods are required. E.g.

x = np.resize(np.sort(np.random.randint(0,10+1,50))[:5],[5,1])

That quickly becomes difficult not only to write but also to read: we are asked to both write and read the code "inside out" from right to left.

Update A "winner" was declared below. Making a slight modification - renaming from ChainWrapper to Wrp just for brevity - here is the usage on both a list and an (numpy) ndarray:

Wrp(list(range(0,10,2))).append(8).append(10).insert(0, "hello").reverse().unwrap()[0:2]
# list:[10, 8]
import numpy as np
Wrp(np.linspace(0, 9, 10)).reshape(5, 2)[1:3, 0])
#  np.array: array([2., 4.])

The point is : append, insert reverse and reshape return None . The Wrp detects that and returns self instead. Frequently I want to write a lambda that performs more than one operation on a list. It was not possible to do so without the above.

So instead of doing something like x=np.sort(np.random.randint(0,10+1,50))[:5], you want to be able to do it in the way you describe so you can string many methods using the builder pattern? — jawsem
– jawsem, Commented Mar 15, 2020 at 13:38

AKX · Accepted Answer · 2020-03-15 20:15:28Z

2

Not returning a mutated object is a Pythonism that reminds you of the fact that no new object was created. Practically all standard library functions that do internal mutation return None for this reason.

You could write a wrapper of your own (and with some getattr magic, make it automatic), but it's probably not quite worth it.

EDIT: If you need this just for chaining, you could do something like

def chain(a, f):
    f(a)
    return a

x = chain(
    np.random.randint(0,10+1,50),
    lambda m: m.sort(),
)[:5]

or even fancier,

def hyperchain(val, *fs):
    for f in fs:
        res = f(val)
        if res is not None:
            val = res
    return val

to let you chain value-returning things and None-returning things:

x = hyperchain(
    np.random.randint(0,10+1,50),
    lambda m: m.sort(),
    lambda m: m[:5],
)

EDIT 2: Here's the aforementioned getattr wrapper idea -- not saying this is a good idea, or perfect, but here we go:

from functools import wraps


class ChainWrapper:
    def __init__(self, target):
        self._target = target

    def __getattr__(self, key):
        attr = getattr(self._target, key)
        if callable(attr):

            @wraps(attr)
            def wrapped_func(*args, **kwargs):
                retval = attr(*args, **kwargs)
                if retval is None:
                    retval = self
                return retval

            return wrapped_func
        return attr

    def __str__(self):
        return self._target.__str__()

    def __repr__(self):
        return f"<chain-wrapped {self._target!r}"

    def unwrap(self):
        return self._target

    # TODO: implement other things such as __getitem__ and __setitem__
    #       to just proxy through


l = [1, 2, 4, 8]

lw = ChainWrapper(l)
print(lw.append(8).append(10).insert(0, "hello").reverse())

This outputs

[10, 8, 8, 4, 2, 1, 'hello']

edited Mar 15, 2020 at 20:15

answered Mar 15, 2020 at 13:39

AKX

171k16 gold badges146 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

WestCoastProjects Over a year ago

I am aware of this being a design decision in python. I can not go back in time and debate on that at the time that decision were made: it's something I get to deal with (frequently..) . It leads to not being able to use the mutating methods in expressions either . I would be fine to go and wrap the entire numpy : but that will not work either since third party libraries will return unwrapped versions of ndarrays.

WestCoastProjects Over a year ago

Thanks: the chaining approach is something I do use: e.g. I have contributed to pipe julienpalard.github.io/Pipe and forked/enhanced the scalaps as infixpy github.com/javadba/infixpy . But explicitly saying lambda time and again is unwieldy. It gets tiring for a functional leaning programmer to use this language but I'm trying..

WestCoastProjects Over a year ago

You alluded to __getattr__ : I am actually more interested in that presently. Would you mind to explore that further?

AKX Over a year ago

That will still have the problem of you needing to re-wrap any numpy array you get from another library, so it'd be unwieldy. But see e.g. stackoverflow.com/questions/26091833/proxy-object-in-python

WestCoastProjects Over a year ago

> unwieldy . Yes agreed the getattr approach is limited in usefulness. I am writing an entire library that extends ndarray so at least within that library I can use this approach

|

Collectives™ on Stack Overflow

Is there a wrapper for numpy / ndarrays that returns the array for each of the mutation methods?

1 Answer 1

11 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related