0

I'd like to encapsulate my calc function and all its parameters inside an object, but vectorize the execution for millions of objects much like how numpy would do it. Any suggestions?

the calculation is still basic arithmetic which numpy should be able to vectorize.

Example code:

import numpy as np
myarray = np.random.rand(3, 10000000)

############################# This works fine: FAST ###################################

def calc(a,b,c):
    return (a+b/c)**b/a


res1 = calc(*myarray)  #0.7 seconds

############################# What I'd like to do (unsuccessfully): SLOW ###################################

class MyClass():
    __slots__ = ['a','b','c']

    def __init__(self, a,b,c):
        self.a, self.b, self.c = a,b,c

    def calc(self):
        return (self.a + self.b / self.c) ** self.b / self.a 

def classCalc(myClass:MyClass):
    return myClass.calc()

vectorizedClassCalc = np.vectorize(classCalc)
myobjects = np.array([MyClass(*args) for args in myarray.transpose()])


res2 = vectorizedClassCalc(myobjects) #8 seconds no different from a list comprehension
res3 = [obj.calc() for obj in myobjects] #7.5 seconds 

perhaps pandas has additional features?

6
  • This creates a lot of overhead because now Python has to deal with millions of different instances. You really should redesign you solution as I don't think, this will get much faster Commented Nov 13, 2018 at 14:29
  • 1
    Your array will consist of pointers to the objects elsewhere in memory, much like a list. Iteration over such an array is like a list comprehension,and no where as fast as the compiled numpy code for numeric dtypes. Search for np.frompyfunc for past discussions on this topic. Commented Nov 13, 2018 at 14:46
  • 1
    hpaulj is pretty spot on as to why you're seeing a slow down: good access patterns over data and contiguous memory are a very large part of why numpy is so fast. Another thing to consider is why you want to use OOP. The function that you have looks perfectly reasonable. A lot of scientific code (especially numpy/scipy heavy code) simply doesn't use OOP, or uses it sparingly when it's convenient or the right tool for the job. OOP is a tool: nothing more. But as an aside, consider just leaving your calc function as is and just pass it in your array: it looks fine as-is. Commented Nov 13, 2018 at 15:13
  • Are you not making a single object that contains the big arrays because you're not sure how many instances you have? Commented Nov 13, 2018 at 15:20
  • An example from a couple of weeks ago: stackoverflow.com/q/53034280/901925 Commented Nov 13, 2018 at 15:32

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.