How can I convert several NumPy arrays with ints to a NumPy array with formatted strings?

Question

I have three arrays of the same length containing integers: years, months and days. I want to create a (NumPy) array of the same length, containing formatted strings like '(-)yyyy-mm-dd' using the format '%i-%2.2i-%2.2i'.

For the scalar case, I would do something like

year=2000; month=1; day=1
datestr = '%i-%2.2i-%2.2i' % (year, month, day)

which would yield '2000-01-01'.

How can I create the vector version of this, e.g.:

import numpy as np
years  = np.array([-1000, 0, 1000, 2000])
months = np.array([1, 2, 3, 5])
days   = np.array([1, 11, 21, 31])
datestr_array = numpy.somefunction(years, months, days, format='%i-%2.2i-%2.2i', ???)

Note that the date range I am interested in lies between the years -2000 and +3000 (CE), and hence both Python's datetime and Pandas' DateTimeIndex offer no solution.

datastr_array = ['%i-%2.2i-%2.2i'.format(years[j], months[j], days[j]) for j in range(len(years))] - while this may look complicated, it is literally just formatting however many times are necessary. — Larry the Llama
– Larry the Llama, Commented Nov 22, 2021 at 19:22
You are right, I thought maybe it was a possible way but looks like always returning a string copy of the array. — RobertoT
– RobertoT, Commented Nov 22, 2021 at 19:49
Fast compiled code mostly works for numeric stuff, not strings — hpaulj
– hpaulj, Commented Nov 22, 2021 at 19:51
Dw @AstroFloyd - look at the code below - it fixes everything — Larry the Llama
– Larry the Llama, Commented Nov 22, 2021 at 19:59
Such a feature is not yet implemented for the datetime type already, so obviously not for multiple arrays (there is no generic formatting function in Numpy yet). I wonder if there is some tricks that can be used to get similar results manually without a loops but the string handling in Numpy is very limited, so I thing the best way may be to use a loop or maybe a C/Cython module (generating a Numpy array since it will not be faster with pure Python strings). — Jérôme Richard
– Jérôme Richard, Commented Nov 22, 2021 at 19:59

Larry the Llama · Accepted Answer · 2021-11-22 21:33:44Z

2

Explanation

Let's create a function that will convert any date without bounds to a yyyy-mm-dd string. We can use string formatting, where we create a predefined string and simply format in the relevant data. We also need to format the length to have zeros at the front to 'fill it out', i.e. 2001-05-20.

To be able to run this function, all the respective years months and days must be grouped together, which can be achieved with a zip function, which groups rows between columns as tuples. Preferably, we will convert this to a numpy array.

Now that we have the data in the correct tupled form, let's parse it through our function. We can create a new array that does this using apply, namely numpy.apply_on_axis(func, axis, data). Because the tuples are in the second axis, the axis parameter must be set to 1.

Code

def FormatDate(data):
    # Where data is a tuple for y, m, d
    return "{0:04}-{1:02}-{2:02}".format(data[0], data[1], data[2]) # Note that this formatting can later be update to account for some weirdness

# Convert the data into tuples where y, m, d are aligned in rows
converted = numpy.array(list(zip(years, months, days)))

# Now, lets apply that function to make the tuples all dates
datestr_array = numpy.apply_along_axis(FormatDate, 1, converted)

edited Nov 22, 2021 at 21:33

answered Nov 22, 2021 at 19:42

Larry the Llama

1,1206 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

12 Comments

Larry the Llama Over a year ago

If you would like an explanation, just add a comment.

AstroFloyd Over a year ago

Thanks Larry. However, Python's datetime gives me a ValueError: year -1000 is out of range. I think datetime doesn't believe the world existed before the year 1 CE, and DateTimeIndex doesn't even believe in the Middle Ages...

Larry the Llama Over a year ago

@AstroFloyd Edited- It can handle extreme dates AND Negatives.. EDIT: the hyphens may be a little confusing - you can change that by just editing the existing ones with .'s, etc. - Such as "{0:04}.{1:02}.{2:02}"

RobertoT Over a year ago

@LarrytheLlama great solution but I think yo should change the axis: 0 to 1 in \datestr_array = numpy.apply_along_axis(FormatDate, 0, converted)\ . Then it will work great!

hpaulj Over a year ago

df[['year','month','day']].values will be an array like converted. pandas may have some str functionality that makes this easier. It uses python strings in object dtype Series. I've seen people use some sort of df.str code, but haven't done anything myself.

|

hpaulj · Accepted Answer · 2021-11-25 16:57:50Z

1

A simple list comprehension will be faster than numpy functions:

['%i-%2.2i-%2.2i'%(y,m,d) for y,m,d in zip(years, months,days)]

for a dataframe

arr = df[['year','month','day']].values   # a (n,3) array
['%i-%2.2i-%2.2i'%(y,m,d) for y,m,d in arr]

Adding an arr=arr.tolist() might add some speed, since iteration of an list is faster than on an array.

answered Nov 25, 2021 at 16:57

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Collectives™ on Stack Overflow

How can I convert several NumPy arrays with ints to a NumPy array with formatted strings?

2 Answers 2

Explanation

Code

12 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Explanation

Code

12 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related