Writing mixed numpy array types to file

Question

I'm trying to write a 3 column data array to a text file. Each row has an associated date and time contained in another array as a string.

For example:

>>> data 
array([[0, 1, 9],
       [1, 1, 2],
       [2, 4, 2],
       [7, 3, 2],
       [2, 2, 7],
       [1, 0, 5]])

>>> dates
array([['2017-07-28T12:00:00'],
       ['2017-07-28T12:01:00'],
       ['2017-07-28T12:02:00'],
       ['2017-07-28T12:03:00'],
       ['2017-07-28T12:04:00'],
       ['2017-07-28T12:05:00']], 
      dtype='<U19')

I'm trying to use numpy.savetxt to write the strings and data to file. Based on the above data, the output file should look like this:

'2017-07-28T12:00:00', 0, 1, 9
'2017-07-28T12:01:00', 1, 1, 2
'2017-07-28T12:02:00', 2, 4, 2
'2017-07-28T12:03:00', 7, 3, 2
'2017-07-28T12:04:00', 2, 2, 7
'2017-07-28T12:05:00', 1, 0, 5

Unfortunately, I can't figure out how to correctly join the string and numeric data to work with savetxt. I have tried concatenating the arrays, joining them in tuples, stacking them, etc, but the solution has alluded me all day. Ideally, I'd like to stick with numpy and not break this out line-by-line in a loop or involve Pandas.

Can anyone recommend a way to make this work?

If you don't like the %s answer, consider list approach. Iterate on the 'rows' and format and write one row at a time. — hpaulj
– hpaulj, Commented Jul 28, 2017 at 20:27

albert · Accepted Answer · 2017-07-28 20:30:11Z

3

Since you want to stack columns (like glueing the two arrays together) you need to use np.column_stack(). However, the resulting array has different datatypes which would result in some issues when calling np.savetxt(). So as a quick'n'dirty solution I provided the fmt=%s argument to use the string formatter:

import numpy as np

a = np.array([
        [0, 1, 9],
        [1, 1, 2],
        [2, 4, 2],
        [7, 3, 2],
        [2, 2, 7],
        [1, 0, 5]
    ])

b = np.array([
        ['2017-07-28T12:00:00'],
        ['2017-07-28T12:01:00'],
        ['2017-07-28T12:02:00'],
        ['2017-07-28T12:03:00'],
        ['2017-07-28T12:04:00'],
        ['2017-07-28T12:05:00']
    ])

out = np.column_stack([b, a])

np.savetxt('output.txt', out, delimiter='\t', fmt="%s")

The output looks like this (tab separated due to delimiter='\t'):

2017-07-28T12:00:00 0   1   9
2017-07-28T12:01:00 1   1   2
2017-07-28T12:02:00 2   4   2
2017-07-28T12:03:00 7   3   2
2017-07-28T12:04:00 2   2   7
2017-07-28T12:05:00 1   0   5

edited Jul 28, 2017 at 20:30

answered Jul 28, 2017 at 20:00

albert

8,69111 gold badges59 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

hpaulj Over a year ago

I expect out is a string type (that is, converting the numbers to their string equivalents). In that case %s is the only fmt option. That's ok as long as you don't need to control things like decimal points. Constructing a structured array with a compound dtype is more work, and also requires a fancier fmt.

jgrant Over a year ago

hpaulj is correct that out ends up as a string type. My example above was slightly simplified and I do require control of decimal points, so unfortunately albert's quick and dirty solution won't work. How does one create a compound data type containing a 2d array that can be written to file?

Collectives™ on Stack Overflow

Writing mixed numpy array types to file

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related