1

I do not understand why "u" has NaNvalues. What wrong am I doing here?

>>> z=pd.DataFrame([['abcb','asasa'],['sdsd','aeio']])
>>> z
          0      1
    0  abcb  asasa
    1  sdsd   aeio
>>> u=pd.DataFrame(z,columns=['hello','ajsajs'])
>>> u
    hello  ajsajs
 0    NaN     NaN
 1    NaN     NaN 
0

1 Answer 1

3

Alternate construction calls

You can use the underlying NumPy array:

u = pd.DataFrame(z.values, columns=['hello','ajsajs'])

  hello ajsajs
0  abcb  asasa
1  sdsd   aeio

Alternately, you could use:

u = z.rename(columns={0: 'hello',1: 'ajsajs'})

And lastly as suggested by @Dark:

u = z.set_axis(['hello','ajsajs'], axis=1, inplace=False)

A small note on inplace in set_axis -

WARNING: inplace=None currently falls back to to True, but in a future version, will default to False. Use inplace=True explicitly rather than relying on the default.

In pandas 0.20.3 the syntax would be just:

u = z.set_axis(axis=1, labels=['hello','ajsajs'])

@Dark's solution appears fastest here.

Why current method doesn't work

I believe the issue here is that there's a .reindex being called when the DataFrame is constructed in this way. Here's some source code where ellipses denote irrelevant stuff I'm leaving out:

from pandas.core.internals import BlockManager

# pandas.core.frame.DataFrame
class DataFrame(NDFrame):
    def __init__(self, data=None, index=None, columns=None, dtype=None,
                 copy=False):
        # ...
        if isinstance(data, DataFrame):
            data = data._data
        if isinstance(data, BlockManager):
            mgr = self._init_mgr(data, axes=dict(index=index, columns=columns),
                                 dtype=dtype, copy=copy)
        # ... a bunch of other if statements irrelevant to your case
        NDFrame.__init__(self, mgr, fastpath=True)
        # ...

What's happening here:

  • DataFrame inherits from a more generic base class which in turn has multiple inheritance. (Pandas is great, but its source can be like trying to backtrack through a spider's web.)
  • In u = pd.DataFrame(z,columns=['hello','ajsajs']), x is a DataFrame. Therefore, the first if statement below is True and data = data._data. What's _data? It's a BlockManager.* (To be continued below...)
  • Because we just converted what you passed to its BlockManager, the next if statement also evaluates True. Then mgr gets assigned to the result of the _init_mrg method and the parent class's __init__ gets called, passing mgr.

* confirm with isinstance(z._data, BlockManager).

Now on to part 2...

# pandas.core.generic.NDFrame
class NDFrame(PandasObject, SelectionMixin):
    def __init__(self, data, axes=None, copy=False, dtype=None,
             fastpath=False):
    # ...

    def _init_mgr(self, mgr, axes=None, dtype=None, copy=False):
        """ passed a manager and a axes dict """
        for a, axe in axes.items():
            if axe is not None:
                mgr = mgr.reindex_axis(axe,
                                       axis=self._get_block_manager_axis(a),
                                       copy=False)
    # ...
        return mgr

Here is where _init_mgr is defined, which gets called above. Essentially in your case you have:

columns=['hello','ajsajs']
axes=dict(index=None, columns=columns)
# ...

When you go to reindex axis and specify a new axis where none of the new labels are included in the old object, you get all NaNs. This seems like a deliberate design decision. Consider this related example to prove the point, where one new column is present and one is not:

pd.DataFrame(z, columns=[0, 'ajsajs'])

      0  ajsajs
0  abcb     NaN
1  sdsd     NaN
Sign up to request clarification or add additional context in comments.

2 Comments

You can also add u = z.set_axis(['hello','ajsajs'],1,inplace=False)
@Dark, really nice option. For the sake of completness we could also add: z.columns = ['hello','ajsajs']

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.