Split a column of string into list of list

Question

How could I split a column of string into list of list?

Minimum example:

import polars as pl

pl.Config(fmt_table_cell_list_len=6, fmt_str_lengths=100)

df = pl.DataFrame({'test': "A,B,C,1\nD,E,F,2\nG,H,I,3\nJ,K,L,4"})

shape: (1, 1)
┌─────────┐
│ test    │
│ ---     │
│ str     │
╞═════════╡
│ A,B,C,1 │
│ D,E,F,2 │
│ G,H,I,3 │
│ J,K,L,4 │
└─────────┘

I try the following, somehow I stop after the first split

df = df.with_columns(pl.col('test').str.split('\n'))

My desired result would be it return a list of list inside the dataframe, so that the list of list is readily to be read by other columns

result = pl.DataFrame({'test': [[["A","B","C",1], ["D","E","F",2], ["G","H","I",3], ["J","K","L",4]]]}, strict=False)

result = result.with_columns(
    get_data = pl.col('test').list[2].list[3].cast(pl.Int64) # Answer = 3
)

shape: (1, 2)
┌──────────────────────────────────────────────────────────────────────────────────────────┬──────────┐
│ test                                                                                     ┆ get_data │
│ ---                                                                                      ┆ ---      │
│ list[list[str]]                                                                          ┆ i64      │
╞══════════════════════════════════════════════════════════════════════════════════════════╪══════════╡
│ [["A", "B", "C", "1"], ["D", "E", "F", "2"], ["G", "H", "I", "3"], ["J", "K", "L", "4"]] ┆ 3        │
└──────────────────────────────────────────────────────────────────────────────────────────┴──────────┘

Dean MacGregor · Accepted Answer · 2025-05-17 14:25:34Z

6

df.with_columns(
    pl.col('test')
      .str.split('\n')
      .list.eval(
          pl.element()
            .str.split(",")
      )
  )

In your example you have a list of mixed strings and numbers which polars doesn't support so your output has to have the numbers as strings.

You say you want to use these lists from other columns readily so you might want to convert to a struct column and unnest it so that you have new flat columns.

answered May 17 at 14:25

Dean MacGregor

20k10 gold badges57 silver badges111 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Baffin Chu May 17 at 15:57

Thanks for the concise and elegant solution! The keyword list, eval, element level operation is the answer I'm looking for!

Collectives™ on Stack Overflow

Split a column of string into list of list

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related