Conditional aggregation of rows in Polars based on consecutive values

Question

I have some data with 2 columns in the following format:

import polars as pl

df = pl.from_repr("""
┌──────┬─────────┐
│ nums ┆ letters │
│ ---  ┆ ---     │
│ i64  ┆ str     │
╞══════╪═════════╡
│ 160  ┆ B       │
│ 540  ┆ I       │
│ 550  ┆ I       │
│ 590  ┆ B       │
│ 370  ┆ I       │
└──────┴─────────┘
""")

And I want to merge the cells that come in the form of B in addition to consecutive I's (i.e. BI*):

shape: (2, 1)
┌─────────────────┐
│ nums            │
│ ---             │
│ list[i64]       │
╞═════════════════╡
│ [160, 540, 550] │
│ [590, 370]      │
└─────────────────┘

How can I achieve something like this with polars?

jqurious · Accepted Answer · 2024-10-17 19:58:48Z

2

You can search for the consecutive "B" -> "I" pairs, toggle those to true and cum_sum between them.

df = pl.DataFrame({
    "nums": [160, 540, 550, 590, 370],
    "letters": ["B", "I", "I", "B", "I"]
})

(df.group_by(
    ((pl.col("letters") + pl.col("letters").shift(-1)).eq_missing("BI"))
    .cum_sum()
    .alias("cum_sum"),
    maintain_order=True,
).agg("nums"))

This outputs:

shape: (2, 2)
┌─────────┬─────────────────┐
│ cum_sum ┆ nums            │
│ ---     ┆ ---             │
│ u32     ┆ list[i64]       │
╞═════════╪═════════════════╡
│ 1       ┆ [160, 540, 550] │
│ 2       ┆ [590, 370]      │
└─────────┴─────────────────┘

edited Oct 17, 2024 at 19:58

jqurious

24.1k6 gold badges24 silver badges43 bronze badges

answered Apr 13, 2022 at 10:37

ritchie46

15.5k2 gold badges45 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

user11718531 · Accepted Answer · 2022-04-13 09:09:29Z

0

If you are happy with the for loop, that's it. I tried also with polars' data manipulation functions but I didn't manage to do it this way.

import polars as pl

df = pl.DataFrame({
    "nums": [160, 540, 550, 590, 370],
    "letters": ["B", "I", "I", "B", "I"]
})

out = []
for d, l in zip(df["nums"], df["letters"]):
    if l == "B":
        out.append([d])
    elif l == "I":
        out[-1].append(d)
    else:
        raise Exception("Unknown letter: " + l)

print(out)

answered Apr 13, 2022 at 9:09

user11718531

Collectives™ on Stack Overflow

Conditional aggregation of rows in Polars based on consecutive values

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related