How to do bucket logic in partition for Iceberg Table using AWS Glue?

Question

# =====================================================
# 🧊 Step 4. Write Data to Iceberg Table (Glue Catalog)
# =====================================================

table_name = "glue_catalog.cmt_test_db.iceberg_table"

(
    df.writeTo(table_name)
      .using("iceberg")
      .tableProperty("format-version", "2")
      .partitionedBy(
          col("city__pk1"),
          expr("bucket(3, `age__pk3`)"),
          expr("bucket(3, `id__pk2`)")
      )
      .createOrReplace()
)

print(f"✅ Iceberg table '{table_name}' created/loaded successfully!")

I am getting error **AnalysisException: Invalid partition transformation: bucket(3, age__pk3)
**
I am able to achieve it using AWS Athena SQL Editor. But I want to do it using Dataframe API!!

Please guide me. Thanks in advance!!

Frank · Accepted Answer · 2025-11-04 19:46:15Z

-1

It works fine using Polars, just use Polars instead. thx

answered Nov 4 at 19:46

Frank

6366 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to do bucket logic in partition for Iceberg Table using AWS Glue?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related