Environment:
- Python 3.9.21
- DuckDB 1.1.3
- pyarrow 18.1.0
- deltalake 18.1.0
Behavior explanation:
- add and udpate string fields in a struct inside a list under the root of the table works fine.
- update sting field in a struct inside a list inside a struct |table->struct->list->struct->field| raise and exception.
exception: duckdb.duckdb.TypeMismatchException: Mismatch Type Error: Type STRUCT(c0 VARCHAR, c1 VARCHAR, c2 STRUCT(status VARCHAR, handledTime VARCHAR)[]) does not match with STRUCT(firstResponseTime VARCHAR, totalResponseTime VARCHAR, stagingData STRUCT(status VARCHAR, handledTime VARCHAR)[]). Cannot cast STRUCTs - element "c0" in source struct was not found in target struct
How to reproduce:
- install the required packages using "pip3 install ..." duckdb | pyarrow | deltalake
- put this file in a working directory. duckdb_delta_merge_struct_bug.txt
- rename extension from .txt to py
- give permission to execute ex. chmod 755 poc1.py
- run this file once and see the printed row on the screen.
- delete the created table directory "poc" in your working directory.
- uncomment merge() on line 152 and run again and see the raised exception.