rename columns in dataframe pyspark adding a string

Question

I have written code in Python using Pandas that adds "VEN_" to the beginning of the column names:

Tablon.columns = "VEN_" + Tablon.columns

And it works fine, but now I'm working with PySpark and it doesn't work. I've tried:

Vaa_total.columns = ['Vaa_' + col for col in Vaa_total.columns]

or

for elemento in Vaa_total.columns:
    elemento = "Vaa_" + elemento

And other things like that but it doesn't work.

I don't want to replace the columns name, I just want to mantain it but adding a string to the beginning.

Possible duplicate of How to change dataframe column names in pyspark? — vvg
– vvg, Commented Jul 17, 2018 at 8:40
I don't think so, there is explained how to replace it but I don't know how I can to add a string to my columns name, I get: AttributeError: can't set attribute. — ITo
– ITo, Commented Jul 17, 2018 at 8:46

ags29 · Accepted Answer · 2018-07-17 08:50:31Z

4

Try something like this:

for elemento in Vaa_total.columns:
    Vaa_total =Vaa_total.withColumnRenamed(elemento, "Vaa_" + elemento)

answered Jul 17, 2018 at 8:50

ags29

2,7061 gold badge11 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

vvg · Accepted Answer · 2018-07-17 08:52:51Z

0

I linked similar topic in comment. Here's example adapted from that topic to your task:

dataframe.select([col(col_name).alias('VAA_' + col_name) for col_name in dataframe])

answered Jul 17, 2018 at 8:52

vvg

6,40522 silver badges36 bronze badges

sargupta · Accepted Answer · 2022-04-02 10:00:48Z

0

Standard format of writing it:

renamed_df = df.withColumnRenamed(col_name, "insert_text" + col_name) for col_name in dataframe.columns])

answered Apr 2, 2022 at 10:00

sargupta

1,03316 silver badges28 bronze badges

There is a bracket missing somewhere in your solution