Unable to insert data in Postgres using Jdbc

Question

I am attempting to insert data into a PostgreSQL database using PySpark with JDBC. However, during the data insertion process, it is unexpectedly attempting to recreate the table and producing the following output.

org.postgresql.util.PSQLException: ERROR: relation "account" already exists

I am trying to use the below code snippet to write the data in Postgres.

def postgres_writes(url, driver, username, password, table_name, df):
df.write \
  .format("jdbc") \
  .option("url", url) \
  .option("dbtable", table_name) \
  .option("user", username) \
  .option("password", password) \
  .option("driver", driver) \
  .mode("append") \
  .save()

I want to append the data in an existing table.

Vijay_Shinde · Accepted Answer · 2023-06-21 10:24:38Z

1

Please make sure that, you are passing appropriate jdbc url. As per Postgres documentation Documentation Link you can use below JDBC URL's forms

jdbc:postgresql:database
jdbc:postgresql:/
jdbc:postgresql://host/database
jdbc:postgresql://host/
jdbc:postgresql://host:port/database
jdbc:postgresql://host:port/

Also try to recreate the table and try to load the data again. There might be some issue at Spark session level.

edited Jun 21, 2023 at 10:24

answered Jun 21, 2023 at 4:43

Vijay_Shinde

1,3523 gold badges17 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

SK ASIF ALI Over a year ago

I am using jdbc:postgresql://host:port/database format

Vijay_Shinde Over a year ago

Ok. I tried your scenario. I am able to append the data.

SK ASIF ALI Over a year ago

Is there any configuration I need to change in the DB side?

SK ASIF ALI Over a year ago

Deleting and recreating the table resolve the issue

Collectives™ on Stack Overflow

Unable to insert data in Postgres using Jdbc

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related