Write to cloudsql in Google DataFlow

Question

I am processing data using Google DataFlow is there any way to write the results to cloudsql ?

Community · Accepted Answer · 2020-06-20 09:12:55Z

2

There is not currently a built-in way of doing this, but you can use the Sink API to write to arbitrary external systems.

Edit: The Sink API is likely overkill for writing to cloudsql; you can just directly make insert calls from a DoFn.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Nov 11, 2016 at 19:29

danielm

3,01012 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Kassem Shehady Over a year ago

Thanks danielm. Is it a good practice to write to cloudsql from dataflow ?

danielm Over a year ago

Sure, you can write to any external data storage system you need from Dataflow. Depending on your usecase, writing to BigQuery may also work, and Dataflow supports that out of the box.

Marcin Zablocki Over a year ago

I wouldn't say that SinkAPI is overkill, since it allows you to write to CloudSQL in a transaction - either everything is saved or nothing.

Collectives™ on Stack Overflow

Write to cloudsql in Google DataFlow

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related