1

I have create a outbox messages in my FastAPI/SQLAlchemy/Postgres application. The application is hosted in GCP. Now I need to publish this events, "reading" from the outbox table. I need to have this events to be published in real time. What are the options that i have?

  1. A Cron that will read these messages and published it. The problem with this solution is the messages could be delayed. for the sake of nature of a cron job.
  2. Use a CDC, like GCP Datastream. It seems complex as as Datastream GCP still doesnt have a simple way to move these messages to pub/sub. Actually we need to move the messages for a GCP storage, and process from there.
  3. Debezium. The problem is that it will be a new moving piece in my architecture, that we need to manage.
Improve this question
3
  • how about using postgreSQL triggers ? take a look at this: stackoverflow.com/questions/63680356/… Commented Jun 6, 2024 at 12:21
  • If the notification failes, Does postgres will retry? In this solution, I suppose that we still have a dual writter problem. Commented Jun 6, 2024 at 13:46
  • what would cause the trigger to fail ? pg is a relatively stable tool, your retry system should be in the pubsub part anyway Commented Jun 6, 2024 at 14:04
Related questions
3982
How do I get the current time in Python?
2190
How do I measure elapsed time in Python?
1826
How do I get time of a Python program's execution?
Load 6 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.