Skip to main content
295 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
Best practices
0 votes
0 replies
33 views

QuickBooks Online's API documentation includes a recommendation that users use webhooks instead of change-data-capture to synchronize their entity records with QuickBooks's. Why are webhooks ...
In Hoc Signo's user avatar
  • 551
0 votes
1 answer
85 views

I’m using PostgreSQL as the source database with Change Data Capture (CDC) enabled via publications. I have two related tables: -- Table 1: orders CREATE TABLE orders ( id UUID PRIMARY KEY, ...
jasraj bedi's user avatar
  • 161
1 vote
1 answer
63 views

We need to capture changes done in the PostgreSQL DB tables, and are planning on using CDC for the same. We have a requirement to ignore/filter events which were written inn the DB by a specific user/...
RaRa's user avatar
  • 306
0 votes
0 answers
24 views

I am looking for a way to design a stream that ONLY emits columns that have changed in a record. If I use the stream, it emits the entire record even if only one of the column values has changed in ...
Saqib Ali's user avatar
  • 4,551
0 votes
1 answer
95 views

We have a Kafka cluster that we're trying to connect Debezium to. We are able to successfully deploy a Producer or Consumer using the following producer.config/consumer.config (these are temporary ...
user1913559's user avatar
  • 395
0 votes
0 answers
84 views

I'm running a Flink cluster in Docker in my local env, and I've copied these jar files to the /opt/flink/lib/ of the image: flink-cdc-dist-3.3.0.jar flink-cdc-pipeline-connector-mysql-3.3.0.jar flink-...
Rinze's user avatar
  • 3
1 vote
1 answer
60 views

I want to have a ADF job that triggers on a SQL (Azure SQL DB) table re-load. I know that there is a Change Data Capture (CDC) trigger that can be used in ADF, but that seems to be something I would ...
M Rothwell's user avatar
  • 57
0 votes
1 answer
326 views

I am fairly new to Azure Data Factory (ADF) but have been learning and experimenting with some of its advanced features. I'm currently working on a use case involving Change Data Capture (CDC) and ...
Nahid Talukdar's user avatar
  • 3
3 votes
2 answers
132 views

I have a table in SQL Server that tracks the status of different IDs over time. The table includes both historical data and current data, and I need to write a query that will return the correct ...
Shay Davidovitch's user avatar
  • 35
1 vote
1 answer
359 views

Issue I am trying to make a query to the Salesforce API to retrieve the "Selected Entities" that are listed under "Change Data Capture" using jsforce v1.11.1. The goal would be to ...
Erik Hirsch's user avatar
  • 31
0 votes
1 answer
31 views

I’m looking to implement Change Data Capture(CDC) so I can synchronize other systems with the changes. I learned that PostgreSQL can use Debezium for CDC. Can PieCloudDB also support Debezium? Or does ...
Joe's user avatar
  • 89
0 votes
1 answer
895 views

I am using debezium in distributed mode to push the cdc events of mssql db to kafka topics. Command: %KAFKA_HOME%\bin\windows\connect-distributed.bat Debezium starts fine, I deployed the connector ...
Roobal Jindal's user avatar
  • 294
0 votes
2 answers
745 views

I'm working on a project related to building streaming change data capture using Kafka Connect. The source of changes is MySQL, then they are sent to the corresponding Kafka topic, and then applied to ...
Golikov Andrey's user avatar
  • 11
3 votes
1 answer
261 views

Should I rely on seqval for ordering change data captured from change tables in SQL Server, documentation provides contradictory statements in two places. If not, is there any option? If we check cdc....
toros's user avatar
  • 31
0 votes
0 answers
241 views

As a test, I enabled CDC on a DB and table for specific columns. CREATE TABLE [dbo].[testCDC] ( [ID] [bigint] IDENTITY(1,1) NOT NULL, [Col1] [nchar](10) NULL, [Col2] [nchar](10) NULL, ...
Chris's user avatar
  • 183
1 vote
0 answers
452 views

What is the function of microBatchDF._jdf.sparkSession().sql in this code? def upsert_to_delta(microBatchDF, batchId): microBatchDF.createOrReplaceTempView("updates") microBatchDF....
Sajid Sarkar's user avatar
  • 35
0 votes
1 answer
436 views

We are using CDC of a SQL Server database for the audit trail of an application. Data capture was enabled and works correctly, however the retention of the cleanup job was too short and deleted ...
Mareczek's user avatar
  • 316
1 vote
0 answers
190 views

Good day! I was using pgsync for change data capture (CDC) from postgres (wal=logical) to elasticsearch. What I noticed while using pgsync is that it captures all data from postgres on initial run. ...
ayb jax's user avatar
  • 161
3 votes
0 answers
914 views

I am trying to generate a hash based on multiple columns in a table. The columns could be of different types. I currently have some complex code that one needs to remember to update anytime another ...
Raj Rao's user avatar
  • 9,204
0 votes
1 answer
397 views

I am working on POC to migrate from Debezium to AWS DMS for PostgreSQL. PostgreSQL as Source and Kafka as Destination (AWS MSK). In Debezium i used to see before and after image for updates, but in ...
Pred's user avatar
  • 1
0 votes
1 answer
502 views

We have our application using the on-premise CDP (Cloudera) cluster for submitting pyspark jobs. Version of spark is 2.x We are now exploring the option to have CDC datasets processed and merge with ...
lbvirgo's user avatar
  • 404
0 votes
1 answer
292 views

While connecting to mongodb from goldengate big data(19c classic) GGSCI interface. I'm facing below error GGSCI (meteor-VirtualBox) 1> alter credentialstore add user mongodb://@meteor.rjzzi46....
Rajesh Lohar's user avatar
  • 68
0 votes
1 answer
420 views

I enabled change data capture and set the retention to 2 days. I have CU update 31 and above installed. Old CDC records are not deleting. The SQL Agent cleanup jobs are running successfully. What ...
JS JHA's user avatar
  • 1
1 vote
0 answers
287 views

I have a local instance of a kafka connect cluster. I am trying to replicate data from a DB2(v11.5.8.0) database. I am using debezium connector(v2.2.1) to capture a table in DB2 that contains a column ...
George Bernard Tan's user avatar
  • 11
0 votes
0 answers
826 views

I have a SQL Server instance, and a read-only replica of that instance that is used for ETL and analytics pipelines. The source instance has change data capture (CDC) enabled. What are best practices ...
Hugo's user avatar
  • 637

1
2 3 4 5 6