Do we need Debezium to write YugabyteDB CDC to Kafka?

General
1

In PostgreSQL, we usually use Debezium to write PostgreSQL CDC to Kafka.
Just want to confirm, we do not need Debezium because Yugabyte provides its own “Kafka Connect YugabyteDB Source Connector” for YugabyteDB CDC, correct?

2

Hi @Hongbo-Miao,

We’re actively working on CDC and Debezium integration atm. @suranjan_yb can add more details, but if there is a particular deployment you’re trying to achieve, we’re happy to talk to you about it.

1 Like
3

Thanks @karthik ! :smiley:

I am trying to do

YugabyteDB CDC → Kafka → Elasticsearch

especially YugabyteDB CDC → Kafka part.

However, after reading

I am still quite confused about how.

I remembered the page https://docs.yugabyte.com/latest/integrations/change-data-capture/cdc-kafka/ mentioned a “Kafka Connect YugabyteDB Source Connector”. However, due to the page is broken, I cannot check it currently.

Debezium is to help write PostgreSQL CDC to Kafka.

@suranjan_yb is helping Debezium part. Hmm, if we have “Kafka Connect YugabyteDB Source Connector”, do we still actually need Debezium?
Does that mean we gonna have two ways to write YugabyteDB CDC to Kafka? One way is using Kafka Connect YugabyteDB Source Connector, and the other one is using Debezium once @suranjan_yb finish?

BTW, the answer at cdc - How to capture YugabyteDB changes to kafka? - Stack Overflow
does not really answer its question. I commented under the answer.

4

Hi @Hongbo-Miao,
"Kafka Connect YugabyteDB Source Connector” is based on Kafka JDBC Source Connector and not a CDC Connector. It has its own inefficiency like (requires you to have auto-increment column, timestamp column in the table)
The above doc was for the beta version of our earlier CDC connector which we removed due to it being in active development and causing confusion due to the changed semantics.

We are working on our own “Kafka Connect Yugabyte CDC Connector” and will integrate the same with Debezium too as that is also based on Kafka Connect and eventually have it as one.

1 Like