What it is
Change data capture is a way to read the record of changes a database already keeps and publish those changes as a stream. Inserts, updates and deletes all pass through an internal log before they become visible. CDC turns that log into events other systems can consume: which row changed, what it looked like before, and what it looks like after.
Compare that with a nightly extract. A job dumps forty tables at 2am and the warehouse sits a day behind. Two edits to the same row collapse into one. Deletes disappear unless someone remembered to add a soft delete column. Change data capture reads the changes themselves, so those gaps close.
Keep the mechanism separate from the product. CDC is not a backup, a queue or a reporting tool. It is the capture step. Loading a warehouse, updating a search index, invalidating a cache or triggering a workflow are separate decisions with their own design.
How it works
Log-based capture is the default for serious work. Postgres calls it logical decoding and reads the write-ahead log. MySQL reads the binary log, SQL Server reads the transaction log, Oracle reads redo, and MongoDB exposes change streams. The capture process connects as a replica and tails the log, so read cost on the primary stays low compared with querying tables.
Two older methods still show up. Trigger-based capture copies each changed row into a shadow table. It works on engines with poor log access, but it adds a write to every transaction and needs a trigger on every table. Polling checks an updated_at column on a schedule. It is simple, and it misses deletes and any row changed twice between runs.
Production pipelines snapshot first, then stream. The tool takes a consistent read of existing rows, records the log position, and continues from there, so you get a full picture without stopping writes. Debezium is the common open source tool. Each cloud has managed equivalents.
After capture you need an event contract. Decide the payload shape, how tables and columns are named, and how schema changes such as a renamed column reach consumers. Delivery is usually at least once, so consumers deduplicate on primary key plus log position. Ordering holds per key, not across the whole stream.
Why it matters for a business
Fresh data changes what people can ask. A finance team that reconciles orders overnight can reconcile in a dashboard. An inventory team can keep two systems aligned in seconds instead of fighting a nightly sync that collides with daytime edits.
The load story matters too. Batch extracts hit the primary in a narrow window, which pushes teams toward read replicas, larger instances and maintenance windows. Change data capture spreads the same work across the day at low volume, and it stops every downstream team from writing its own extract query.
It also makes replacing a legacy system less risky. Run old and new side by side, stream changes from the old one into the new one, compare outputs, and cut over when the numbers match. Audit trails come almost free, since the before and after image of each row is captured.
Name the delay you are fixing before you build. A price change that takes a day to reach the storefront, an agent who cannot see a recent order, a risk check that runs after the transaction. That delay is the business case.
Common mistakes
Tables without a primary key. Capture needs a stable identifier to pair the before and after image and to deduplicate. Tables keyed on a nullable or mutable column cause trouble. Fix the keys, or leave those tables on batch.
Treating schema change as an afterthought. A renamed column in the source breaks every consumer that reads by name. Put a versioned event contract or a schema registry in place, and require review before DDL reaches production.
Ignoring retention and slot growth. In Postgres, a replication slot that no consumer reads keeps write-ahead log files on disk until the primary fills up. In MySQL the binary log expires on its own, so a consumer that falls behind loses events permanently. Alert on lag and on disk use.
Assuming exactly once and global order. Retries happen and duplicates arrive. Events for two different keys can reach a consumer out of order relative to each other. Build idempotent consumers that read the event timestamp, not the arrival time.
How to get started
Pick one flow with a visible payoff, such as orders into the warehouse or products into search. List the tables it touches, their primary keys, row counts, update rates and longest running transactions. Then check engine settings. Postgres needs wal_level set to logical and spare replication slots. MySQL needs row-based binary logging with full row images and a retention window longer than your snapshot takes.
Run the first pipeline against a non-production copy. Measure how long the snapshot takes and what it does to the source, then choose a window or throttle it. Put alerts on lag, event volume and error rate before go-live, not after.
Write the event contract and the schema change process while the pipeline is small. Agreeing naming and versioning with two consumers is far easier than with twelve.
Scope honestly. One pipeline is a few weeks of work for a team that has done it before. Ten pipelines across four engines plus a warehouse remodel is a program. Some teams start with a paid discovery. The Cloudverg Discovery and roadmap sprint is one option: fixed scope, two weeks, $9,500/month, with the fee credited to phase one if you start within 60 days. A managed connector service or building it in house with Debezium are others.
Frequently asked questions
Is change data capture the same as replication?
No. Replication copies data into another database, usually with the same schema. Change data capture hands events to whatever consumes them, which could be a warehouse, a search index or an application. Many tools use the same log reading mechanics for both.
Does change data capture slow down the source database?
Log-based capture adds little load because it reads a file the engine already writes. The heavy moment is the initial snapshot, which reads full tables, so schedule or throttle it. Trigger-based capture is different, since it adds a write to every transaction.
Do I need Kafka for change data capture?
No. Kafka and similar log services are common because they buffer events and let several consumers read the same stream. A small setup can write straight to a warehouse or object storage. Pick the simplest sink that meets your latency and replay needs.