Transactions and Data Consistency
The 3-Line Engineering Rule
Never perform a dual-write (e.g. write to Database AND publish to Kafka) in a single method without the Transactional Outbox pattern.
Dual-writes have no atomic guarantee; if the DB commit succeeds but Kafka drops connection, downstream systems fall permanently out of sync.
Persist the domain entity AND the outbound event into the same relational database in one ACID transaction, then use Change Data Capture (Debezium) or a reliable poller to publish.
Architectural Blueprint & Sequence
Hover over nodes to inspect state transitionsClient Request
POST /orders
Atomic ACID Tx
BEGIN TRANSACTION
orders Table
INSERT order (ID: 101)
outbox Table
INSERT event payload
Debezium CDC / Poller
Reads outbox table via WAL
Kafka Event Broker
Publish OrderCreated event
Key Engineering Takeaways
- Avoid 2-Phase Commit (2PC) in distributed microservices; it causes high latency and availability bottlenecks.
- Dual-writes are inherently broken: one side will eventually fail.
- The Transactional Outbox Pattern guarantees at-least-once message publishing.
- Consumers must be idempotent to handle duplicate deliveries gracefully.
The Problem & Production Reality
In a monolithic database, transactions are simple: `BEGIN TRANSACTION ... COMMIT`. In microservices, operations span multiple databases and message brokers. Attempting to write to a local database and immediately send a message to Kafka creates the classic Dual Write Problem.
Anti-Pattern: The Fatal Dual-Write
Writing to a relational database and immediately calling a message broker in the same Java method.
// ❌ ANTI-PATTERN: Dual write without atomicity
@Transactional
public void createOrder(Order order) {
orderRepository.save(order); // DB commit succeeds
// If Kafka broker is down or network blips here:
kafkaTemplate.send("order-created", new OrderCreatedEvent(order));
// Tx rolls back or leaves Kafka un-notified! Data inconsistency guaranteed.
}Production Pattern: Transactional Outbox with Change Data Capture
Save both the order and an outbox event in the same atomic database transaction. An asynchronous CDC pipeline (like Debezium) streams the outbox table to Kafka.
// ✅ PRODUCTION PATTERN: Atomic Transactional Outbox
@Service
public class OrderService {
private final OrderRepository orderRepo;
private final OutboxRepository outboxRepo;
@Transactional
public Order createOrder(CreateOrderCommand cmd) {
Order order = new Order(cmd.getCustomerId(), cmd.getItems());
orderRepo.save(order);
// Atomic write to outbox table within same ACID transaction
OutboxMessage outbox = new OutboxMessage(
"Order",
order.getId().toString(),
"OrderCreated",
JsonUtils.toJson(new OrderCreatedEvent(order.getId(), order.getTotal()))
);
outboxRepo.save(outbox);
return order; // Both committed atomically. Debezium or background poller will publish to Kafka!
}
}Trade-off Analysis & Staff Recommendations
| Aspect | Pros / Strengths | Cons / Hazards | Staff Recommendation |
|---|---|---|---|
| Debezium CDC vs Scheduled Outbox Poller | Debezium reads the database write-ahead log (WAL) with sub-second latency and zero DB polling query overhead. | Requires deploying and maintaining Kafka Connect and Debezium instances. | Use a simple scheduled SQL poller (`SELECT ... FOR UPDATE SKIP LOCKED`) for small systems; upgrade to Debezium for > 100 events/sec. |
| Saga Orchestration vs Choreography | Orchestration (central coordinator) keeps business workflow visible in one place. | Single coordinator service handles all transition decisions. | Use Orchestrated Sagas (Temporal or Camunda) for financial or multi-step checkout flows. |