Event-driven architecture: Kafka and the outbox pattern

Overview

Microservices break most often where they call each other synchronously: one slow service slows the whole chain, one failing service fails the others. In this training we build, end to end, an architecture where services share state through events and each service keeps a local copy of the data it needs. On our own platform, MicroFrame, synchronous calls between services are ruled out; the patterns and pitfalls in this training come from doing that work.

Course content

1. The cost of synchronous calls

• Cascading failure and added latency
• Temporal coupling: both services must be up
• When a synchronous call may stay
• Deciding to move to events

2. Modelling events

• Events, commands and queries
• Naming events in the past tense: OrderPlaced
• Which fields an event carries
• Thin events and state-carrying events
• Who owns an event contract

3. Kafka basics

• Brokers, topics and partitions
• Replication, ISR and min.insync.replicas
• Producer acknowledgements: acks and the idempotent producer
• Consumer groups, offsets and rebalancing

4. Ordering and keys

• Ordering is guaranteed only within a partition
• Choosing the key: the aggregate ID
• The hot-partition problem
• Lab: moving two services from a call to an event

5. The transactional outbox

• The dual-write problem: database and queue
• The outbox table, written in the same transaction
• Relays: polling versus CDC (Debezium)
• Forgetting to register an event type: silent loss
• Lab: an outbox with EF Core

6. The consumer side

• At-least-once delivery and duplicates
• The inbox table and idempotent handling
• Commit order: handle first, then the offset
• Ambient context in the consumer: tenant, user, system identity

7. Error handling

• Transient versus permanent errors
• Retries and backoff
• Dead-letter queues and reprocessing
• Poison messages and halt policies

8. Schema evolution

• Adding new fields as optional with defaults
• Removing and renaming fields
• Schema registry and contract tests
• Lab: growing an event without breaking old consumers

9. State propagation and local projections

• Compacted topics and tombstones
• A local copy in the consumer: no calls for reads
• Holding traffic until the projection is seeded
• Lab: replacing a synchronous check with a projection

10. Replay and backfill

• Bootstrapping a new service from history
• One-off backfills and marker tables
• One-off work across pods: leader election

11. Sagas and workflows

• Choreography and orchestration
• Request and reply events: XRequested and XCreated
• Compensating steps
• Timeouts and stuck flows

12. Operations

• Consumer lag and outbox backlog
• Topic settings: replication factor, retention, cleanup.policy
• Carrying the trace ID in messages
• Alerts and incident reviews

Schedule

Each module takes about 4 hours, half a day; 48 hours in total.

• Day 1: modules 1 and 2
• Day 2: modules 3 and 4
• Day 3: modules 5 and 6
• Day 4: modules 7 and 8
• Day 5: modules 9 and 10
• Day 6: modules 11 and 12

Prerequisites

• Experience building services with C# and ASP.NET Core
• Entity Framework Core and relational databases
• Running containers locally with Docker
• A laptop with Docker Desktop and the .NET 10 SDK

Training format

• On site or remote: at your office or in live virtual sessions.
• Hands-on: every day ends with lab sessions where you apply that day’s topic to a working example.
• Lab environment: participants get a prepared working environment and a sample code repository.
• Tailored to your team: content and examples are adapted to your stack after a short call before the training.
• Materials: slides, lab guides and reference code stay with you after the training.