Command Context Consistency

The missing piece to the Event Sourcing puzzle for all devs who want to believe, but still are confused

How to change an Event Store in a consistent manner? Unless this question has a very simple, concrete answer, Event Stores, Event Sourcing, Event Modelling will lack a lot of appeal to many, many developers.

There are answers to that question out there, but they are either too complex or too limiting for stressed out developers, I think.

• “Eventual consistency will take care of all consistency needs” is maybe correct and ultimately comprehensive, but it's impractical for every day use and mere mortals.

• “Check the Event Store version before appending new events. If it's not as expected, abort event recording.” sounds familiar to anyone who's ever used optimistic locking in RDBMS but is too broad brush. It just does not scale. Hence a more fine grained variant has been invented: “Check a stream's version before appending new events” with a stream only containing a subset of all events: the events pertaining to a single entity. But what to do if events to be recorded refer to more than one such stream?

My guess is, in order to find a conceptually simple and practical and widely applicable answer we need to step back for a minute. Event Sourcing has to be dissociated from the burden of object-orientation to which the notion of an entity is closely related.

RDBMS are based on a concept: the relational calculus. Only then were they implemented. Event Stores are lacking this grounding; they are children of pragmatism. Hence they are pretty tightly coupled to technology right from the start. That was great to get the idea off the ground; and it's perfectly fine for enthusiasts. But I think it's a hindrance for the average developers.

So, let me try and start at the beginning:

What's consistency anyway?

Why not just append new events to an Event Store? Because this might lead to inconsistencies in the state recorded in the Event Store.

The Hello World example for consistency is transferring money between two bank accounts. The relevant application state are two accounts with their balances, e.g. there are 100€ on Peter's account and 50€ on Janine's account.

If Peter is sending Janine 10€ a consistent state after the transfer is 90€ on Peter's account and 60€ on Janine's.

That's obvious - but what does consistency mean?

Constraints

Consistency means that rules are observed. State changes are subject to constraints. The rules in this example could be:

1. Both the sender's and the receiver's accounts need to exist, e.g. Peter and Janine both have a still an open account with the bank.

2. The amount to send must be larger than 0.

3. The sender's account needs to be funded enough, e.g. Peter's account balance: 100€ >= amount to transfer: 10€.

4. The total sum of money on all participating accounts is the same after the transaction, e.g. before: 100€+50€ = after: 90€+60€.

These rules and constraints usually are called business rules or business logic.

If and only if all rules are observed a state change may occur. That's the case in the above example. But if Peter's account balance was only 9€, rule #3 would be violated; the transaction could not be processed.

Command

Such business rules are applied when changing the application state. According to the CQS principle that's only happening during execution of a command request.

Whenever a command is triggered/received, its processor needs to observe the relevant “rules and regulations” before actually recording new events to update the application state.

The definition of a command for the money transfer scenario could look like this: transferMoney(sender,receiver,amount). And transferMoney("Peter", "Janine", 10) would describe the concrete transaction for the above example.

In case of success an event like moneyTransferred("Peter", "Janine", 10) could be recorded. Or two events: moneySent(“Peter”, 10, “abc”) and moneyReceived(“Janine”, 10, “abc”) with “abc” being a transaction ID.

Remember: CQS says, commands can succeed or fail, but do not return a result (except for an information about success/failure); queries on the other hand always succeed and deliver a result. If transferMoney should fail, that probably should be recorded, too, with e.g. moneyTransferFailed("Peter", "Janine", 10, "Insufficient funds").

Context

Depending on the command and its parameters/payload, more or less data has to be checked against the business rules to ensure consistency.

Sometimes no application state has to be consulted. Rule #2 can be check by just looking at the amount. However, that's a boring case when talking about consistency.

Most of the times, though, application state is involved in constraint checks. That's when things become interesting (and maybe controversial). In Event Sourced application that state resides in the Event Store; it has to be retrieved for the consistency check.

Here are some sample events along the lines of the banking scenario:

...
moneyWithdrawn("John", 250)
...
accountOpened("Peter", 5)
...
moneyTransferred("John", "Peter", 5)
...
accountOpened("Mary", 100)
...
interestRateSet("Peter", 0.3)
...
moneyTransferFailed("Mary", "John", 500, "Insufficient funds")
...
accountClosed("John")
...
accountOpened("Janine", 5)
...
accountAudited("Peter")
...
moneyDeposited("Peter", 90)
...
moneyDeposited("Janine", 45)
...

• When a command closeAccount("Mary") arrives, which events are relevant? Primarily it's accountOpened and accountClosed with reference to Mary. Constraint: Only if ever an account was opened for her and not closed then the current account can be closed.

• When a command transferMoney("Peter", "Janine", 10) is triggered, which events are relevant? Given the above list it's accountOpened, accountClosed, moneyDeposited, moneyWithdrawn, moneyTransferred: the existence of the account and the current balance need to be established and checked. Of course only Peter's and Janine's events are referenced in this case.

Context I call all the events relevant for a command during consistency check.

For both above commands the events interestRateSet, accountAudited, and moneyTransferFailed are irrelevant. They are not part of their contexts.

To retrieve the context events from the Event Store query criteria are applied during Event Store replay. It's like with RDBMS: execution of a query selects a subset of all data. In its simplest form it looks like this:

query = ...
context = eventstore.Replay(query)

How exactly the query is phrased, is a technical detail. Most important is what it consists of:

• a set of event types revelant to the context

• conditions for the properties of each event matching the event types specified, e.g. the account holder names

• as an optimization: a position in the Event Store from which to start replaying events, e.g. as an Event Store version or timestamp

What's missing? Any notion of streams, entities, aggregates, the pet concepts of many Event Sourcing enthusiasts. Why? Because I think they are optimizations and should not be part of a general concept for keeping state consistent.

To me the minimal event is looking somewhat like this:

{
  type: "moneyTransferred",
  payload: {
    sender: "Peter",
    receiver: "Janine",
    amount: 10
  },
  sequenceNumber: 282292,
  timestamp: 2025-06-23T11:21
}

• The top level properties state what all events share:

  • a type which corresponds to a certain payload structure

  • a position in the ever expanding space of the Event Store (sequenceNumber); this can also serve as a Event Store version

  • a position in time from when the first event was recorded until now (timestamp)

• Only the payload property differs between event types. It contains whatever you deem helpful in tracking the changing state of an application.

What's missing? Any trace of fixed IDs. Even though I believe, equipping data with an indentity (entities) if helpful, I do not think this should be forced on events. Events need to be “ID-agnostic”. Data identity is orthogonal to Event Sourcing.

Context model

What is a command supposed to make of a bunch of events, its context? The raw context as a subset of all events in the Event Store usually is of little help. What's needed is a projection, i.e. a data structure built from those events. This is a tailor made model to make appyling the “rules and regulations” most easy. I call this the context model.

A context model can have all shapes and sizes. It can be a single value or a complex network of objects. Yes, objects as in object-orientation. Why not? Anything goes! Each command is free to distill whatever it needs from the context.

This is the opposite of the Single Model™️ that is ruling over countless applications keeping them inflexible and complex. Call it what you like, domain model or business model or your central database: it’s like dragging a ball on a chain with you. Centralization of data might be necessary, but if that comes with a one-size-fits-all form/schema, your alarm bells should starting to ring.

With command specific context models Event Sourcing escapes the Single Model™️ Fallacy.

As for the above commands the context models could look like this:

closeAccount("Mary"):

{ accountOpen: true }

and

transferMoney("Peter", "Janine", 10):

{
  accountBalances: [
    { accountHolder: "Peter", balance: 100.00 },
    { accountHolder: "Janine", balance: 50.00 }
  ]
}

Based on these context models all necessary decisions can be made. Pure functions are enough for that even. No side effects happening. All necessary data from the application state in the Event Store is right at the fingertips of command execution.

The context model's sole purpose is to make consistency checks easy; for that they can be temporary, as short lived as command execution. That's contrary to the usual long lived domain models whose purpose is to keep data, to be the single source of truth regarding the application state.

With Event Sourcing the single source of truth is the Event Store. It's the “shared reality” of all parts of an application be that command handlers or query handlers. Or - if you like - of all feature slices in a Vertical Slice Architecture. The Event Store decouples all those parts; only the bare minimum is shared. Each part is free to interpret and project its events in any way it likes.

The Event Store holds all the facts pertinent to an application: all that has happened and was deemed worthy of recording. But this kind of granulate material (or “factlets”) is unwieldy for most purposes. Hence each request (command or query) focuses only on a subset, its context. And from the events in that context very specific models are built to help process requests.

• For queries the context model is the basis for what is returned as a response.

• For commands the context model is the basis for checking rules and applying constraints to assess which events should be generated.

Think of command execution as a pipeline:

build query >> replay >> build model >> check >> generate >> record

or a bit more concretely as pseudo-code:

execute_command(params) {
  query = build_query(params)
  context = eventstore.replay(query)
  model = build_model(context)
  assessment = check(model, params))
  events = generate(model, params, assessment)
  eventstore.record(query, context, events)
}

Pre-processing

All data necessary for processing the command is prepared:

• First, a query is set up to draw a boundary around a subset of the application’s state; only certain relevant events are taken into focus.

• Then the Event Store executes the query to retrieve this command context.

• Finally, the command context is projected into a command specific data structure to serve consistency checking: the context model.

Processing

Processing is the heart of handling a command. Consistency must be upheld! New facts are created.

• The rules are applied to the context model and the parameters, the constraints are validated; everything has to match perfectly.

• New events are generated according to the result of the consistency check.

Post-processing

The facts have to be persisted.

• Finally the events are recorded to update the application's state in the Event Store with new facts. This is where the real consistency magic happens.

Persisting consistency

Consistency is ensured in two phases:

• Phase #1 is during during command processing. That's the obvious task of a command: Check if all “rules and regulations” can be applied. Only if that's the case, the intended state change can be initiated by creating success events.

• Phase #2 follows and actually records the success events in the Event Store — if, well, if it hasn't assumed a conflicting state in the meantime.

Pre-processing and processing take time. Between finishing context event replay and storing success events, another distributed part of an application could have created new facts to be considered by the command. If the command fails to realize that, inconsistencies might arise in the Event Store by recording new events.

This must not happen! When recording new events it must be ensured that the prerequisites for them still hold.

This is ensured by making recording a two step, atomic process:

1. The query is executed again. Only if no new events appear in the context, nothing relevant has changed in the Event Store. New Events might have been recorded in the meantime, but as long as they do not match the query criteria, they don't belong to the context. They would not change the consistency check results.

2. Only if no context change is detected, the events generated by the command are actually recorded.

This is generalized optimistic locking for Event Stores. This, to me, was the missing piece in the Event Sourcing puzzle. Many developers have a hard time imagining persistence without a single schema. Dissolving the single data model into a myriad of state differences aka events? That takes quite a leap of faith. But to also give up cherished ACID transactions and optimistic locking is unimaginable. How to ensure data consistency in a straight forward manner? Thinking in streams and aggregates is too much to ask of them.

Event Sourcing seems promising from afar — but when approaching and finding out, consistency requires jumping through hoops, many say no to it.

Hopefully this will change with what I call Command Context Consistency (CCC) as described here. It's conceptually parsimonious, I think. It's a general solution with a minimum of concepts.

1. Check consistency against a context.

2. Ensure context stability upon event recording.

This last step I was missing until recently when Rico Fritzsche pointed me to an article on Dynamic Consistency Boundaries (DCB) . But after initial joy about DCB's intent to finally slay the DDD aggregate I found the approach too specific, too complicated. Nevertheless I took something important away from it: the two step recording of events for general optimistic locking.

More examples

This all might still sound abstract to you. So let me try to illustrate with a couple of more examples:

Grant a threshold discount

Imagine a company which wants to grant every 100th order in a day a discount of 10%. How to do that with CCC Event Sourcing?

• Command: createOrder(...)

• Business rules: If there are 99 or 199 etc. orders today, then the current order to be recorded should get a discount.

• Context model: To check the business, rule a very simple model is sufficient, e.g. { numberOfOrdersToday: 78 }.

• Context: The only relevant event type to generate the context model from is orderPlaced(). The query should find all such events with a timestamp of today.

In this example no IDs are required to identify relevant events.

Enroll student in course

In the DCB literature this scenario is discussed as to being difficult to align with the notion of aggregates. I agree. But I also venture to say: the solution of DCB - introducing tags for events - is too specific and not necessary.

• Command: enrollStudentInCourse(studentID, courseID), two entities are referenced

• Business rules: Each student may only enroll in 3 courses, and each course may only have 10 students enrolled.

• Context model: To check the constraints a very simple model is sufficient again, e.g. { nCourses: 2, nStudents: 10 }. If either business rule fails, the student is not enrolled in the source.

• Context: Relevant events are studentRegisteredInCourse(studentID, courseID) and studentUnregisteredFromCourse(studentID, courseID). All events where either ID matches the respective command parameter will be retrieved by the query.

In this example IDs are used to succinctly identify students and courses. Both are deemed to be entities.

However, the IDs are just part of the payload of the events. Nothing has to be invented for this scenario. Identity, as I've stated already, should be orthogonal to Event Sourcing, I think. No DCB tags necessary; it's all a matter of a pragmatic payload.

Checking inventory upon order creation

This also is a typical scenario making people scratch their heads when thinking about Event Sourcing. And it's a pet example for all fans of DDD aggregates: When creating an order with a number of items each linked to a product with an inventory, it should be checked if all items are in stock to be able to fulfill the order right away.

• Command: createOrder(customerID, orderItem{productID, quantity, ...}[]) with each order item referencing a product with its ID.

• Business rule: The available inventory of each product referenced should be larger or equal to the quantities demanded by the order items.

• Context model: This time the data structure is a bit more elaborate to make checking the business rule easy, e.g. [{productID: "abc", inventory: 10}, {productID: "987", inventory: 99}, ...]

• Context: Which events to take into account depends on the delivery process, I think. But these will do to illustrate the scenario, I guess: inventoryStockedUp(productID, quantity), inventoryReserved(productID, quantity), inventoryReleased(productID, quantity), quantityFoundDuringStocktaking(productID, quantity). Let me explain: At any time the inventory can be increased through new deliveries. When an order is created the required quantity for its fulfillment is reserved; if an order is cancelled, this quantity is released. Once an order is fulfilled more events might be recorded pertaining to the inventory, but that's not really relevant since the quantities had been reserved and were not available to the new order. Finally, it might come to light that the inventory is not what the system thought it was due to loss, quality problems etc. Then it has to be corrected.

In this scenario an arbitrary, possibly large number of IDs are relevant. But that does not require the invocation of the notion of an aggregate (DDD) or the introduction of tags (DCB). The query is the same for retrieving the context as for checking its stability when recording new events. To record an inventoryReserved() event for each product referenced in in order items is just fine. It's a granular way of updating the facts.

The command is the “agent” responsible for enforcing any business rules. If it should happen that several commands follow the same rules, some code or model sharing might seem in order. But to enforce a central “institution” like an aggregate right from the start to take care of all consistency checks to me sounds at least like a premature optimization.

Conclusion

The initial question that kept dampening my enthusiasm for promoting Event Sourcing more has found an answer. Consistency is ensured in a conceptually very straightforward manner:

• Define and retrieve the context for the command to be processed. Find all events relevant to its rules and constraints. Check them against a model projected from the context. And if all is ok, generate new events to be recorded.

• Before recording the new events check the stability of the context. If it's unchanged, append the new events to the Event Store. All in one atomic operation.

That's it. Easy to understand, isn't it. Very close to how consistency is handled when storing data in an RDBMS. The main difference being the lack of a query language for events. But as Rico showed in his article you can map a context query to SQL.

My hope is for Event Sourcing to become more popular, even mainstream now that the consistency question for many, many cases has found such a straightforward answer. Event Sourcing is such a boon for understandable, testable, flexible code structures and also team productivity. And last but not least think of the new features you can build on it because Event Sourcing is like a Black Box in an airplane, it's a time machine for your application's state.