Exploring Event Validation

Event schemas are at the core of any event-driven application.

They enforce standards and contracts between producers and consumers. Producers have a reasonability to publish the event in the correct format, and by subscribing these events consumers establish a contract between producer and consumer.

Having that contract between producers and consumers is important, and changes to that schema could break downstream consumers. So how can we protect ourselves?

I reached out on Twitter and LinkedIn to ask people their thoughts on event validation with event-driven architectures and got a wide range of thoughts and responses.

Event architecture: Event validation Question ❓

Do you validate your events?

Do you validate the event before it's published, or event when it's consumed. Or both? Or something else?

Interesting to see what people think 👀

— David Boyne 🚀 (@boyney123) July 29, 2022

This short blog post will summarise the responses, ideas that emerged and my thoughts.

Let's dive in.

Event Validation

There is a clear theme in the responses around event validation. That event validation should be done on producers and consumers but the type of validation varies.

Producer Validation

When publishing events, engineers are using synchronous validation methods to make sure the payload matches the event schema, this gives a level of confidence that the producer is producing what they agreed (the contract).

People are also using and exploring asynchronous methods to validate and control who can produce / subscribe to events on the bus, although it's worth noting it add latency and complexity (handling errors) it seems like shared pattern.

Luc van Donkersgoed has a great video around this topic worth checking out.

Consumer Validation

Consuming events requires a different type of validation.

Many engineers validate the events they receive, but only validate the required fields they need and ignore fields they don’t need.

When subscribing to events we naturally create a contract between producer and consumer. If we only validate the fields we are interested in we can reduce that coupling between contracts (vs validating the whole payload even if fields are not used).

Engineers are also exploring the idea of trusting the source.

If we trust the producer, can we skip validation? But how can we trust the source? It could depend on the locality of the producers and consumers.

Locality of producers and consumers

Quoting other people here: locality matters. The distance between the components must always be considered. At all levels. If contract validation makes sense there’s probably some distance between the components.

— Carlos Espin (@CAEspin) July 31, 2022

When consuming events within our architecture they can come from many places. Examples of these include internal domain events, events that cross bounded context / teams and 3rd party events.

If we want to use validation depending on the consumer/producer relationship, is there a method we can use? Some engineers believe it could be in the locality of the producer and consumer.

• If the producer and consumer are close (maybe local, or within the same bounded context) could we be relaxed on the validation / checks required?
• If the distance between producer and consumer is greater (across teams, multi bus topologies, 3rd party events) validation could make more sense?

Something worth thinking about.

Summary

When building software, we want confidence that our applications are going to work and scale over time. Event validation is pattern that developers are using, but they change the level of validation based on consumer, producer and trust between them.

Common patterns seem to suggest that we validate event structure when publishing events and verify/validate event fields in consumers.

It should be validated by both, if you strive for loosely coupled systems. For a Producer, to make sure the data and format is respected before being pushed. For a Consumer, to prevent invalid data being consumed, causing corruption or triggering unwanted processes.

— Thim (@timhaselaars) August 3, 2022

Locality seems like an interesting idea to consider, we see these patterns everywhere when dealing with APIS (internal or external), the level of checking / validation can change based on the trust you have with the producer. Understanding the trust and locality between domain events, bounded contexts, 3rd party events could be worth exploring.

I think it’s important to remember that producers and consumers are coupled by schemas / contracts. Consumers can come and go, but whilst they are consuming events the contract is bound.

Treating events as first class citizens is important and if validation gives you the confidence to scale your event architectures go for it!

If you have any thoughts, feel free to join the conversation on Twitter or LinkedIn, I would love to hear what you think.

Until next time!