Think Before Coding

To content | To menu | To search

Friday, April 11, 2014

Monoidal Event Sourcing

I Could not sleep… 3am and this idea…

 

Event sourcing is about fold but there is no monoid around !

 

What’s a monoid ?

 

First, for those that didn’t had a chance to see Cyrille Martaire’s  fabulous explanation with beer glasses, or read the great series of post on F# for fun and profit, here is a quick recap:

 

We need a set.

Let’s take a simple one, positive integers.

 

And an operation, let say + which takes two positive integers.

 

It returns… a positive integer. The operation is said to be closed on the set.

 

Something interesting is that 3 + 8 + 2 = 13 = (3 + 8) + 2 = 3 + (8 + 2).

This is associativity: (x + y) + z = x + (y + z)

 

Le last interesting thing is 0, the neutral element:

x + 0 = 0 + x = x

 

(N,+,0) is a monoid.

 

Let say it again:

(S, *, ø) is a monoid if

  • * is closed on S  (* : S –> S > S)
  • * is associative ( (x * y) * z = x * (y * z) )
  • ø is the neutral element of * ( x * ø = ø * x = x )

warning: it doesn’t need to be commutative so x * y can be different from y * x !

 

Some famous monoids:

(int, +, 0)

(int, *, 1)

(lists, concat, empty list)

(strings, concat, empty string)

 

The link with Event Sourcing

 

Event sourcing is based on an application function apply : State –> Event –> State, which returns the new state based on previous state and an event.

 

Current state is then:

fold apply emptyState events

 

(for those using C# Linq, fold is the same as .Aggregate)

 

Which is great because higher level functions and all…

 

But fold is event more powerful with monoids ! For integers, fold is called sum, and the cool thing is that it’s associative !

 

With a monoid you can fold subsets, then combine them together after (still in the right order). This is what give the full power of map reduce: move code to the data. Combine in place, then combine results. As long as you have monoids, it works.

 

But apply will not be part of a monoid. It’s not closed on a set.

 

To make it closed on a set it should have the following signature:

 

apply: State –> State –> State, so we should maybe rename the function combine.

 

Let’s dream

 

Let’s imagine we have a combine operation closed on State.

 

Now, event sourcing goes from:

 

decide: Command –> State –> Event list

apply: State –> Event –> State

 

to:

decide: Command –> State –> Event list

convert: Event –> State

combine: State –> State –> State

 

the previous apply function is then just:

apply state event = combine state (convert event)

 

and fold distribution gives:

 

fold apply emptyState events = fold combine emptyState (map convert events) 

 

(where map applies the convert fonction to each item of the events list, as does .Select in Linq)

 

The great advantage here is that we map then fold which is another name for reduce !

 

Application of events can be done in parallel by chuncks and then combined !

 

Is it just a dream ?

 

Surely not.

 

Most of the tricky decisions have been taken in the decide function which didn’t change. The apply function usually just set state members to values present in the event, or increment/decrement values, or add items to a list… No business decision is taken in the apply function, and most of the primitive types in state members are already monoids under there usual operations…

 

And a group (tuple, list..) of monoid is also a monoid under a simple composition:

if (N1,*,n1) and (N2,¤,n2) are monoids then N1 * N2 is a monoid with an operator <*> ( (x1,x2) <*> (y1,y2) = (x1*y1, x2¤y2)) and a neutral element (n1,n2)…

 

To view it more easily, the convert fonction converts an event to a Delta, a difference of the State.

 

Those delta can then be aggregated/folded to make a bigger delta.

It can then be applied to a initial value to get the result !

 

 

The idea seams quite interesting and I never read anything about this.. If anyone knows prior study of the subject, I’m interested.

 

Next time we’ll see how to make monoids for some common patterns we can find in the apply function, to use them in the convert function.

Tuesday, December 17, 2013

Generalized Conway's law

Disclaimer: This article is not just about Conway's lay which I'd call Classical Conway's law, it's already on Wikipedia. You'll find its Generalized form at the en of the post..

BuildStuff 2013 has been really inspiring. Talking with all those people from different CS culture always raise the level of the conversation.

 

Several times came Conway's law into the discussion, as Pieter Hintjens noticed in his blog post yesterday.

 Classical Conway's law

Conway's law is quite simple. It states that:

organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

 

À partir de l’adresse <http://en.wikipedia.org/wiki/Conway%27s_Law>

 

It's often seen as a joke to mock companies organization, like this famous drawing:

 

 

But it is more than that.

First, it can be a force to drive a change in the system.

 

When two parts are too strongly tied and mangled together, it's a good idea to move teams apart. Communication friction will increase, and eventually, teams will find way to decouple parts so that friction can be reduced.

 

Pieter's example of LSHS vs LSNS is totally an example of this.

 

Second, it is both ways.

A messy system structure reflects a messy team organization.

But we can go far beyond.

 

Generalized Conway's law

System should match the structure of any external concern it interacts with

 

Of course, the Conway's law is included here. There's a match between people building the system and it's structure.

 

But we can here also include design methods like Domain Driven Design which address a different concern the same way:

Align the system structure with the structure of the Domain you're working in/for

 

Using Domain Events and Event Sourcing is a way to make explicit something that is usually implicit in most systems: Time

 

CQRS will reflect the fact that outside the system, getting information and acting on it are two totally different things.

 

Architecture in general is also included: find the match between your system and non functional requirements.

  • Deployment
  • Security
  • Performance
  • Maintainability

At a lower level, it is also known that code clarity reflects the level of the developers understanding.

 

All those external forces will shape the system, and any mismatch in structure will create friction.

And any smell in the system is a sign that something is not understood or working well outside of the system.

 

Generalized Conway's law gives a higher frame of thought to find what's ok and what's wrong in all the dimensions of building systems in interaction with their environment.