Think Before Coding

To content | To menu | To search

Sunday, December 4, 2016

fck: Fake Construction Kit

Yeah it's christmas time again, and santa's elves are quite busy.

And when I say busy, I don't mean:

Santa's elves

I mean busy like this:

Santa's elves

So they decided to build some automation productivity tools, and they choose Santa's favorite language to do the job:

F# of course !

F# scripting

No body would seriously use a compiled language for automation tools. Requiring compilation or a CI server for this kind of things usually kills motivation.

Of course it is possible to write bash/batch files but the syntax if fugly once you start to make more advanced tools.

Python, JavaScript, Ruby or PowerShell are cool, but you end up as often with scripted languages with dynamic typing which you'll come to regret when you have to maintain it on the long term.

F# is a staticaly typed language that can be easily scripted. Type inference make it feel like shorter JavaScript but with far higher safety !

Writing F# script is easy and fast. Test it from the command line:

vim test.fsx

Then write:

printfn "Merry Christmas !"

press :q to exit

now launch it on linux with:

fsharpi --exec test.fsx

or on windows:

fsianycpu --exec test.fsx


The only problem is that typing the fshapi --exec this is a bit tedious.

Bash/Batch to the rescue

We can create a bash/batch script to puth in the path that will launch the script (for linux):

vim test

fsharpi --exec test.fsx

chmod +x test

or one windows

vim test.cmd

fsianycpu --exec test.fsx    

Done !

Better, but now we need to write a bash and/or a batch script for each F# script.

fck bash/batch dispatcher FTW !

We create a fck file (don't forget to chmod +x it) that takes a command

#!/usr/bin/env bash

# fck tool path
fckpath=$(readlink -f "$0")
# fck tool dir
dir=$(dirname $fckpath)

# packages if needed
if [ ! -d "$dir/fck-cmd/packages" ]
pushd "$dir/fck-cmd" > /dev/null
    mono "$dir/fck-cmd/.paket/paket.bootstrapper.exe" --run restore
popd > /dev/null

# script command if it exists
if [ -e $script ]
    mono "$dir/fck-cmd/packages/FAKE/tools/FAKE.exe" "$script" -- $@

# shell command if it exists
elif [ -e $shell ]
    eval $shell $@

# help
pushd "$dir/fck-cmd" > /dev/null
    mono "$dir/fck-cmd/packages/FAKE/tools/FAKE.exe" "$dir/fck-cmd/fck-help.fsx" -- $cmd $@
popd > /dev/null

and the batch version:

@echo off
set encoding=utf-8

set dir=%~dp0
set cmd=%1
set script="%dir%\fck-cmd\fck-%cmd%.fsx"
set batch="%dir%\fck-cmd\fck-%cmd%.cmd"

set "args="
if "%~1" neq "" (
  set args=%args% %1
  goto :parse
if defined args set args=%args:~1%

if not exist "%dir%\fck-cmd\packages" (
pushd "%dir%\fck-cmd\"

if exist  "%script%" (
"%dir%/fck-cmd/packages/fake/tools/fake.exe" "%script%" -- %args%
) else if exist "%batch%" (
pushd "%dir%\fck-cmd\"
"%batch%" %cmd% %*
) else (
"%dir%/fck-cmd/packages/fake/tools/fake.exe" "%dir%

Forget the paket part for now.

The bash take a command argument, and check whether a fck-cmd/fck-$cmd.fsx file exists. If it does, run it ! It also works with shell scripts name fck-$ or batch scripts fck-$cmd.cmd to integrate quickly with existing tools.

Fake for faster startups

When F# scripts start to grow big, especially with things like Json or Xml type providers, load time can start to raise above acceptable limits for a cli.

Using Fake to launch scripts takes adventage of it's compilation cache. We get the best of both world:

  • scriptability for quick changes and easy deployment
  • automaticly cached jit compilation for fast startup and execution

We could have written all commands in a single fsx file and pattern maching on the command name, but once we start to have more commands, the script becomes bigger and compilation longer. The problem is also that the pattern matching becomes a friction point in the source control.


At some point we have recuring code in the tools. So we can create helper scripts that will be included by command scripts.

For instance parsing the command line is often useful so I created a helper:

// culture invariant, case insensitive string comparison
let (==) x y = String.Equals(x,y, StringComparison.InvariantCultureIgnoreCase)

open System.Xml.Linq

module CommandLine =
    // get the command line, fck style...
    let getCommandLine() = 
        |> Array.toList
        |> List.skipWhile ((<>) "--")
        |> List.tail

    // check whether the command line starts with specified command
    let (|Cmd|_|) str cmdLine =
        match cmdLine with
        | s :: _ when s == str -> Some()
        | _ -> None 

We use the -- to delimit arguments reserved for the script. Since Fake is used to launch scripts, we can also include FakeLib for all the fantastic helpers it contains.

Here is a sample fck-cmd/fck-hello.fsx script that can write hello.

It uses FakeLib for the tracefn function and FckLib for getCommandLine.

You can call it with (once fck is in your Path environment variable):

fck hello Santa


A tool without help is just a nightmare, and writing help should be easy.

The last part of fck bash script lanch the fck-help.fsx script:

This script tries to find a fck-xxx.txt file and display it, or fallbacks to fck-help.txt.

For exemple the help for our fck hello command will be in fck-hello.txt:

fck hello [<name>]

Display a friendly message to <name> or to you if <name> is omitted.

Of course we can the pimp the fck-help.fsx to parse the txt help files and add codes for colors, verbosity etc.


Deployment is really easy. We can clone the git repository, and add it to $PATH.

Run the commands, it will automatically restore packages if missing, and lanch the script.

To upgrade to a new version, call fck update, defined in :

script=$(readlink -f "$0")
dir=$(dirname $script)

pushd "$dir" > /dev/null
git pull
mono "$dir/.paket/paket.bootstrapper.exe" --run restore
popd > /dev/null

or batch fck-update.cmd:

git pull

Yep, that's that easy

Happy Christmas

Using Santa's elves tools, I hope you won't be stuck at work on xmas eve ! Enjoy !

The full source is on github

val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
val set : elements:seq<'T> -> Set<'T> (requires comparison)

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.set
val not : value:bool -> bool

Full name: Microsoft.FSharp.Core.Operators.not

Thursday, December 17, 2015

Ukulele Fun for XMas !


This post is part of the F# Advent Calendar in English 2015 project. Check out all the other great posts there! And special thanks to Sergey Tihon for organizing this.

Hi something fun and not too technical for end the year !

As everyone knows, the favorite instrument of Santa Claus is Ukulele ! So let's play some music, and especialy some Ukulele !

First thing first, let's create functions for notes. We start with C at octave 0, and have a progression by half tones.

So C is 0, D is 2, E is 4.

Since there is only a half tone between E and F, F is 5.

F is 7, A is 9, B is 11, and we reach next octave at 12, which is C 1 :

open System

let C n = 12 * n
let D n = C n + 2
let E n = C n + 4
let F n = C n + 5
let G n = C n + 7
let A n = C n + 9
let B n = C n + 11 

For sharps and flat, lets define two functions that had and remove a half tone

let sharp n = n + 1
let flat n = n - 1

We can now create names for each note :

let Cd = C >> sharp
let Db = D >> flat
let Dd = D >> sharp
let Eb = E >> flat
let Fd = F >> sharp
let Gb = G >> flat
let Gd = G >> sharp
let Ab = A >> flat
let Ad = A >> sharp
let Bb = B >> flat

There is no E sharp or F flat because it is F and E respectively, same thing for B and C...

Will create a structure with a custome comparison/equality that doesn't take the octave into account by using a 12 modulus, this will prove usefull to work with chords:

type Note(note : int) =
    member __.Note = note
    override __.GetHashCode() = note % 12 

    override __.Equals other =
        match other with
        | :? Note as other ->
            note % 12 = other.Note % 12
        | _ -> false

    static member names = 
        [| "C"
           "B" |]
    member __.Display = 
        let name = Note.names.[note % 12]
        let octave = note / 12
        sprintf "%s %d" name octave

    override this.ToString() = this.Display
    interface IEquatable<Note> with
        member __.Equals other =
            note % 12 = other.Note % 12
    interface IComparable<Note> with
        member __.CompareTo other =
            compare (note % 12) (other.Note % 12) 
    interface IComparable with
        member __.CompareTo other =
            match other with
            | :? Note as other -> 
                compare (note % 12) (other.Note % 12)
            | _ -> 1 

    static member (+) (string: Note, fret: int) =
        Note (string.Note + fret)

let notes = Note

Ukulele Strings

A Ukulele has 4 strings.

The funy thing is that the 1st one is higher than the second one, where on most string instruments strings are in progressive order.

This is simply due to the limited size of the Ukulele, a low first string would not sound good, so it is adjusted to the next octave.

This gives use the following:

let strings = notes [G 4;C 4;E 4; A 4]


Instead of hard-encoding ukulele chords, we will compute them !

So a bit of theory about chords.

Chords are defined by their root note and the chord quality (major, minor).

The chords start on the root note, and the chord quality indicates the distance to other notes to include in the chord.

On string instrument, the order and the height of the actual notes are not really important for the chord to be ok. So we can use a note at any octave.

Now, let's define the chord qualities.

First, Major, uses the root note, 3rd and 5th, for instance for C, it will be C, E, G, which gives intervals of 0, 4 and 7 half tones from root.

let quality = notes >> Set.ofList

let M n = quality [n ; n + 4; n+7] 

Then, Minor, uses the root note, the lower 3rd and 5th. For C it will be C, E flat, G, so intervals of 0, 3 and 7 half tones for root.

let m n = quality [n; n + 3; n+7] 

The 7th adds a 4th note on the Major:

let M7 n = quality [n; n + 4; n+7; n+11 ]


As on a gitare, a ukulele has frets, places where you press the string with your finger to change the tone of a string.

0 usually represent when you don't press a string at all, and pinching the string will play the string note.

When pressing fret 1, the note is one half tone higher, fret 2, two half tone (or one tone) higher.

So pressing the second fret on the C 4 string give a D 4.

Our first function will try pressing on frets to find frets for notes that belong to the chord

let findFrets chord (string: Note) =
    |> List.filter (fun fret -> 
        Set.contains (string + fret) chord)
    |> (fun fret -> fret, string + fret)

The result is list of pair, (fret, note) that can be used on the strnig

The second function will explore the combinaison of frets/note and keep only those that contains all notes of the chords.

Ex: for a C Major chord, we need at least a C, a E and a G.

using frets 0 on string G, 0 on string C, 3 on string E, and 3 on string A, we get G, C, G, C.

All notes are part of the chord, but there is no E... not enough. 0,0,0,3 is a better solution.

The function explore all possible solution by checking notes on string that belong to the chord, and each time remove a note from the chord. At the end, there should be no missing note.

At each level sub solutions are sorted by a cost. Standard Ukulele chords try to place fingers as close to the top as possible. So lewer frets are better.

The cost function for a chords is to sum square of frets. If there is any solution, we keep the one with the lowest cost.

let rec filterChord chord missingNotes solution stringFrets  =
    match stringFrets with
    | [] -> 
        if Set.isEmpty missingNotes then 
            Some (List.rev solution)
    | string :: tail -> 
        |> List.filter (fun (_,note) -> 
            chord |> Set.contains note)
        |> List.choose (fun (fret,note) -> 
            filterChord chord (Set.remove note missingNotes) ((fret,note) :: solution) tail)
        |> List.sortBy(fun s -> 
            List.sumBy (fun (fret,_) -> fret*fret) s)
        |> List.tryHead

making a cord is now simple.

Compute the note in the chord using quality and root.

For each string, map possible frets the belong to the chord, then filter it.

let chord root quality =    
    let chord = quality (root 4)
    |> (findFrets chord)
    |> filterChord chord chord []
    |> Option.get

We can now try with classic chords:

let CM = chord C M

and the result is:

[(0, G 4); (0, C 4); (0, E 4); (3, C 5)]

Now C minor:

let Cm = chord C m

which is exactly what you can find on a tab sheet:

[(0, G 4); (3, D# 4); (3, G 4); (3, C 5)]
chord D m
chord A M
chord A m

chord G m
chord E M

Printing chords

To print chords, we will simply use pretty unicode chars, and place a small 'o' on the fret where we should place fingers:

let print chord  =
    let fret n frt = 
        if n = frt then 
    let line chord n  =
            |> (fst >> fret n)
            |> String.concat ""       
    printfn "┬┬┬┬"
    |> (line chord)
    |> String.concat "\n┼┼┼┼\n" 
    |> printfn "%s"

Let's try it

chord C M |> print

It prints


Another one

chord G M |> print

and we get


Playing chords

We can also play chords using NAudio.

You can find NAudio on

For simplicity I will use the midi synthetizer:

#r @"packages\NAudio\lib\net35\NAudio.dll"

open NAudio.Midi
let device = new MidiOut(0)
MidiOut.DeviceInfo 0
let midi (m:MidiMessage) =  device.Send m.RawData

let startNote note volume = 
    MidiMessage.StartNote(note, volume, 2) |> midi

let stopNote note volume = 
    MidiMessage.StopNote(note, volume, 2) |> midi

let sleep n = System.Threading.Thread.Sleep(n: int)

Now we can define a function that will play a chord.

The tempo is used as a multiplicator for a the chord length.

Longer tempo means slower.

For better result we introduce an arpegio, a small delay between each note. Don't forget to remove this time from the waiting length...

The direction indicate if the cords are strumed Up, or Down. In the Up case we reverse the chord.

type Direction = Dn of int | Up of int

let play tempo arpegio (chord, strum)  =
    let strings, length = 
        match strum with 
        | Dn length -> chord, length
        | Up length -> List.rev chord, length 

    |> List.iter (fun (_,(n: Note)) -> 
        startNote n.Note 100 ; sleep arpegio )

    let arpegioLength = 
        List.length chord * arpegio

    sleep (length * tempo - arpegioLength)

    |> List.iter (fun (_,(n: Note)) -> 
        stopNote n.Note 100 )

To strum a chord, we give a list of length, and a chord, and it will apply the cord to each length:

let strum strm chord =
    let repeatedChord = 
        |> (fun _ -> chord) repeatedChord strm

Now here is Santa Clause favorite song, Get Lucky by Daft Punk.

First the chords :

let luckyChords = 
    [ //Like the legend of the Phoenix,
      chord B m
      // All ends with beginnings.
      chord D M
      // What keeps the planets spinning,
      chord (Fd) m
      // The force from the beginning.
      chord E M ]

Then strum, this is the rythm used to play the same chord, it goes like, Dam, Dam, Dam Dala Dam Dam:

let luckyStrum = 
    [ Dn 4; Dn 3; Dn 2; Dn 1; Up 2; Dn 2; Up 2]

and the full song :

let getLucky =
    |> List.collect (strum luckyStrum)

And now, let's play it :

|> List.replicate 1
|> List.concat
|> List.iter (play 130 30)

And the tab notations for the song !

|> List.iter print



I hope this small thing was entertaining and that it'll get you into ukulele !

For excercise you can:

  • implements more chords
  • Better printing
  • add more liveliness and groove by adding some jitter to the strum...
  • add the lyrics for Karaoke !
  • try with other songs !
  • try the same for a 6 string gitar !

Now it's your turn to rock !

Wednesday, December 24, 2014

Speaking computers for more fun !

I didn't try it on mono, but it should also work with some tweaking, see details here

Xmas is a good time to surprise kids, and what's more fun than a talking computer ?!

Hello world !

Nothing's easier, and this kind of Hello World will appeal them to programming in a flash :

#r "System.Speech"
open System.Speech.Synthesis

let synt = new SpeechSynthesizer()
let say s = synt.Speak(s: string)

say "Hello world !" 

Of course, if you're french like me, it'll say this with an awful french accent - something like hélo ouorld !

But you can select a different voice if available by providing hints:

open System.Globalization
let english = CultureInfo.GetCultureInfo("en-US")

synt.SelectVoiceByHints(VoiceGender.NotSet, VoiceAge.NotSet, 1, english)
say "Hello world !"

Far better !

Can you beat it ?

Now, a talking fizz buzz, up to 100 ! Can you beat it ?

[1 .. 100]
|> (fun n -> 
    match n%3, n%5 with
    | 0, 0 -> "FizzBuzz"
    | 0, _ -> "Fizz"
    | _, 0 -> "Buzz"
    | _ -> string n )
|> List.iter say

Even harder !

Now with a recognizer, we can wait for voice user input.

The problem with the Grammar API is that it's totally mutable and not really DSL oriented. Let's correct that :

open System.Speech.Recognition

type Grammar =
    | Phrase of text:string * result: string
    | Lst of Grammar list
    | Alt of Grammar list
    | Repeat of min: int * max: int * Grammar

let rec build = function
    | Phrase (text, result) -> 
        // Just build the a single phrase

    | Lst grammars -> 
        // Append parts of grammars one after the other
        let builder = GrammarBuilder()
        |> build
        |> List.iter builder.Append

    | Alt alternatives -> 
        // Create alternatives
        let choices =
            |> build 
            |> List.toArray

    | Repeat(min, max, grammar) -> 
        // Repeat a part of the grammar
        GrammarBuilder(build grammar, min, max)

This is not a full DSL for speach recognition, you can look at all the GrammarBuilder methods to add more possibilities.. Even here, I'll use only Phrase and Alt.

Now, we need a recognizer and wire the grammar with functions that will be called when a part of the grammar is recognized or rejected. It is mandatory to set grammar's culture to the recognizer's culture. There's usually a single recognizer installed by default on your system and it uses installed system's culture. In my case, it'll be french.

let recog = new SpeechRecognizer()

let recognize grammar recognized rejected  = 
    let builder = build grammar
    builder.Culture <- recog.RecognizerInfo.Culture
    printfn "%A" recog.RecognizerInfo.Culture
    recog.LoadGrammar(Grammar builder)
    recog.SpeechRecognized |> Event.add (fun e -> recognized e.Result.Text (string e.Result.Semantics.Value))
    recog.SpeechRecognitionRejected |> Event.add (fun e -> rejected ())

We can then use this to create a little Christmass quizz thanks to the FSharp.Data FreeBase Type Provider !

We'll use free base to find a list of Actors who plaid Santa in movies.

For this, install the FSharp.Data NuGet:

nuget install FSharp.Data -o packages -x

The dll should be in .\packages\FSharp.Data\lib\net40\FSharp.Data.dll

#r @"packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data

let fb =FreebaseData.GetDataContext()

Let's build the grammar

let santaActorsFilms =
    fb.``Arts and Entertainment``
      .``Film characters``
      .``Santa Claus``
      .``Portrayed in films``
    |> (fun c -> c.Actor.Name, c.Film.Name)
    |> Seq.toList

let santaActorsGrammar =
    |> (fun (actor,film) -> Phrase(actor, film))
    |> Alt

Here is the function to call when an actor is recognized.

I tried to pass a discriminated union as a value, but even if the API uses an object, the documentation states that it has to be a bool, an int or a string. I used only strings here.

let recognized text value =
    say (sprintf "True ! %s was Santa in %s" text value)

Here is the function when the speech could not be matched with the grammar.

It is also possible to get the audio of the text in this case. I decided to ignore it due to time constraints.

let rejected () = say "No, Not a Santa !"

Now, let's run it !!

recognize santaActorsGrammar recognized rejected

At this point the speech recognition configuration should appear if it's the first time you use it.

Once done you should be able to try the quizz !

If your OS culture is not english, don't hesitate to use a local accent for actor's name !


I hope you had fun with this API, and that you'll want to tweak it for your own demo !

The full code - using FSharp.Formatting - is on my gist

Happy Christmass !

namespace System
namespace System.Speech
namespace System.Speech.Synthesis
val synt : SpeechSynthesizer

Full name: XMas fun.synt
Multiple items
type SpeechSynthesizer =
  new : unit -> SpeechSynthesizer
  member AddLexicon : uri:Uri * mediaType:string -> unit
  member Dispose : unit -> unit
  member GetCurrentlySpokenPrompt : unit -> Prompt
  member GetInstalledVoices : unit -> ReadOnlyCollection<InstalledVoice> + 1 overload
  member Pause : unit -> unit
  member Rate : int with get, set
  member RemoveLexicon : uri:Uri -> unit
  member Resume : unit -> unit
  member SelectVoice : name:string -> unit

Full name: System.Speech.Synthesis.SpeechSynthesizer

SpeechSynthesizer() : unit
val say : s:string -> unit

Full name: XMas fun.say
val s : string
SpeechSynthesizer.Speak(promptBuilder: PromptBuilder) : unit
SpeechSynthesizer.Speak(prompt: Prompt) : unit
SpeechSynthesizer.Speak(textToSpeak: string) : unit
Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

type string = System.String

Full name: Microsoft.FSharp.Core.string
namespace System.Globalization
val english : CultureInfo

Full name: XMas fun.english
Multiple items
type CultureInfo =
  new : name:string -> CultureInfo + 3 overloads
  member Calendar : Calendar
  member ClearCachedData : unit -> unit
  member Clone : unit -> obj
  member CompareInfo : CompareInfo
  member CultureTypes : CultureTypes
  member DateTimeFormat : DateTimeFormatInfo with get, set
  member DisplayName : string
  member EnglishName : string
  member Equals : value:obj -> bool

Full name: System.Globalization.CultureInfo

CultureInfo(name: string) : unit
CultureInfo(culture: int) : unit
CultureInfo(name: string, useUserOverride: bool) : unit
CultureInfo(culture: int, useUserOverride: bool) : unit
CultureInfo.GetCultureInfo(name: string) : CultureInfo
CultureInfo.GetCultureInfo(culture: int) : CultureInfo
CultureInfo.GetCultureInfo(name: string, altName: string) : CultureInfo
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int, culture: CultureInfo) : unit
type VoiceGender =
  | NotSet = 0
  | Male = 1
  | Female = 2
  | Neutral = 3

Full name: System.Speech.Synthesis.VoiceGender
field VoiceGender.NotSet = 0
type VoiceAge =
  | NotSet = 0
  | Child = 10
  | Teen = 15
  | Adult = 30
  | Senior = 65

Full name: System.Speech.Synthesis.VoiceAge
field VoiceAge.NotSet = 0
Multiple items
module List

from Microsoft.FSharp.Collections

type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IEnumerable
  interface IEnumerable<'T>
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_>
val map : mapping:('T -> 'U) -> list:'T list -> 'U list

Full name:
val n : int
val iter : action:('T -> unit) -> list:'T list -> unit

Full name: Microsoft.FSharp.Collections.List.iter
namespace System.Speech.Recognition
type Grammar =
  | Phrase of text: string * result: string
  | Lst of Grammar list
  | Alt of Grammar list
  | Repeat of min: int * max: int * Grammar

Full name: XMas fun.Grammar
union case Grammar.Phrase: text: string * result: string -> Grammar
union case Grammar.Lst: Grammar list -> Grammar
type 'T list = List<'T>

Full name: Microsoft.FSharp.Collections.list<_>
union case Grammar.Alt: Grammar list -> Grammar
union case Grammar.Repeat: min: int * max: int * Grammar -> Grammar
val min : e1:'T -> e2:'T -> 'T (requires comparison)

Full name: Microsoft.FSharp.Core.Operators.min
Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name:

type int = int32

Full name:

type int<'Measure> = int

Full name:<_>
val max : e1:'T -> e2:'T -> 'T (requires comparison)

Full name: Microsoft.FSharp.Core.Operators.max
val build : _arg1:Grammar -> GrammarBuilder

Full name: XMas
val text : string
val result : string
Multiple items
type GrammarBuilder =
  new : unit -> GrammarBuilder + 7 overloads
  member Append : phrase:string -> unit + 7 overloads
  member AppendDictation : unit -> unit + 1 overload
  member AppendRuleReference : path:string -> unit + 1 overload
  member AppendWildcard : unit -> unit
  member Culture : CultureInfo with get, set
  member DebugShowPhrases : string
  static member Add : phrase:string * builder:GrammarBuilder -> GrammarBuilder + 4 overloads

Full name: System.Speech.Recognition.GrammarBuilder

GrammarBuilder() : unit
GrammarBuilder(phrase: string) : unit
GrammarBuilder(alternateChoices: Choices) : unit
GrammarBuilder(key: SemanticResultKey) : unit
GrammarBuilder(value: SemanticResultValue) : unit
GrammarBuilder(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
GrammarBuilder(phrase: string, minRepeat: int, maxRepeat: int) : unit
GrammarBuilder(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit
Multiple items
type SemanticResultValue =
  new : value:obj -> SemanticResultValue + 2 overloads
  member ToGrammarBuilder : unit -> GrammarBuilder

Full name: System.Speech.Recognition.SemanticResultValue

SemanticResultValue(value: obj) : unit
SemanticResultValue(phrase: string, value: obj) : unit
SemanticResultValue(builder: GrammarBuilder, value: obj) : unit
val grammars : Grammar list
val builder : GrammarBuilder
GrammarBuilder.Append(value: SemanticResultValue) : unit
GrammarBuilder.Append(key: SemanticResultKey) : unit
GrammarBuilder.Append(alternateChoices: Choices) : unit
GrammarBuilder.Append(builder: GrammarBuilder) : unit
GrammarBuilder.Append(phrase: string) : unit
GrammarBuilder.Append(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
GrammarBuilder.Append(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit
GrammarBuilder.Append(phrase: string, minRepeat: int, maxRepeat: int) : unit
val alternatives : Grammar list
val choices : GrammarBuilder []
val toArray : list:'T list -> 'T []

Full name: Microsoft.FSharp.Collections.List.toArray
Multiple items
type Choices =
  new : unit -> Choices + 2 overloads
  member Add : params phrases:string[] -> unit + 1 overload
  member ToGrammarBuilder : unit -> GrammarBuilder

Full name: System.Speech.Recognition.Choices

Choices() : unit
Choices(params phrases: string []) : unit
Choices(params alternateChoices: GrammarBuilder []) : unit
val min : int
val max : int
val grammar : Grammar
val recog : SpeechRecognizer

Full name: XMas fun.recog
Multiple items
type SpeechRecognizer =
  new : unit -> SpeechRecognizer
  member AudioFormat : SpeechAudioFormatInfo
  member AudioLevel : int
  member AudioPosition : TimeSpan
  member AudioState : AudioState
  member Dispose : unit -> unit
  member EmulateRecognize : inputText:string -> RecognitionResult + 2 overloads
  member EmulateRecognizeAsync : inputText:string -> unit + 2 overloads
  member Enabled : bool with get, set
  member Grammars : ReadOnlyCollection<Grammar>

Full name: System.Speech.Recognition.SpeechRecognizer

SpeechRecognizer() : unit
val recognize : grammar:Grammar -> recognized:(string -> string -> unit) -> rejected:(unit -> unit) -> bool

Full name: XMas fun.recognize
val recognized : (string -> string -> unit)
val rejected : (unit -> unit)
property GrammarBuilder.Culture: CultureInfo
property SpeechRecognizer.RecognizerInfo: RecognizerInfo
property RecognizerInfo.Culture: CultureInfo
val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
SpeechRecognizer.LoadGrammar(grammar: Grammar) : unit
event SpeechRecognizer.SpeechRecognized: IEvent<System.EventHandler<SpeechRecognizedEventArgs>,SpeechRecognizedEventArgs>
Multiple items
module Event

from Microsoft.FSharp.Control

type Event<'T> =
  new : unit -> Event<'T>
  member Trigger : arg:'T -> unit
  member Publish : IEvent<'T>

Full name: Microsoft.FSharp.Control.Event<_>

type Event<'Delegate,'Args (requires delegate and 'Delegate :> Delegate)> =
  new : unit -> Event<'Delegate,'Args>
  member Trigger : sender:obj * args:'Args -> unit
  member Publish : IEvent<'Delegate,'Args>

Full name: Microsoft.FSharp.Control.Event<_,_>

new : unit -> Event<'T>

new : unit -> Event<'Delegate,'Args>
val add : callback:('T -> unit) -> sourceEvent:IEvent<'Del,'T> -> unit (requires delegate and 'Del :> System.Delegate)

Full name: Microsoft.FSharp.Control.Event.add
val e : SpeechRecognizedEventArgs
property RecognitionEventArgs.Result: RecognitionResult
property RecognizedPhrase.Text: string
property RecognizedPhrase.Semantics: SemanticValue
property SemanticValue.Value: obj
event SpeechRecognizer.SpeechRecognitionRejected: IEvent<System.EventHandler<SpeechRecognitionRejectedEventArgs>,SpeechRecognitionRejectedEventArgs>
val e : SpeechRecognitionRejectedEventArgs
property SpeechRecognizer.Enabled: bool
namespace FSharp
namespace FSharp.Data
val fb : FreebaseData.ServiceTypes.FreebaseService

Full name: XMas fun.fb
type FreebaseData =
  static member GetDataContext : unit -> FreebaseService
  nested type ServiceTypes

Full name: FSharp.Data.FreebaseData

<summary>Typed representation of Freebase data. See for terms and conditions.</summary>
FreebaseData.GetDataContext() : FreebaseData.ServiceTypes.FreebaseService
val santaActorsFilms : (string * string) list

Full name: XMas fun.santaActorsFilms
property FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.S: FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.Film_characterDataIndividualsIndexedS

<summary>An indexing of specific named individuals of type &apos;Film character&apos; in the web data store</summary>
module Seq

from Microsoft.FSharp.Collections
val map : mapping:('T -> 'U) -> source:seq<'T> -> seq<'U>

Full name:
val c : FreebaseData.ServiceTypes.Film.Film.PerformanceData
property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Actor: FreebaseData.ServiceTypes.Film.Film.ActorData

property Runtime.Freebase.IFreebaseObject.Name: string
property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Film: FreebaseData.ServiceTypes.Film.Film.FilmData

val toList : source:seq<'T> -> 'T list

Full name: Microsoft.FSharp.Collections.Seq.toList
val santaActorsGrammar : Grammar

Full name: XMas fun.santaActorsGrammar
val actor : string
val film : string
val recognized : text:string -> value:string -> unit

Full name: XMas fun.recognized
val value : string
val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
val rejected : unit -> unit

Full name: XMas fun.rejected

Sunday, April 27, 2014

Monoidal Event Sourcing Examples

Last time we tried to imagine how we could change usual event sourcing definition$ to get a monoidal approach.


Here are some examples to make it work in real life.

Monoids properties

We'll try to do it properly, and test that the proposed solutions are proper monoids.


Let's recap what defines a monoid:

  • closure of operation
  • associativity
  • neutral element

How to test it

the closure of operation is given by the signature of the operation. In F# it should have a signature like 'a -> 'a –> 'a which represents a function that takes to arguments of type 'a and return a result of the same type 'a.


For associativity, we'll use the following test function:

let isAssociative (*) x y z =
   ((x * y) * z) = (x * (y * z))

It take a function that will be called '' and three values x y z. It will then test the associativity of '' using FsCheck, a F# port of QuickCheck.


FsCheck test the given property for a wide range of values. We'll just have to provide the operator, and FsCheck will check it the associativity holds for a lot of values.


For the neutral element, will use the following test function:

let isNeutralElement (*) neutral x =
    x * neutral = neutral * x


Here we'll have to provide the the operator - called '*' inside the function - and the neutral element.

Trivial cases

There are some obvious trivial cases.


Let's try to follow the number of occurrences of a specific event in state. The number of time a user did



The map function is simply:

let map = function
    | TheUserDidSomething -> 1
    | _ -> 0


The combine operation is then really simple:

let combine acc v = acc + v

and the neutral element is obviously 0.


No need to check it with FsCheck, (N, +, 0) is a monoid..


Another a bit less obvious is when an event sets a value while others don't.


For instance, let's keep track of the last user address when a user moves.


For combination, we'll use the 'right' function, which always takes it's rightmost argument:

let right x y = y


The adress should, of course, not change on other events, and for that, we'll use an option:

let map' = function
    | UserMoved (user, newAddress) -> Some newAddress
    | _ -> None


The right function can then be tweaked to take the right argument only when it has a value:

let right' x y =
    match x,y with
    | x, None -> x
    | _, y -> y

right' has a signature of 'a option -> 'a option -> 'a option, so it's closed on operation.


It's associative since, whatever x y z, (x right' y right' z) return the last defined term, however composed.


None is the neutral element. Added to the left, it keeps what's on the right, added to the right, it keeps what's on the left.


We can check it with FsCheck:

Check.Quick (isNeutralElement right' None)
Check.Quick (isAssociative right')

A less trivial case

But what about mixing both ? Some events change the value by increments, while some other set a specific value ? Like a stopwatch that increments with a button to reset it.


Can we model this kind of thing as a monoid ?


We have an increment or a reset value, let's model it as a discriminated union:

type ChangeReset<'T> =
    | Change of 'T
    | Reset of 'T



A map function, that map events to state change, would be something like:

let map'' = function
    | TimePassed duration -> Change duration
    | ButtonPushed -> Reset 0.
    | _ -> Change 0.


The first two cases are a direct mapping, for other events, we use the Change 0. values that actually use the neutral element of the underlying monoid. Adding 0 will not change the value.


We have here an underlying monoid, here, talking about duration, we use numbers, with addition and 0.


But imagine we want to add items to a list, but sometime, reset the list to a specific one like the empty list.


we can define a high order combine operation like this:

let combine' (*) x y =
    match x,y with
    | Change xi, Change yi -> Change (xi * yi)
    | Reset xv, Change yi -> Reset (xv * yi)
    | _ , Reset yv -> Reset yv


It combines to changes as a new change using the underlying monoid operation - called '*' here. It combines changes as a change.


The second line states, that a value (Reset) combined with a change will apply the change to the value.


But the third line says that when a Reset value is on the right of anything, it overrides what's on it's left.

This operation is by it's signature closed on the ChangeReset<'t> type.


It's associative, because while combining changes, it has the associativity of the underlying monoid, and when combining Reset values it has the associativity of the right operation.


The neutral element is a change with the value of the neutral element of the underlying monoid.


We can verify it with FsCheck:

Check.Quick (isNeutralElement (combine' (+)) (Change 0))
Check.Quick (isAssociative (combine' (+)))

General structure

I had some remarks that we needed it to be a group and not only a monoid. The right' and combine function clearly don't define a group because elements don't have a inverse/opposite.


What would be the opposite of Reset 5 ? It set the value 5 whatever is on its left.


The idea is to take the set of state values S.


Then take the set of changes from states in S to other states in S that are represented by the events.


Make the union of it. SC = S U C. SC will contain more that valid aggregate states. It will also contain things that are not state, like a value that indicate that the state did not change.


But we will ensure the following things: the function that convert to changes will return items of C:

map: Event -> C 


The combine operation will be closed on SC, so it can accept States from S and change from C, and it will be closed on SC: combine:

SC -> SC -> SC


But it will also ensure that when a state is given on the left, it will also return a state:

combine:  S -> SC -> S


The right' operation ensure this. The state need a value at its start, you can add as many None on the right, it'll still have it's value, and any new value will return this value.


For the ChangeReset type, the State is represented by the Reset values -that are actual values- it's S, while changes are represented by the Change values that define C.


As long as a Reset value is given at the outmost left, the result will be a Reset value that is part of S.

With this, we don't need a group, but we can do with something that is only slightly more restraint than a monoid, only to keep the semantic of state.

But we need more that a single value !

Of course aggregate state is usually composed of more than a single value.


Let's start small and consider a pair.

type State<'a,'b> = 'a * 'b


If 'a and 'b are monoids we can easily combine them :

let combine'' (+) (*) (xa,xb) (ya,yb) = (xa + ya, xb * yb)

where + is the operation on 'a and * the operation on 'b.


The neutral element is simply

let neutral neutrala neutralb = (neutrala, neutralb)

You can easily check that combine is closed on operation, associative, and that neutral is the neutral element.


recursively, you can build a triple as a pair of a pair with a single element (('a,'b), 'c), then a quadruple and any tuple.


Since a structure - a record - is just a tuple with names, any struct with monoidal members is also a monoid.


And since in Event Sourcing, all the tricky decisions are done before emitting an event, applying the event should just be setting values, incrementing / decrementing values, adding / removing items to sets or lists, all those operation are monoidal, hence almost all aggregates should be convertible to monoidal Event Sourcing.


Of course it doesn't mean you should do Event Sourcing this way, but it was rather funny to explore this !


Have fun !

Friday, April 11, 2014

Monoidal Event Sourcing

I Could not sleep… 3am and this idea…


Event sourcing is about fold but there is no monoid around !


What’s a monoid ?


First, for those that didn’t had a chance to see Cyrille Martaire’s  fabulous explanation with beer glasses, or read the great series of post on F# for fun and profit, here is a quick recap:


We need a set.

Let’s take a simple one, positive integers.


And an operation, let say + which takes two positive integers.


It returns… a positive integer. The operation is said to be closed on the set.


Something interesting is that 3 + 8 + 2 = 13 = (3 + 8) + 2 = 3 + (8 + 2).

This is associativity: (x + y) + z = x + (y + z)


Le last interesting thing is 0, the neutral element:

x + 0 = 0 + x = x


(N,+,0) is a monoid.


Let say it again:

(S, *, ø) is a monoid if

  • * is closed on S  (* : S –> S > S)
  • * is associative ( (x * y) * z = x * (y * z) )
  • ø is the neutral element of * ( x * ø = ø * x = x )

warning: it doesn’t need to be commutative so x * y can be different from y * x !


Some famous monoids:

(int, +, 0)

(int, *, 1)

(lists, concat, empty list)

(strings, concat, empty string)


The link with Event Sourcing


Event sourcing is based on an application function apply : State –> Event –> State, which returns the new state based on previous state and an event.


Current state is then:

fold apply emptyState events


(for those using C# Linq, fold is the same as .Aggregate)


Which is great because higher level functions and all…


But fold is event more powerful with monoids ! For integers, fold is called sum, and the cool thing is that it’s associative !


With a monoid you can fold subsets, then combine them together after (still in the right order). This is what give the full power of map reduce: move code to the data. Combine in place, then combine results. As long as you have monoids, it works.


But apply will not be part of a monoid. It’s not closed on a set.


To make it closed on a set it should have the following signature:


apply: State –> State –> State, so we should maybe rename the function combine.


Let’s dream


Let’s imagine we have a combine operation closed on State.


Now, event sourcing goes from:


decide: Command –> State –> Event list

apply: State –> Event –> State



decide: Command –> State –> Event list

convert: Event –> State

combine: State –> State –> State


the previous apply function is then just:

apply state event = combine state (convert event)


and fold distribution gives:


fold apply emptyState events = fold combine emptyState (map convert events) 


(where map applies the convert fonction to each item of the events list, as does .Select in Linq)


The great advantage here is that we map then fold which is another name for reduce !


Application of events can be done in parallel by chuncks and then combined !


Is it just a dream ?


Surely not.


Most of the tricky decisions have been taken in the decide function which didn’t change. The apply function usually just set state members to values present in the event, or increment/decrement values, or add items to a list… No business decision is taken in the apply function, and most of the primitive types in state members are already monoids under there usual operations…


And a group (tuple, list..) of monoid is also a monoid under a simple composition:

if (N1,*,n1) and (N2,¤,n2) are monoids then N1 * N2 is a monoid with an operator <*> ( (x1,x2) <*> (y1,y2) = (x1*y1, x2¤y2)) and a neutral element (n1,n2)…


To view it more easily, the convert fonction converts an event to a Delta, a difference of the State.


Those delta can then be aggregated/folded to make a bigger delta.

It can then be applied to a initial value to get the result !



The idea seams quite interesting and I never read anything about this.. If anyone knows prior study of the subject, I’m interested.


Next time we’ll see how to make monoids for some common patterns we can find in the apply function, to use them in the convert function.

Saturday, January 4, 2014

Event Sourcing. Draw it

Here is a drawing to show the interaction between the Decide and Apply functions:

Tuesday, December 17, 2013

Generalized Conway's law

Disclaimer: This article is not just about Conway's lay which I'd call Classical Conway's law, it's already on Wikipedia. You'll find its Generalized form at the en of the post..

BuildStuff 2013 has been really inspiring. Talking with all those people from different CS culture always raise the level of the conversation.


Several times came Conway's law into the discussion, as Pieter Hintjens noticed in his blog post yesterday.

 Classical Conway's law

Conway's law is quite simple. It states that:

organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations


À partir de l’adresse <>


It's often seen as a joke to mock companies organization, like this famous drawing:



But it is more than that.

First, it can be a force to drive a change in the system.


When two parts are too strongly tied and mangled together, it's a good idea to move teams apart. Communication friction will increase, and eventually, teams will find way to decouple parts so that friction can be reduced.


Pieter's example of LSHS vs LSNS is totally an example of this.


Second, it is both ways.

A messy system structure reflects a messy team organization.

But we can go far beyond.


Generalized Conway's law

System should match the structure of any external concern it interacts with


Of course, the Conway's law is included here. There's a match between people building the system and it's structure.


But we can here also include design methods like Domain Driven Design which address a different concern the same way:

Align the system structure with the structure of the Domain you're working in/for


Using Domain Events and Event Sourcing is a way to make explicit something that is usually implicit in most systems: Time


CQRS will reflect the fact that outside the system, getting information and acting on it are two totally different things.


Architecture in general is also included: find the match between your system and non functional requirements.

  • Deployment
  • Security
  • Performance
  • Maintainability

At a lower level, it is also known that code clarity reflects the level of the developers understanding.


All those external forces will shape the system, and any mismatch in structure will create friction.

And any smell in the system is a sign that something is not understood or working well outside of the system.


Generalized Conway's law gives a higher frame of thought to find what's ok and what's wrong in all the dimensions of building systems in interaction with their environment.

Monday, July 29, 2013

Command Events and Context

A remark by @serialseb after my previous post :


First, here is some the distinction between a Command and an Event.

A command express what you want to do. Usually expressed with a verb in the present tense:

A event express what actually happened. Usually expressed with a verb in the past tense;

As you can anticipate, what you request is not always what actually happens:

A Command is still an Event ?

Let’s go a bit meta.

The fact that you decide you want to do something is something that happens in time. So it can also be considered as an event !

True… but what’s the usefulness of this ?

Someone who would like to study the correlation between your desires and their realizations will model both as events..

This can be the same in a tracking context. Track both causes and effects..

Should we just ignore the distinction then ?

Surely not !

It’s here again a matter of context. Of Bounded Context.

Let’s take a sample with different kind of concepts, Value Object and Entities, to highlight how the same thing can take different modeling forms in different contexts:

Money is usually modeled as a Value Object, which implies equality by Value. I give you $10 you give me $10, we’re even, even if I give you a bill and you give me some coins.

In the context of fraud tracking, this can be very different. Bills have a tracking number, and you can model it as entity.

If you make both context work together you’ll go through an Anti Corruption Layer for context mapping. You’ll not model all your accounting with entities for money because some other context models it this way !

In the case of Command and Events, some contexts can be interested in the fact that a decision was made, so this will be modeled as an event. But in the context where this decision happens, the decision is different from the actual outcome, and not being clear about this difference can lead to several issues..

For example if you model a withdrawal with  a Withdrawal concept: It is not clear whether it represe,ts the fact that you asked it to happen or the fact that is has actually been done.

Being explicit on it with WithdrawMoney / MoneyWithdrawn removes the ambiguity.

In the context of event sourcing, this distinction is very important as the previous article showed. As is the distinction between upstream events and internal events, since upstream events is input and will only produce things you’d like to happen, so are closer to commands from the system point of view.

Of course if your context is not about time and decision, don’t use these concepts that would be useless. You’re writing a compiler ? Forget about all this.

Should I have Command DTOs ?

not necessarily, a command can be modeled as a function/method call. So creating Command object is not mandatory.

It can still be useful for dispatch, storage for diagnostics, functional test etc.

Sunday, July 28, 2013

Event Sourcing vs Command Sourcing

Last month, I made a presentation about Event Sourcing - a shorter version of my DevoxxFr talk. After me, Etienne and Maher from Sfeir, did a presentation on the same subject and their architecture inspired by LMAX.

I immediately noticed their reference to this Event Sourcing page by Martin Fowler, and started to see several point of pain lurking in their long term maintenance…

I won’t make a flaming post against Martin Fowler that has written lots of interesting stuff. Even this article says nothing wrong. It just takes a way that can cause long term pains as expressed in the page itself…

Sourcing Events, but which Events ?

The article starts with a totally valid definition of Event Sourcing:

Capture all changes to an application state as a sequence of events.

The question then is… where do these events come from ?

In this case when the service is called, it finds the relevant ship and updates its location. The ship objects record the current known state of the ships.
Introducing Event Sourcing adds a step to this process. Now the service creates an event object to record the change and processes it to update the ship.

A C# sample a bit further to make it clearer:

class EventProcessor...
IList log = new ArrayList();
public void Process(DomainEvent e) {

As you can notice, the Event is produced before reaching the event processor…

Constrast this with the following version:

class Cargo
IList log = new List();
private State currentState;
public Cargo(IEnumberable events)
foreach(var @event in events)
Apply((dynamic) @event);
public void Arrive(Port port)
// logic to verify the action can be done
// based on current state and command parameters
if (IsAlreadyInPort) throw Exception();
// create an event of what happened with this action
// it should not mutate state,
// but it can capture external state when it arrives
// it's also based on current state and command parameters
var @event = new ShipArrived(id, port, DateTime.Now)
// apply change du to the event
// it should require only current state and
// events will be published to the rest of the system
// from there.. This is where further side effect will
// occure
private void Apply(ShipArrived @event)
// no decision should happen here !
currentState.Port = @event.Port;
currentstate.LastMove = @event.Time;

From a functional point of view this pattern can be build from two pure functions:

Command -> State -> Event list
State -> Event -> State

Here, stored event has been produced by the aggregate itself ! The output is stored.

Nice, but why should I care ?

After all, since Martin says the first version is ok… let’s go !

This would be without noticing several warnings in the rest of the article.

External Systems

from the same page:

One of the tricky elements to Event Sourcing is how to deal with external systems that don't follow this approach (and most don't). You get problems when you are sending modifier messages to external systems and when you are receiving queries from other systems.

Many of the advantages of Event Sourcing stem from the ability to replay events at will, but if these events cause update messages to be sent to external systems, then things will go wrong because those external systems don't know the difference between real processing and replays.

The second version doesn’t suffer this problem…

Because rebuilding the state (like done in the constructor) only use the Apply method (or the ApplyStateChange function in the functional version)..

This Apply method only works with internal state and produces no external side effects..

External Queries

Another problem arising with Martin Fowler’s proposal:

The primary problem with external queries is that the data that they return has an effect on the results on handling an event. If I ask for an exchange rate on December 5th and replay that event on December 20th, I will need the exchange rate on Dec 5 not the later one.

Here again, the second version doesn’t suffer the problem..

The data from the external system will be used to build the event. It can be directly stored in it (like the current time in the sample), but can also be used in a computation. For instance, command contains prices in USD, query an current rate from USD to EUR, compute a price in EUR and put it in the event.
The rate at the time of the computation is baked in the event ! No need to remember the rate value afterward, especially no need to complex external system gateway.

It could still be better for debugging purpose to put the used rate explicitly in the event.

But the second version intrinsically handles this issue gracefully…

External Interactions

Both queries and updates to external systems cause a lot of complication with Event Sourcing. You get the worst of both with interactions that involve both. Such an interaction might be a an external call that both returns a result (a query) but also causes a state change to the external system, such as submitting an order for delivery that return delivery information on that order. 

Problem solved by version 2….

Code Changes

So this discussion has made the assumption that the application processing the events stays the same. Clearly that's not going to be the case. Events handle changes to data, what about changes to code?
The third case is where the logic itself changes over time, a rule along the lines of "charge $10 before November 18 and $15 afterwords". This kind of stuff needs to actually go into the domain model itself. The domain model should be able to run events at any time with the correct rules for the event processing. You can do this with conditional logic, but this will get messy if you have much temporal logic. The better route is to hook strategy objects into a Temporal Property: something like chargingRules.get(aDate).process(anEvent). Take a look at Agreement Dispatcher for this kind of style.

Wooo… when I read this, it’s a red flag for me ! I never want to deal with this kind of problems !

Especially if they’re expected to happen for sure !

How does it go with the second version ?

Events are produced by the code that contains the logic. Before November 18, the events emitted where based on code that charge $10. After, the code charges $15.

When using the Apply method, it doesn’t have to know how much to charge, it’s already in saved events !

There is no need to keep an history of versions of domain logic - except in your source control !

It can even cop with changes far more complex that the one in this sample. In any case, all data needed to compute current state has been put in the event.

Correcting logic bugs

One of the advantages advanced by Martin Fowler, is that you can change how you take the decision after the fact.

But if an event is an event, it already happened, and there’s no way we can go back in time to change it. We wont be able to change external side effects anyway, so just accept it.

It’s still possible to apply compensations… like accountants. When they charged you to much, they don’t use a time machine to make has if nothing happened.. the just add a chargeback entry at the end of the ledger.

Command Sourcing ?

I call the pattern proposed by Martin Fowler Command Sourcing.

A Command is a request made to the system to do something. At this point a lot of thing can still happen. It can fail, it can be influenced by external state..

An event is something that happen and that cannot be changed.

You can protest that an Arrival Event is an event, not a command.

Sure, but for the system it’s an upstream event. Not something that happened in the system.

Where’s the difference in the second verions ?

The upstream version will go to a reactor that will produce an Arrive command (notice the present tense) inside the system.
The system will then produce a ShipArrived event (notice the passed tense). This event has been emitted by the system itself.

The Martin Fowler’s version takes a shortcut: bypassing the reactor emitting a command, but it is equivalent to sourcing commands.

Martin, this is a call to action !

Since a lot of people have read, and will read the entry on your web site, can you add something about the pattern described here to reduce the pain of people that will want to start with Event Sourcing ?

Thursday, April 4, 2013

C# Static interfaces - Take 3 - redirect

The actual post has been moved to a new address

C# Static interfaces - Take 3

You may believe it or not, but the post that drains most of the traffic of this blog, is the one about C# static interfaces !


In october 2009, I simply tried to imagine where the idea of C# static interfaces could lead us, and, since then, I have more viewed pages (> 15%) on this post than on my home page !


And since then, nothing moved in this area in the C# langage, and I don’t expect it to happen soon.


But some other thing happened…




Yes F# is out and running on almost all platforms, and it can do what I described in the previous post.


The thing is called Statically Resolved Type Parameters and is closer to C++ templates than from C# generics.


The trick is that you can define an inline function with statically resolved types, denoted by a ^ prefix. The usage of defined methods on the type is not given here by an interface, but by a constraint on the resolved type :

let inline count (counter: ^T) =
    let value = (^T: (member Count : int) counter) 

here , the count function takes a counter of type ^T (statically resolved).

The second line express that ^T actually should have a member Count of type int, and that it will call it on counter to get the result value !


Magic !


Now, we can call count on various types that have a Count member property like :

type FakeCounter() =
    member this.Count = 42;


type ImmutableCounter(count: int) =
    member this.Count = count;
    member this.Next() = ImmutableCounter(count + 1)


type MutableCounter(count: int) =
    let mutable count = 0
    member this.Count = count;
    member this.Next() = count <- count + 1

without needing an interface !

For instance :

let c = count (new FakeCounter())

True, this is compile time duck typing !


And it works with methods :

let inline quack (duck: ^T) =
    let value = (^T: (member Quack : int -> string) (duck, 3))  

This will call a Quack method that takes int and returns string with the value 3 on any object passed to it that has a method corresponding to the constraint.

And magically enough, you can do it with static methods :

let inline nextThenstaticCount (counter: ^T) =
    (^T: (member Next : unit -> unit) counter)
    let value = (^T: (static member Count : int) ()) 

this function calls an instance method called Next, then gets the value of a static property called Count and returns the value !

It also works with operators :

let inline mac acc x y = acc + x * y

notice the signature of this function :

acc: ^a -> x: ^c -> y: ^d ->  ^e
    when ( ^a or  ^b) : (static member ( + ) :  ^a *  ^b ->  ^e) and
         ( ^c or  ^d) : (static member ( * ) :  ^c *  ^d ->  ^b)

It accepts any types as long as they provide expected + and * operators.


The only thing is that a specific implementation of the function will be compiled for each type on which it’s called. That’s why it called statically resolved.


You can use this kind of method from F# code but not from C#.



No need for static interfaces in C#, use F# !

Wednesday, April 3, 2013

Time is Money! DevoxxFr

Last week I made a big 3 hour presentation (oops I forgot the pause..) at the DevoxxFr conference.


It was the second edition of this big Java oriented event in Paris, and a big success: 1400 attendees, 180 speakers for 160 presentations.


I’ve seen a lot of interesting talks there and met nice people ! I can sincerely recommend it to anyone, even not fluent in Java.


I posted the slides of my talk on SlideShare – french only.


I’ll post a link the the recorded video as soon as it’s available.


The slides don’t contain the details of the F# live coding of a Uno game, but it was quite similar to my SimpleCQRS F# implementation on GitHub.


Someone noticed it was a F# presentation in a JVM conference while there was no F# at the last TechEd…

Sunday, March 3, 2013

Entities and Repository injection - follow up


I have still new comments and there are some Stack Overflow questions on the subject that show the response to the question is still unclear…


Should you inject, or even use repositories in you entities ?

The response is two fold, and both go in the same direction, you shouldn’t.


Of course, we’re discussing design here, so if in your specific case you find reasons more important than those invoked here, there’s always a possibility to decide to go as you wish…


Injection and DDD patterns

Where talking Injection and Entities/Repositories here, so the first reason concerns what injection means in a Domain Driven Design environement.


The wikipedia definition of the dependency injection pattern only makes references to objects or components without defining explicitly which kind of object or which kind of component…


But a famous injection anti pattern can give use more information on what are injected components.

I call the Service Locator anti pattern.


This make it clear that what we are looking for is a way to find service in a supple and configurable way.

I won’t explain here why the service locator is an anti pattern, but it makes it obvious that the use of injection is to provide access to services in the code.


Since services are supposed to be stateless, their lifecycles are usually quite flexible. A service could be instanciated on each call, on each request or as a single instance without much difference. The injected service dependencies enable to compose services to replace implementations by configuration or for tests..

But even at runtime. A live configuration change could indicates to replace an injected service on next instantiation.


Services and repositories are quite obviously not injected with Entities/Aggregates:

  • A repository is here to instantiate entities, it’s its own job, it doesn’t need to be injected with entities…
  • When a service needs an entity, it can be injected with a repository, and get the entity through it.

But could entities be injected with services or repositories ?


An aggregate or an entity is not stateless as a service, it is statefull, and its member fields are here to maintain current state.

It seems quite odd to use those fields to store references to services that are obviously not part of the state.

It also links the referenced service lifecycle to the lifecycle of the entity itself.


Most entities instanciation schemes in web applications are on a per call basis, and since most web application don’t do anything between calls, the response to the lifecycle problem is simply that everything in created and destroyed on each call.

But it is only a specific simplistic way to consider object lifecycle. What if you kept your entities longer in memory and reused them on next call to avoid roundtrips with the data storage ?

Having a reference to a service inside an entity state would make the live reconfiguration far harder to achieve : You would have to trash all current state and reload everything. There is no such problem with services since they’re meant to be stateless.


Entities fields are meant to store state, no to reference services. Their lifecycles should not be coupled.


Consistency boundary


The second reason is about the aggregate consistency boundary.


Why would you typically need a reference to a repository inside an aggregate ?


First reason would be to load sub entities when needed. It’s also called delay load.


You shouldn’t need delay load in aggregates

The reason comes from the aggregate definitions. From wikipedia:

Aggregate: A collection of objects that are bound together by a root entity, otherwise known as an aggregate root. The aggregate root guarantees the consistency of changes being made within the aggregate by forbidding external objects from holding references to its members.


The definition talks about object being bound together. Those objects – entities – have strong relationships, or grouping them in an aggregate makes little sense. When the relation is strong, you will need for each

aggregate method to change all objects in coordination, or none. If not, parts of your aggregate should not be here.

If an aggregate needs all its state for each operation, there is strictly no reason to use delay load.

Load all, use all, save all.


The second reason would be to load entities in other aggregates.

You shouldn’t need references to entities in other aggregates

or at least not directly. The service calling the method on the aggregate will find information needed to call the method (which can contain information coming from other aggregates), but if you need a direct reference to another entity, it’s a clear indication that the aggregated boundary is wrong ! Put that other entity in the aggregate !


Thus once your aggregate modeling is ok, there is really no reason to use a repository inside an entity.

Thursday, June 21, 2012

SimpleCQRS the F# version

Here it is ! As promised after greg’s talk at dddx  (you can already see all the presentations online !), a F# version of SimpleCQRS, a simple, quick EventSourcing + CQRS sample to see it in action.


Why rewrite it in F# ?


This is not just a simple copy of the C# version. The point was first to write it in a functional language, because event sourcing is inherently functional.


In C#, an aggregate method looks like this :


public void CheckIn(int count)
   if(count <= 0)
        throw new InvalidOperationException(
              "must have a count greater than 0 to add to inventory");
   ApplyChange(new ItemsCheckedInToInventory(_id, count));


The ApplyChange method is defined in the AggregateRoot base class.

It dispatches the event to a state denormalizer and add the Event to uncommitted changes.


Here is a denormalizer to apply a state change following an event :


private void Apply(InventoryItemDeactivated e)
   _activated = false;


All this is fine, but why should the aggregate be mutable when the event stream is highly a append only store of immutable events.


Watch greg’s talk carefully, both methods can be transformed to an immutable equivalent easily.


First the goal of the public method is to determine which event to raise based on command parameters and current state. The CheckIn method can be defined by the following method signature:

int –> State –> Event  or Func<int,State,Event>


Instead of calling an Apply change internally, the method simply returns an event :


let checkIn count s =
    if count <= 0 then
       raise (InvalidOperationException
           "must have a count greater than 0 to add to inventory")
       fire {ItemsCheckedInToInventory.Id= s.Id; Count = count}


where the fire function simply creates a Event array from a single event :


let fire o =
    [o :> Event]

(notice the [o:> smiley here !)

An array is returned here so that a method can fire/return several events using a simple :: syntax.


The state application can also become immutable. The function gives next state based on event and previous state :


State –> Event –> State or Func<State,Event,State>


let applyOnInventoryItem s (e: Event) =
   match e with
   | :? InventoryItemCreated as e -> {Id = e.Id; Activated = true }
   | :? InventoryItemDeactivated as e -> {s with Activated = false; }
   | _ –> s

no need for several methods here, every thing is straight forward. Match the event using its type.

The first event is a creation, so a state record is creates.

The second events copies state s with Activated set to false. No change occurs here, a copy is returned.

The _ match specifies that any other event simply return previous state.




The current state is a left fold of passed events

Sure and the replayWith method is simply here to do this :


let replayWith  =


let replayInventoryItem =

    replayWith applyOnInventoryItem { Id = Guid.Empty; Activated = false}


the replayInventoryItem is a function that takes a Event seq aka IEnumerable<Event>. It will start with the empty state, then for each event, call the applyOnInventoryItem function with previous state, current event, and iterate with new state.


The result is the current state.


Command handlers

The event handlers use the following functions :


let load id =
   eventStore.GetEventsForAggregate id |>

let save = eventStore.SaveEvents

let applyOn id version f =
   load id |>
   f |>
   save id version

load simply pass events for the aggregate with identifier id to the replayInventoryItem

save is simply a short cut for the event store SaveEvents method

applyOn loads an aggregate to current state, pass state to f, a function that returns an event seq, then save it to the event store.


Here is a sample of its use :


member x.Handle (c: CheckInItemsToInventory) =

   checkIn c.Count |>

   applyOn c.InventoryItemId c.OriginalVersion

The checkIn function actually expect an second State argument, it signature is int –> State –> Event seq


After passing the c.Count integer argument this is now a State –> Event seq function.

When passed to the applyOn function, the state issued from the load call will be passed to it, resulting in an Event seq that will be passed to the save function.



There are a few things to notice in this implementation.

  1. It is very short, much shorter that C# version.
  2. There is no InventoryItem class. The InventoryItem module contains a State record, the representation of the aggregate internal state, functions to determine raised events, and functions to determine next state based on previous state and event. No base class is needed for event dispatch and uncommitted event handling.
  3. There is no repository. Actually the load and save methods do what a repository does, but it’s so simple that no class is required.

I did not talk about the read model that is a bit less interesting here.

Thursday, June 14, 2012

Make it simpler : get rid of Mocking Fx

In the wake of Rinat Technology Demons, I’d add Mocking frameworks to the list.


There are several reasons to do that.



One of the usage of Mocking - or isolation – frameworks is stubbing. A component that access external resources or has complex behavior will be changed for a fake to have simpler predictable results.


  • When the component interface is small, the stub declaration code is often more complicated that writing a fake implementation directly.
  • When the interface has many methods, it makes things easier, but here you’ll ignore a code smell. It makes things easy in situation you should not reach !

There are also easy patterns that make the use of mocking irrelevant. A common one is configuration loading.

Lots of developers think: “I need a class to access my configuration. Let’s make a class for this”:


    public class Configuration


        public string EndPoint


            get { return ConfigurationManager.AppSettings["EndPoint"]; }



        // other properties here//



Then they realize they’ll have to isolate from ConfigurationManager, so they add a interface :



    public interface IConfiguration


        string EndPoint { get; }



    public class Configuration : IConfiguration


        public string EndPoint


            get { return ConfigurationManager.AppSettings["EndPoint"]; }



        // other properties here//


And they can moke the configuration using a lot of boring code.


what about this ?



    public class Configuration


        public string EndPoint { get; set; }

        // other properties here//



    public class ConfigurationLoader


        public Configuration Load()


            return new Configuration


                    EndPoint = ConfigurationManager.AppSettings["EndPoint"]

                    // other properties here//





Passing a different configuration in your test is a simple new Configuration {}… and you can use the ConfigurationLoader class in you main function.


A good thing here, is that you don’t suffer a read of the configuration file at each call without needing to introduce lazy loading or other tweaks that make things worse..



When you need to see if a call happened, you can make an implementation of the interface that takes a lambda :


    public interface IFancyService


        void DoSomething(int arg);



    public class MockFancyService : IFancyService


        private Action<int> doSomething;


        public MockFancyService(Action<int> doSomething)


            this.doSomething = doSomething;



        public void DoSomething(int arg)






    public class Test


        void DoSomthingHasBenCalled()


            bool wasCalled = false;

            var fancyService = new MockFancyService(i => { wasCalled = true; });


            var compoenent = new ComponentThatUseFancyService(fancyService);



            Assert.That(wasCalled, Is.True);



Or make the fake class record the call, and add a getter to check. It’s not that complicated.

Monday, March 5, 2012

Caching done right

I was trolling on twitter Saturday, when I saw tweet by Nate Kohari and some answers :



I immediately thought :

If you have one problem and use cache to solve it, you now have two problems.


Where’s the problem ?


The time to retrieve the data is not negligible due to frequency of request and/or time taken by calculation + data access. So we put data to cache so that we don’t have to endure this time on each call.


But then comes the problem of cache expiration:

  • We can use a duration.. but what is the acceptable delay ?
  • We can make a check to original data to check if it has changed. It’s more accurate, but incurs a new data access.

Moreover, checking if it changed is often not enough, we also need to find what changed.


And deriving what happened from state is basically reverse engineering. I’m an engineer. I prefer forward engineering.



Let’s do it forward


It’s actually easy, it’s the whole point of CQRS.


Let’s build a system that raises Domain Events, and we can denormalize events to a Persistent View Model.


We just have to listen to events and change the representation we want to send to the users:

  • The events contain everything we need to do fine grained updates easily.
  • We can can compute denormalizations asynchronously if it’s time consuming
  • We can store it in a relational database, a document database, or in memory
  • We can choose any form of denormalization since it’s a denormalization (object graph, but also flat string, json, html …)
  • It will be up to date quickly because it will be updated when the original data changed
  • The first client that makes a request after a change will not endure a cache miss that can be long to process since computing is done on change, and not on request.

A good way to Keep It Simple, Stupid!

Sunday, February 26, 2012

NuRep your local NuGet+symbols+source repository

As some of you already know, I'm a proponent of DRY : Do Repeat Yourself, but code reuse has some value and nuget is a good way to manage it.


So far, the advantages I see in using a package manager are:

  • easy get/update of external projects and their dependencies
  • on demand dependency update (instead of forced dependency update)
  • makes it easier to modularize dependencies

All this thing are out of the box when using OSS projects published on, and you can host your own nugets using NuGet.Server.


The debugging story is also quite good. can host symbols and source packages and be used as source server. You can directly step in your favorite OSS source code without having to compile it yourself. They even provide private repositories.


But sending your company's source code to an external service is not always compatible with internal policy.


In this configuration, using your own nugets leads to a poor dev experience when you have no way to step in your own code : compiled in Release and potentially not your latest code version, indicating the source code in another directory will not give good results.


You need a source server.


NuRep to the rescue


NuRep is a nuget repository based on NuGet.Server but it is also a symbols + code server.


When creating your nuget package, specify the –Symbols flag, and nuget will create a .nupkg and a .symbols.nupkg that you can push to NuRep (http://myserver/nurep/api/v2/package).


Then configure visual studio's symbols servers (Tools / Options / Debugging / Symbols / add http://myserver/nurep/symbols )


Don't forget to enable source server in Debugging / General options, and to disable Just my code.


That's it.


Now you'll step into the exact code that was used to compile the nuget version.


Have fun !

Friday, December 2, 2011

I love SQL Server and cultures... NOT !

When developing a large system, all is not unicorns and rainbows.

For now, everybody was working on a single SQL dev server and friction is high.


That’s why I’m working on SQL scripts management with mercurial and powershell to the rescue, so that any developer can trash his own SQLExpress instance, and rebuild everything needed in a single command. (I’ll maybe blog about all that later).


We have loads of stored procs.. I know people don’t like it, but it acts as a strong sanity layer when the database schema is so ugly your eyes bleed when you look at it.


Yesterday, I run a stored proc, and I get the following error :

The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.


Why the f**k.


The procedures is using a scalar function :

ALTER FUNCTION [dbo].[DateMaxValue]()
RETURNS datetime
RETURN '9999-12-31 23:59:59.998'


It’s working on other servers.. why doesn’t it work here.

After several tries, I try with the date ‘9999-12-01’ and I get the following date:

Year: 9999

Month: 01

Day: 12


Yes.. the date is interpreted as YYYY-dd-MM on a French server.


Even when you use the YYYY-??-?? format, SQL Server still try to mess with culture month/day ordering !


You can use the SET DATEFORMAT dmy or SET DATEFORMAT mdy to change this, but it will apply only in current session, and you cannot use it in a stored proc.


You can change the server culture, but it wont change anything. The dmy/mdy setting is ultimately in the Login culture.


You read it right :

  • For an English Login the function above works.
  • For a French Login the function above fails miserably.

There is no way to my knowledge to specify a strict date parsing in a stored proc or function.


So generates your logins with scripts, and enforce the same culture for all logins.


It’s just profoundly broken.

Tuesday, June 21, 2011

Event Sourcing and CQRS, Dispatch options 2

In the part one comments, Clement suggested a more efficient solution than registering handler in constructor.


The proposed solution is to have a RegisterAllEvents virtual method in which event handler registration would occur. This method is a method instance to have access to this but will be called only once per class. The registration use Expression<Action<T>> to access the expression tree and extract the method info of the handler. This enables type checking, make R# happy – no unused methods – and make reflection not too painful.


Good solution.


I didn't go that far because with Event Sourcing, you usually keep aggregates in memory, so aggregates are instantiated once per service lifetime.
I just crafted a small performance test :



using System;
using System.Collections.Generic;
using System.Diagnostics;

namespace AggregatePerfTest
class Program
static void Main(string[] args)
var watch = new Stopwatch();

const int count = 10000000;
Guid id = Guid.NewGuid();

for (int i = 0; i < count; i++)
new AggregateRegisteredOncePerInstance(id);




for (int i = 0; i < count; i++)
new AggregateRegisteredOncePerClass(id);




public class AggregateRegisteredOncePerClass
private readonly Guid id;

private static readonly object ClassInitLock = new object();
private static bool initialized;

public AggregateRegisteredOncePerClass(Guid id)
{ = id;

lock (ClassInitLock)
if (!initialized)
initialized = true;
// registration happens only once here

public Guid Id
get { return id; }

public class AggregateRegisteredOncePerInstance
private readonly Guid id;
private readonly Dictionary<Type, dynamic> handlers =
new Dictionary<Type, dynamic>(5);
public AggregateRegisteredOncePerInstance(Guid id)
{ = id;

public Guid Id
get { return id; }

public void DoSomething()

private void OnSomethingHappened(int message) { }
private void OnSomethingHappened(double message){ }
private void OnSomethingHappened(float message) { }
private void OnSomethingHappened(long message) { }

protected void Apply<T>(T @event)
handlers[typeof (T)](@event);

protected void Register<T>(Action<T> handler)
handlers.Add(typeof(T), handler);

The code is straight forward, I just created two aggregate classes :

  • one with registration in .ctor based on this post code
  • one without any registration at all, considering that doing it once is the same as not doing it for large numbers, but I added a lock section with a boolean check to simulate what will done on each instance creation.

I created 10.000.000 instances for each, and you get:

  • 3978ms for the one with .ctor registrations,
  • 377 ms for the one without.

It's true that it makes a difference. But how many aggregates do you have in your system ?


With 10.000 aggregate you're still under 8ms. I think you can afford that.


This is then a trade-off between performance and simplicity :

  • If you have very large numbers, go for expression tree parsing, class lock management etc.
  • In any other situation I recommend using registration in .ctor that makes the code easy to implement in approximately 5min.

Tuesday, June 14, 2011

DDDx 2011

I’m just back from DDDx 2011, and it was great !

The event happened Friday 10 at Skills Matter in London, with great speakers, coffee and food.


You can see all the talks on Skills Matter website. Congratulation to the team that released the videos on the web in less that an hour.


It was also the occasion to meet IRL DDD practitioners I usually find on twitter.


You can also register for next year now for only 50£.


So Hurry up !

- page 1 of 4