You may believe it or not, but the post that drains most of the traffic of this blog, is the one about C# static interfaces !

In october 2009, I simply tried to imagine where the idea of C# static interfaces could lead us, and, since then, I have more viewed pages (> 15%) on this post than on my home page !

And since then, nothing moved in this area in the C# langage, and I don’t expect it to happen soon.

But some other thing happened…

F#

Yes F# is out and running on almost all platforms, and it can do what I described in the previous post.

The thing is called Statically Resolved Type Parameters and is closer to C++ templates than from C# generics.

The trick is that you can define an inline function with statically resolved types, denoted by a ^ prefix. The usage of defined methods on the type is not given here by an interface, but by a constraint on the resolved type :

let inline count (counter: ^T) =
    let value = (^T: (member Count : int) counter) 
    value

here , the count function takes a counter of type ^T (statically resolved).

The second line express that ^T actually should have a member Count of type int, and that it will call it on counter to get the result value !

Magic !

Now, we can call count on various types that have a Count member property like :

type FakeCounter() =
    member this.Count = 42;

or

type ImmutableCounter(count: int) =
    member this.Count = count;
    member this.Next() = ImmutableCounter(count + 1)

or

type MutableCounter(count: int) =
    let mutable count = 0
    member this.Count = count;
    member this.Next() = count <- count + 1

without needing an interface !

For instance :

let c = count (new FakeCounter())

True, this is compile time duck typing !

And it works with methods :

let inline quack (duck: ^T) =
    let value = (^T: (member Quack : int -> string) (duck, 3))  
    value

This will call a Quack method that takes int and returns string with the value 3 on any object passed to it that has a method corresponding to the constraint.

And magically enough, you can do it with static methods :

let inline nextThenstaticCount (counter: ^T) =
    (^T: (member Next : unit -> unit) counter)
    let value = (^T: (static member Count : int) ()) 
    value

this function calls an instance method called Next, then gets the value of a static property called Count and returns the value !

It also works with operators :

let inline mac acc x y = acc + x * y

notice the signature of this function :

acc: ^a -> x: ^c -> y: ^d ->  ^e
    when ( ^a or  ^b) : (static member ( + ) :  ^a *  ^b ->  ^e) and
         ( ^c or  ^d) : (static member ( * ) :  ^c *  ^d ->  ^b)

It accepts any types as long as they provide expected + and * operators.

The only thing is that a specific implementation of the function will be compiled for each type on which it’s called. That’s why it called statically resolved.

You can use this kind of method from F# code but not from C#.

Anyway…

No need for static interfaces in C#, use F# !

I have still new comments and there are some Stack Overflow questions on the subject that show the response to the question is still unclear…

Should you inject, or even use repositories in you entities ?

The response is two fold, and both go in the same direction, you shouldn’t.

Of course, we’re discussing design here, so if in your specific case you find reasons more important than those invoked here, there’s always a possibility to decide to go as you wish…

Injection and DDD patterns

Where talking Injection and Entities/Repositories here, so the first reason concerns what injection means in a Domain Driven Design environement.

The wikipedia definition of the dependency injection pattern only makes references to objects or components without defining explicitly which kind of object or which kind of component…

But a famous injection anti pattern can give use more information on what are injected components.

I call the Service Locator anti pattern.

This make it clear that what we are looking for is a way to find service in a supple and configurable way.

I won’t explain here why the service locator is an anti pattern, but it makes it obvious that the use of injection is to provide access to services in the code.

Since services are supposed to be stateless, their lifecycles are usually quite flexible. A service could be instanciated on each call, on each request or as a single instance without much difference. The injected service dependencies enable to compose services to replace implementations by configuration or for tests..

But even at runtime. A live configuration change could indicates to replace an injected service on next instantiation.

Services and repositories are quite obviously not injected with Entities/Aggregates:

• A repository is here to instantiate entities, it’s its own job, it doesn’t need to be injected with entities…
• When a service needs an entity, it can be injected with a repository, and get the entity through it.

But could entities be injected with services or repositories ?

An aggregate or an entity is not stateless as a service, it is statefull, and its member fields are here to maintain current state.

It seems quite odd to use those fields to store references to services that are obviously not part of the state.

It also links the referenced service lifecycle to the lifecycle of the entity itself.

Most entities instanciation schemes in web applications are on a per call basis, and since most web application don’t do anything between calls, the response to the lifecycle problem is simply that everything in created and destroyed on each call.

But it is only a specific simplistic way to consider object lifecycle. What if you kept your entities longer in memory and reused them on next call to avoid roundtrips with the data storage ?

Having a reference to a service inside an entity state would make the live reconfiguration far harder to achieve : You would have to trash all current state and reload everything. There is no such problem with services since they’re meant to be stateless.

Entities fields are meant to store state, no to reference services. Their lifecycles should not be coupled.

Consistency boundary

The second reason is about the aggregate consistency boundary.

Why would you typically need a reference to a repository inside an aggregate ?

First reason would be to load sub entities when needed. It’s also called delay load.

You shouldn’t need delay load in aggregates

The reason comes from the aggregate definitions. From wikipedia:

Aggregate: A collection of objects that are bound together by a root entity, otherwise known as an aggregate root. The aggregate root guarantees the consistency of changes being made within the aggregate by forbidding external objects from holding references to its members.

The definition talks about object being bound together. Those objects – entities – have strong relationships, or grouping them in an aggregate makes little sense. When the relation is strong, you will need for each

aggregate method to change all objects in coordination, or none. If not, parts of your aggregate should not be here.

If an aggregate needs all its state for each operation, there is strictly no reason to use delay load.

Load all, use all, save all.

The second reason would be to load entities in other aggregates.

You shouldn’t need references to entities in other aggregates

or at least not directly. The service calling the method on the aggregate will find information needed to call the method (which can contain information coming from other aggregates), but if you need a direct reference to another entity, it’s a clear indication that the aggregated boundary is wrong ! Put that other entity in the aggregate !

Thus once your aggregate modeling is ok, there is really no reason to use a repository inside an entity.

In the wake of Rinat Technology Demons, I’d add Mocking frameworks to the list.

There are several reasons to do that.

Stubs

One of the usage of Mocking - or isolation – frameworks is stubbing. A component that access external resources or has complex behavior will be changed for a fake to have simpler predictable results.

• When the component interface is small, the stub declaration code is often more complicated that writing a fake implementation directly.
• When the interface has many methods, it makes things easier, but here you’ll ignore a code smell. It makes things easy in situation you should not reach !

There are also easy patterns that make the use of mocking irrelevant. A common one is configuration loading.

Lots of developers think: “I need a class to access my configuration. Let’s make a class for this”:

Then they realize they’ll have to isolate from ConfigurationManager, so they add a interface :

And they can moke the configuration using a lot of boring code.

what about this ?

Passing a different configuration in your test is a simple new Configuration {}… and you can use the ConfigurationLoader class in you main function.

A good thing here, is that you don’t suffer a read of the configuration file at each call without needing to introduce lazy loading or other tweaks that make things worse..

Mocks

When you need to see if a call happened, you can make an implementation of the interface that takes a lambda :

Or make the fake class record the call, and add a getter to check. It’s not that complicated.

As some of you already know, I'm a proponent of DRY : Do Repeat Yourself, but code reuse has some value and nuget is a good way to manage it.

So far, the advantages I see in using a package manager are:

• easy get/update of external projects and their dependencies
• on demand dependency update (instead of forced dependency update)
• makes it easier to modularize dependencies

All this thing are out of the box when using OSS projects published on nuget.org, and you can host your own nugets using NuGet.Server.

The debugging story is also quite good. symbolsource.org can host symbols and source packages and be used as source server. You can directly step in your favorite OSS source code without having to compile it yourself. They even provide private repositories.

But sending your company's source code to an external service is not always compatible with internal policy.

In this configuration, using your own nugets leads to a poor dev experience when you have no way to step in your own code : compiled in Release and potentially not your latest code version, indicating the source code in another directory will not give good results.

You need a source server.

NuRep to the rescue

NuRep is a nuget repository based on NuGet.Server but it is also a symbols + code server.

When creating your nuget package, specify the –Symbols flag, and nuget will create a .nupkg and a .symbols.nupkg that you can push to NuRep (http://myserver/nurep/api/v2/package).

Then configure visual studio's symbols servers (Tools / Options / Debugging / Symbols / add http://myserver/nurep/symbols )

Don't forget to enable source server in Debugging / General options, and to disable Just my code.

That's it.

Now you'll step into the exact code that was used to compile the nuget version.

Have fun !

In the part one comments, Clement suggested a more efficient solution than registering handler in constructor.

The proposed solution is to have a RegisterAllEvents virtual method in which event handler registration would occur. This method is a method instance to have access to this but will be called only once per class. The registration use Expression<Action<T>> to access the expression tree and extract the method info of the handler. This enables type checking, make R# happy – no unused methods – and make reflection not too painful.

Good solution.

I didn't go that far because with Event Sourcing, you usually keep aggregates in memory, so aggregates are instantiated once per service lifetime.
I just crafted a small performance test :

using System;
using System.Collections.Generic;
using System.Diagnostics;
 
namespace AggregatePerfTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var watch = new Stopwatch();
 
            const int count = 10000000;
            Guid id = Guid.NewGuid();
 
            watch.Start();
            for (int i = 0; i < count; i++)
                new AggregateRegisteredOncePerInstance(id);
 
            watch.Stop();
 
            Console.WriteLine(watch.Elapsed.TotalMilliseconds);
 
            watch.Reset();
            watch.Start();
 
            for (int i = 0; i < count; i++)
                new AggregateRegisteredOncePerClass(id);
 
            watch.Stop();
 
            Console.WriteLine(watch.Elapsed.TotalMilliseconds);
 
        }
    }
 
    public class AggregateRegisteredOncePerClass
    {
        private readonly Guid id;
 
        private static readonly object ClassInitLock = new object();
        private static bool initialized;
 
 
        public AggregateRegisteredOncePerClass(Guid id)
        {
            this.id = id;
 
            lock (ClassInitLock)
            {
                if (!initialized)
                {
                    initialized = true;
                    // registration happens only once here
                }
            }
        }
 
        public Guid Id
        {
            get { return id; }
        }
    }
 
    public class AggregateRegisteredOncePerInstance
    {
        private readonly Guid id;
        private readonly Dictionary<Type, dynamic> handlers = 
                         new Dictionary<Type, dynamic>(5);
        public AggregateRegisteredOncePerInstance(Guid id)
        {
            this.id = id;
            Register<int>(OnSomethingHappened);
            Register<double>(OnSomethingHappened);
            Register<float>(OnSomethingHappened);
            Register<long>(OnSomethingHappened);
        }
 
        public Guid Id
        {
            get { return id; }
        }
 
        public void DoSomething()
        {
            Apply(1);
        }
 
        private void OnSomethingHappened(int message) { }
        private void OnSomethingHappened(double message){ }
        private void OnSomethingHappened(float message) { }
        private void OnSomethingHappened(long message) { }
 
        protected void Apply<T>(T @event)
        {
            handlers[typeof (T)](@event);
        }
 
        protected void Register<T>(Action<T> handler)
        {
            handlers.Add(typeof(T), handler);
        }
    }
}

The code is straight forward, I just created two aggregate classes :

• one with registration in .ctor based on this post code
• one without any registration at all, considering that doing it once is the same as not doing it for large numbers, but I added a lock section with a boolean check to simulate what will done on each instance creation.

I created 10.000.000 instances for each, and you get:

• 3978ms for the one with .ctor registrations,
• 377 ms for the one without.

It's true that it makes a difference. But how many aggregates do you have in your system ?

With 10.000 aggregate you're still under 8ms. I think you can afford that.

This is then a trade-off between performance and simplicity :

• If you have very large numbers, go for expression tree parsing, class lock management etc.
• In any other situation I recommend using registration in .ctor that makes the code easy to implement in approximately 5min.

How do we usually manage time in applications ?

Timers, threads, concurrency locks…

If we want to practice Domain Driven Design, we’re surely at the wrong level of abstraction.

What is time, btw ?

Tricky question. We know what time is, but… giving a definition is not that easy.

What defines time ? The second ?

Not really. It is used as a measure of time, but it doesn’t seem sufficient.

Let’s have a look at Wikipedia’s definition of time  :

Time is a part of the measuring system used to sequence events, to compare the durations of events and the intervals between them, and to quantify rates of change such as the motions of objects. […]

Now we have something interesting : Time is what happens between events.

But what is this thing between events.

The definition of the measure unit surely can give us further insight.

Lest have a look at Wikipedia’s definition of the second :

[…] Since 1967, the second has been defined to be

the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium-133 atom.

The second is defined as a count of transitions between states of an atom of cesium.

We measure time by considering that the ~time~ between those state transitions is constant.

What if it’s not ?

…

It’s not that important if other events seem synchronized with those events. Will come back to this later.

Events

Let’s step back a bit.

How do you feel time passing ?

By looking at your watch ?

Maybe, but how could you explain that an hour sometimes seems so long, and sometimes passes in a flash ?

Time seems slow and empty when you’re bored.

Times seems fast and full when you’re busy with interesting things.

When you’re bored, it’s because few interesting things happen.

You can deduce from this that your personal state transitions are those interesting things that happen.

These are meaningful events. Things that happens and change you deeply.

Of course a lot of things happen between those meaningful events, you’re moving, thinking. Your blood flows through your body, but it is just maintenance move. You don’t change deeply.

Maybe some things happen between state transition of a cesium atom, but since we cannot notice it and give it a meaning for now, it has no influence.

But when when a meaningful event happens, you change. You’re not the same before and after.

This is what time is about, and this is why it’s one way.

Before –> Event –> After

Events define time by causality

This perception of meaningful events is surely a reason why people say the time pass faster when old. In your 6 first years any event around you is meaningful. Any event make you change since you have no previous knowledge. Then has time goes by, you integrate knowledge and filter things you already know, you’ve already seen. When old, a year can more easily seem the same than the year before.

But some people continue to enjoy and learn as much as they can to still have a long now.

When do your system change ?

Your system never change for no reason.

It’s always because a meaningful event happened.

This event can be a user interaction, a call from an external system, a sensor trigger…

And when things change because it’s midnight ?

It simply means that midnight is a meaningful event in your system.

Where are those meaningful events in your code ? Hidden in infrastructure code ?

I hear Greg Young say :

Make the implicit explicit !

And it’s simple :

Use Domain Events.

Once you’ve introduced Domain Events in your domain model, you have made Events and so Time explicit in your domain.

There is no change in the domain that is not due to an Event.

The events appear everywhere in the Ubiquitous Language :

• When the client HasMoved to a new location, send him a welcome kit.
• When a RoomHasBeenOverbooked try to relocate the customer
• Every day at midnight = MidnightOccured, change last minute prices.

I’m sure you can find examples in your own domain. If your domain is business related, it has to deal with time because business is about time and money.

Time is now part of your Ubiquitous language and you have an implementation for it.

And that’s huge.

I’ve been posting a sample SmtpListener on my repository a few days ago.

It’s infrastructure stuff.. what does it have to do with what I’m usually talking about here ?

It’s about Reactive Programming.

Sometimes you have to integrate with legacy systems that’ll send you things through email.

Ok, I know, it kind of sucks as an integration mechanism, but still… you have to do this way.

The usual way to receive email in an application is to set a mailbox on a mail server (think Exchange), and pull mailboxes periodically to see if there’s something new there.

There are two bad things here :

• People tend to use their enterprise mail server for this. The mail sever is often vital for your business and screwing it up with a bug can have a big impact on your organization.
• Email are pushed to you.. why would you pull it. You can push it directly on your service bus !

So, I prototyped a small smtp listener that could easily be integrated with whatever you want.

You need to add a MX entry in your dns zone configuration so that other servers find it, and get a valid certificate if you want to use secured TLS connections (I’ve disabled certificated verification for demo purpose).

But as you can see, the code is very simple since the .Net framework has already what’s needed.

The TcpListener is used to receive connections that’ll provide a TcpClient with underlying streams.

The SslStream class is used to encapsulate tcp streams to add Ssl encryption.

I’m using the reactive framework to convert the Begin/EndAcceptTcpClient methods to an Observable to avoid writing the accept loop myself.

Then implementation of the protocol is very easy, you can find an overview and sample on wikipedia.

The RFCs can easily be found :

RFC 5321: Simple Mail Transfer Protocol

RFC 3207: SMTP Service Extension for Secure SMTP over Transport Layer Security

Of course you can use it freely, and propose changes to make it better since it’s not attack proof.

This code is not resistant to known potential SMTP attacks, including dangling connections, long lines etc..

I recently had a remark from Frederic Fadel from Aspectize, telling me about Event Sourcing something like:

Why would you like to write SQL to write data to your read model when our product can do it for you ?

I acknowledge that their product is fancy. You simply declare your db data schema, your UI and services and bind it all together.

But it doesn’t fit well with CQRS and Event Sourcing. And I want to do Event Sourcing for domain and business reasons, not technical reasons.

But he was write on this point :

I don’t want to write SQL to denormalize my events to my queryable storage.

What are my options ?

• Writing SQL by hand, but testability is low, and you’ll get a mix of C# to get data from the events, and SQL for the Update queries.
• Using an ORM. When doing NHibernate you don’t really write SQL. Testability wont be great anyway.

The problem with ORMs

ORM are usually better at getting data than at changing it. You can do it, but let’s look at what happens.

The ORM loads data from your Db into entities that will be tracked by an identity tracker. Then you change the values in memory . Then the ORM will find what needs to be sent back to the server and make a query to the Db so that the change happens.

But what I need to do is a lot simpler. Just emit some INSERT, UPDATE or DELETE based on current table values and event data.

With an ORM, what happens if the data is changed between loading and saving ? I’ll have to manage some versioning and/or transaction. And I’ll make two roundtrips to the server needlessly.

Here comes Duck

Duck is a kind of ORM oriented toward Delete Update Create.

Don’t ask Duck to load data in memory, it simply can’t.

You simply express how data should change based on current row content and values that you’ll pass.

It avoids the first roundtrip to the database, and make shorter code to express the change.

Let’s see how to use it

First, you should declare a class that has the structure of your table with public get/set properties, and marked with a Table attribute :

     [Table]
       class Species
   {
         public Guid Id { get; set; }
          public string Name { get; set; }
         public string BinomialName { get; set; }
         public bool IsEndangered { get; set; }
           public int Population { get; set; }
      }

It contains current observed species at an observatory.

Then a simple new statement, let’s say that a new species has be registered at the observatory :

     var duck = new DuckContext'(connectionString);
 var speciesId = Guid.NewGuid();
      duck.In<Species>()
          .Insert(() => 
            new Species
             {
          Id = speciesId,
            Name = "Mallard",
            BinomialName = "Anas platyrhynchos",
          IsEndangered = false,
            Population = 50
          });

Nothing tricky here..

The observatory noticed a population decay, the species is endangered :

     duck.In<Species>()
          .Where(r => r.Id == speciesId)
         .Update(r => new Species {
              Population = r.Population - 40,
              IsEndangered = true});

Here, the use of the current value of Population will not load current value. It will the following statement :

UPDATE Species
SET
    Population = Population - 40,
    IsEndangered = 1
WHERE
    Id = @p0

I chose  to create a new Row from the old one rather than change the original one for two reasons :

• It makes rows immutable and you don’t have to think about execution order between fields assignments. It’s the way SQL works
• Linq Expressions cannot represent statement blocks and assignments in C#3, Duck would have been .Net only…

The –40 is directly in the query here because we used a constant. I we where using a variable, query would contain a parameter

Now the species has disappeared, it should be removed from the observed species (though it could be just an update somewhere else) :

     duck.In<Species>()
          .Where(r => r.Id == speciesId)
         .Delete();

Testability

To run your test you just have to use the InMemoryDuckContext… you have then access to Table<T>() that’ll enable you to set up your data and verify after execution that things changed as expected. I’ll talk a bit more about it soon.

Try it now, it’s OSS

You can grab the code at bitbucket and try it now :

http://bitbucket.org/thinkbeforecoding/duck

It’s in F# ! Writing a AST analyzer in F# is far more easy, concise and expressive than in C#. You’ll just have to reference Duck in you project, there’s no direct F# dependency.

Next episode will be about how to mix it with Rx (Reactive Framework) to declare your event handling logic.

Hope you like it, and don’t hesitate to give feedback and suggestions.

So we have our events serialized in our event store. Deserializing events is not an issue, until we start to make them evolve and need to manage several versions.

Since we never modify what has been log, we’ll have to deal with old versions anyway.

A simple way to do it is to maintain every versions of the events in the projects, and make the aggregate root accept all of them. But it will soon charge the aggregate root with a lot of code and will make it bloated rapidly.

This is why you can usually introduce a converter that will convert any version of the event to the last one (usually you provide methods to update to next version, and iterate until last version so that this part of the code is incremental). This is a convenient way to address the problem, but you still have classes v1, v2 … vn that you keep in your project only for versioning purpose even if you don’t use it anymore in your production code.

Events as documents

It is easy do deserialize an event as an object or a document, you only need to split two responsibilities in you deserialization process :

• Stream reading
• Object building

The deserializer will be in charge of reading the data, it reads the bits, and get the meaning from context, it will tell the Object Builder about objects types, fields names and value.

On its side, the ob ject builder will instantiate the objects, set fields values depending on names.

You can provides two distinct Object Builders. The strongly typed one will instantiate concrete .net types and set fields using reflection. The document builder one, will instantiate objects that will only be property bags.

When deserializing an event in its last version, you can use directly the strongly typed one, but when reading an previous version of the event, you can deserialize it as a document and give it to the converter.

The converter will then add/remove properties from the document to make it up to date, and the document will be used to create a concrete .net type of the last event version.

Here the process is quite the same, you should provide a document reader that will use the strongly typed object builder to instantiate the event.

There’s no need to keep every version of you Event Classes now since you can manipulate old versions as documents.

Using dynamic in C#4

Document manipulation can make things a bit messy since it can be hard to understand the original structure of the object. This is where you can use the DLR DynamicObject class to make the property bag (the document) a dynamic object that you’ll be able to use as any standard .net object.

This way, in the converter you can manipulate old versions of the events as .net objects without having to keep all those old classes that won’t be used anymore.

Leonardo had a question about reloading huge amounts of events.

It’s true that some Aggregate Roots have very long lifetimes with lots of events, and it can become a problem.

There are two things involved to resolve this problem :

Snapshots

Ok, the philosophy of event sourcing is to store changes instead of state, but we’ll still need state in our Aggregate Roots, and getting it from scratch can be long.

Take a snapshot every n events (you’ll see that n can be quite high), and store it alongside events, with the version of the aggregate root.

To reload the Aggregate Root, simply find the snapshot, give it to the Aggregate root, the replay events that happened after the snapshot.

You only need the last snapshot for each Aggregate Root, no need to log all passed snapshots.

When you want to change stored state in an Aggregate Root, you won’t be able to used last snapshot since it will not contains expected state. But you can still replay events from scratch when it happens, so you have no loss, and simply take a new snapshot with the new state.

In memory domain

Usually with an ORM, you reload entities from the storage on every unit of work.

But in the case of Event Sourcing, your Aggregate Roots only need to retain state that will be used to take business decisions. You’ll never query state from Aggregate Roots. A large part of the entity state and especially the part that has the biggest memory footprint is usually stored only for queries, like names, descriptions and things like that.

In an Aggregate Root in an Event Sourcing environment, a name or description can simply be checked for validity, put in an event, but don’t need to be kipped in the in memory entity state – the Aggregate Root fields.

You’ll notice that your big domain state can fit in memory once you’ve trimmed it this way.

Now that your model is in memory, no need to reload every events on each unit of work. It happens only once when the Aggregate Root is needed the first time.

Well see soon how you can use this to make your event serialization even faster to have very high business peak throughput.