Understanding ActiveRecord Sessions 2

This article series deals with the internal session keeping of Castle ActiveRecord. This installment introduces the concept of a scope and how scopes are used within ActiveRecord.

Introducing Scopes

Having a session per database call is suboptimal. There is no chance to use transactions, which is a showstopper for most serious uses. Another issue is lazy loading: Lazy loading prevents NHibernate from loading large collections until they are requested, avoiding to load unnecessary data. Lazy loading is however only possible when done within a single session.

Castle ActiveRecord uses scopes to overcome this. A scope represents a single unit of work. It is not necessarily a single transaction, but may span multiple transactions. Prior to NHibernate 2.0, when transactions were not mandatory, scopes did not even have to support transactions.

The scopes that are part of the ActiveRecord package do always share a single session that is used for all persistance related calls. However, it is possible to implement a scope that uses a new session, but the changed semantic should be clearly communicated in such a case.

Localizing Scopes

In order to access scopes without holding a reference to the object, they stored in thread static stacks. The central interface for this is IThreadScopeInfo, which is implemented by various classes to cover different situations such as web applications. WebThreadScopeInfo for example uses a HttpContext for storing the scopes while the default ThreadScopeInfo implementation uses a thread static field.

In order to acquire the current IThreadScopeInfo implementation, ThreadScopeAccessor can be used. This class is a singleton providing the current ScopeInfo under

public IThreadScopeInfo ScopeInfo

Additionally it acts a proxy to this scope, delegating calls to the scope info in the property above. This allows to access the current scope without keeping a field for it using

ThreadScopeAccessor.Instance.XXX();

where XXX is one of the methods defined in IThreadScopeInfo.

With the IThreadScopeInfo available, the current scope can be requested by SessionFactoryHolder using the following methods:

ISessionScope GetRegisteredScope()
bool HasInitializedScope

Scope Initialization

When a scope is created, it has no sessions stored at first. That is intentional. While the ActiveRecordStarter configures the ISessionFactoryHolder at startup, it cannot configure an arbitrary number of ISessionScope implementations.

Therefore the initialization of scopes is implemented using a simple protocol between ISessionFactoryHolder and ISessionScope:

1. Upon creation, the ISessionScope registers itself with the current IThreadScopeInfo.

   void IThreadScopeInfo.RegisterScope(ISessionScope scope)

2. ISessionFactoryHolder fetches the scope the next time it has to deliver a session to the ActiveRecordBase methods.

   bool IThreadScopeInfo.HasInitializedScope
   ISessionScope IThreadScopeInfo.GetRegisteredScope()

3. The ISessionFactoryHolder asks the ISessionScope whether it already has a suitable session stored. Scopes do not hold only one session. Due to different database connections by root type, they need to hold a dictionary of sessions. The scope doesn't know the key to the sessions because the key is provided by the ISessionFactoryHolder everytime a session is required.

   bool ISessionScope.IsKeyKnown(object key)

4. If there is a session registered with the scope, the session is requested and the protocol ends.

   ISession ISessionScope.GetSession(object key)

5. If there is no suitable session available the ISessionFactoryHolder asks the ISessionScope whether it accepts an existing session or if it wants to create its own session.

   bool ISessionScope.WantsToCreateTheSession

6. Depending on the answer to 5, the ISessionFactoryHolder either opens a session itself or provides a suitable ISessionFactory to the ISessionScope so that the session can be created by the scope itself.

   ISession ISessionScope.OpenSession(ISessionFactory sessionFactory, IInterceptor interceptor)

7. The session is registered with the ISessionScope by the holder. This is done regardless who in fact created the session.

   void ISessionScope.RegisterSession(object key, ISession session)

8. The freshly registered session is fetched from the scope.

   ISession ISessionScope.GetSession(object key)

The different Scope Types

Scopes generelly have two attributes that describe there behaviour: by FlushAction and by SessionScopeType. The FlushAction controls whether changes are automatically flushed to the database. The SessionScopeType describes the behaviour of the scope in common, most important whether the scope supports transactions.

If the FlushAction is defined as FlushAction.Never, no writing to the database will occur unless ISessionScope.Flush() is called by the using code. This behaviour gives full control to the using code, but requires it to control flushing. This is important: If the changes are not flushed, queries will still find an older state in the database although the entity has already changed.

FlushAction.Auto instructs the NHibernate session to flush its first-level-cache whenever needed. That means that if the cache contains an unflushed change to an entity, the session will write those changes back before it runs a query against the entity's type.

The SessionScopeType has four values:

SessionScopeType.Undefined
SessionScopeType.Simple
SessionScopeType.Transactional
SessionScopeType.Custom

Undefined should never be used; it is more an error state than a valid scope type. Simple and Transactional define whether the scope supports transactional behaviour. Custom can be used for own implementations that fall in between. An example would be a SessionScope that uses transactions only for specific sessions.

The next part of the series will show how ActiveRecord usage differs when using a scope.

Understanding ActiveRecord Sessions 1

This article series deals with the internal session keeping of Castle ActiveRecord. It begins in explaining how sessions are managed within ActiveRecord and how the user can influence this behavior.

What happens when Save() is called?

The first part of our journey starts in ActiveRecordBase. Whenever a method is called that requires a NHibernate session, it requests one from the following field:

protected internal static ISessionFactoryHolder holder;

Since that field is marked protected internal static, subclasses of ActiveRecordBase can directly access it to acquire a session. This is done with the following methods:

ISession CreateSession(Type type);
void ReleaseSession(ISession session);
void FailSession(ISession session);

The first method requests a session from the ISessionFactoryHolder, that can be used to issue calls to the session, such as Save(), Load() etc. After the operation was completed, ReleaseSession must be called, preferably in a finally block.

If an exception is raised from NHibernate, the session cannot be used anymore. The ISessionFactoryHolder must be notified through the FailSession-method.

To further understand the inner workings of ActiveRecord, we need to look at the default implementation of ISessionFactoryHolder, which is simply called SessionFactoryHolder.

The SessionFactoryHolder keeps a dictionary which maps the configured ActiveRecord root types to ISessionFactory instances. Whenever CreateSession is called, it fetches the ISessionFactory for that type and creates an ISession.

When ReleaseSession is called the session is flushed and disposed, closing any open connections. If you use the holder to acquire sessions directly, keep in mind that it is necessary to release them or you will get a resource leak. Database sessions are among the most critical resources with regard to leaks.

If an exception is raised, the FailSession method will clear the session so that it doesn't throw again when the session is released.

When ActiveRecord is used without any scopes, all of this happens on a single call of any of the data retrieval or persistence methods (Save(), FindAll() etc.):

1. ActiveRecordBase gets an ISession from the SessionFactoryHolder. ActiveRecordMediator simpy calls a static method on ActiveRecordBase, so there are no differences with respect to session handling.
2. ActiveRecordBase performs the desired database operation.
3. If there is no exception, ReleaseSession is called by ActiveRecordBase. In case of an exception, FailSession is called instead and the exception is rethrown.
4. SessionFactoryHolder closes the ISession instance. Upon the next call, a new session is created.

But what if we need to have a single session span multiple commands? This will be handled in the following articles when we talk about scopes.

Active Record and DDD

One of the most fatal mistakes one can conduct when beginning with Domain-Driven Design is doubling Active Record types as domain entities. This does not include Castle ActiveRecord, but all frameworks that map classes and tables one-to-one, and to some extend even more flexible solutions like NHibernate.

How does it start?

This trap is usually hit by developing a data-driven applications using ActiveRecord as an ORM. There is nothing bad with the approach per se, and I'm actually using it myself a lot. Let's see the following code taken from the Castle ActiveRecord GettingStarted section:

[ActiveRecord]
public class Blog : ActiveRecordBase<Blog>
{
	private int id;
	private String name;
	private String author;
	private IList<Post> posts = new List<Post>();

	public Blog()
	{
	}

	public Blog(String name)
	{
		this.name = name;
	}

	[PrimaryKey]
	public int Id
	{
		get { return id; }
		set { id = value; }
	}

	[Property]
	public String Name
	{
		get { return name; }
		set { name = value; }
	}

	[Property]
	public String Author
	{
		get { return author; }
		set { author = value; }
	}

	[HasMany(
		Table="Posts", ColumnKey="blogid", 
		Inverse=true, Cascade= ManyRelationCascadeEnum.AllDeleteOrphan)]
	public IList<Post> Posts
	{
		get { return posts; }
		set { posts = value; }
	}
}

This is straight forward data-driven code and there is nothing bad about it. Note that no business logic is embedded in the class. In the simple GettingStarted example, the logic is buried in the GUI, but in a real application you would perhaps use Transaction Scripts to encapsulate logic in objects.

Setting the trap

The programmer eventually reads Eric Evans great book or hears from a mailing list about DDD. He might remember that Castle ActiveRecord does not require a base class and removes it, using ActiveRecordMediator for database access.

Now that there are only POCOs, our unwary programmer starts adding business logic to the ActiveRecord types. By that, he tries to encapsulate complexity within the "domain model".

But what has happened:

• Most important, he violates the Single Responsibility Principle (SRP); the class is now responsible for multiple aspects:
• Storing data
• Executing business logic
• In the first few iterations, business logic that was previously packed in one method is now cluttered over multiple classes. DDD's supple design promises to mitigate that but a design usually only becomes supple by a lot of refactoring.

The immediate result is big step backwards in maintainability. Over the long term, DDD will have a better maintainability, but you will need a lot of work to reach this state. By that time, the trap has already sprung...

The trap fires

For a while, all will be well. The programmer get accustomed to the code and does some changes. The code gets more complex and a bit unwieldy. Finally, the programmer needs a "break-through"; a big refactoring takes place to make the code more supple.

Now, violating the SRP fires back: It is not possible to refactor the design without writing complex migration scripts for the database. Integration suddenly becomes an issue. However, the redesign is utterly needed because of the business logic embedded in the design.

The typical outcome is that the redesign is put off until "there is more time", or shorter: "never". In the meanwhile, the code base is growing and getting more and more fragile.

How to recover?

The most important task in recovering from such a dilemma is deciding which approach will be used for the application. You can choose a data-driven approach or a domain-driven approach, but not both.

Using DDD

If the complexity of the domain mandates DDD, it is necessary to do it right. This means using the ActiveRecord types as a DAO/DTO layer and building a model upon it that contains the business logic. The model is then decoupled from the data structure. If the model is redesigned, the mapping code requires to adapt, not the structure of the data storage. If the storage structure changes, the mapping code changes and not the model.

This is also the reason why it is possible to use NHibernate directly on a domain model: The NHibernate mapping files are mapping code, written in an XML-based DSL (and soon with a fluent API)

So this is the other way out of the trap when using DDD. If you use Castle ActiveRecord, take the hbm-files created when using the debug switch and use NHibernate directly instead.

Using data-driven design

It is also possible to make a full turn. A domain of modest complexity that defines most of the business cases as sequential workflows and processes, will benefit from using a data-driven approach.

The processes defined in the business domain can be modeled using the Transaction Script pattern and the Active Record model is exactly that: a pattern for accessing an underlying database.

On the "Anemic Model"

Many of the solutions above use a model that is disregarded as anemic by many. But whether a model is anemic, depends on responsibilities rather than on LOC.

Thus an ActiveRecord-model is not anemic because it is responsible for accessing the data store. A DAO/DTO is part of the persistence layer and its responsibility is passing data around. By coincidence, it doesn't need any methods for this task, but it is not anemic.

After all, violating the SRP is always worse than having an anemic model.

Disclaimer

The poor programmer who unwarily builds a trap who fired at himself was of course me.

Emulating MixIns with C#

One of the shortcomings of C# is the missing support of MixIns, which are a popular substitute for multiple inheritance especially in Ruby. MixIns provide a way to put a common functionality in a separate classand use it in another class. That is still easy in C#, one can simply use inheritance. But, what will you do, when your classes are already inheriting from other classes? You won't want to break up your inheritance hierarchie for this, will you? Using a MixIn, you can put a common functionality in a separate class and mix it into multiple classes, regardless of their inheritance hierarchy. The methods and properties of a MixIn-class are merged into the using classes' interface and can also use the original classes members. That's the theory and what is possible in Ruby, but can it be achieved in C#? If you need an example for MixIns, you might want to read the following blog post http://www.juixe.com/techknow/index.php/2006/06/15/mixins-in-ruby/. However, you don't need any Ruby knowledge for the remainder of my post. Ok, what are the characteristics of a MixIn again:

1. Accessible via the using class
2. Has access to the using class' instance members
3. Must not use inheritance

Taking into account that C# is a statically and strongly typed compiled language, this is not possible at first glance. But, if all possibilities of C# are used, they can be fulfilled at least partially.

As an example, I will develop a MixIn that adds reduction-functionality to collections. Reduction is a functional programming concept means that the collection will be reduced (boiled down) to a scalar value using a user-specified delegate. Simple reductions are joining a collection of strings or adding up a collection of integers.

The following code shows how this can be implemented using a conventional static method:

    1 using System.Collections.Generic;

    2 

    3 namespace Example

    4 {

    5     public delegate TItem ReduceDelegate<TItem>(

    6         TItem firstItem, 

    7         TItem secondItem);

    8 

    9     public static class Reduction

   10     {

   11         public static TItem Reduce<TItem>(

   12             IEnumerable<TItem> collection, 

   13             ReduceDelegate<TItem> reduceFunction)

   14         {

   15             TItem result = default(TItem);

   16             bool first = true;

   17             foreach (TItem item in collection)

   18             {

   19                 if (first)

   20                 {

   21                     result = item;

   22                     first = false;

   23                 }

   24                 else

   25                 {

   26                     result = reduceFunction(result, item);

   27                 }

   28             }

   29             return result;

   30         }

   31     }

   32 }

Now the question is, how can this functionality be implemented as a mixin. We will need at least to interfaces for this:

• An interface that allows the mixin access to the using class.
• An external interface that allows a client to use the functionality provided by the mixins.

The code below shows how these are declared:

    1 namespace Mixin

    2 {

    3     public delegate TItem ReduceDelegate<TItem>(

    4         TItem firstItem, 

    5         TItem secondItem);

    6 

    7     public interface IReduceClient<TItem> : IEnumerable<TItem>{}

    8 

    9     public interface IReduceMixin<TItem>

   10     {

   11         TItem Reduce(ReduceDelegate<TItem> reduceFunction);

   12     }

   13 

   14     public interface IWithReduce<TItem> : IReduceClient<TItem>, IReduceMixin<TItem> {}

   15 

   16     public class ReduceMixin<TItem> : IReduceMixin<TItem>

   17     {

   18         public ReduceMixin(IReduceClient<TItem> client)

   19         {

   20             this.client = client;

   21         }

   22 

   23         private readonly IReduceClient<TItem> client;

   24 

   25         public TItem Reduce(ReduceDelegate<TItem> reduceFunction)

   26         {

   27             TItem result = default(TItem);

   28             bool first = true;

   29             foreach (TItem item in client)

   30             {

   31                 if (first)

   32                 {

   33                     result = item;

   34                     first = false;

   35                 }

   36                 else

   37                 {

   38                     result = reduceFunction(result, item);

   39                 }

   40             }

   41             return result;

   42         }

   43     }

   44 }

Line 7 declares the interface that the mixin uses to access the using class (IReduceClient). This can be arbitrary complex, but in this case, we only need an enumeration, so I just inherited from IEnumerable. Line 9 declares the interface that is publicly used to access the mixin's functionality (IReduceMixin). IWithReduce on line 14 just brackets the other two interfaces, so that an using class can specify IWithReduce, which shows the intention better than using the two different interfaces.

The mixin class is shown on lines 16ff. It simply holds a reference to the client, a.k.a. the using class and the Reduce implementation shown above.

Now the problem is how can the mixin added to a client. The code below would be ideal, but won't work with C# 2.0:

    1 namespace Client

    2 {

    3     public class ReducableList<T> : List<T>, IWithReduce<T>

    4     {

    5         // Won't work that way!

    6     }

    7 }

Before I show a (naive) implementation, here is the test code for the whole thing, that shows how it is used:

    1 namespace Test

    2 {

    3     [TestFixture]

    4     public class ReduceTest

    5     {

    6         [Test]

    7         public void TestMixin()

    8         {

    9             ReducableList<string> rl = new ReducableList<string>();

   10             rl.AddRange(new string[] {"a", "b", "c"});

   11 

   12             IReduceMixin<string> reducable = rl;

   13             Assert.AreEqual("a,b,c", reducable.Reduce(delegate(string s1, string s2) { return s1 + "," + s2; }));

   14 

   15             ReducableList<int> rli = new ReducableList<int>();

   16             rli.AddRange(new int[] {1, 2, 3, 4});

   17             Assert.AreEqual(10, rli.Reduce(delegate(int i, int j) { return i + j; }));

   18         }

   19     }

   20 }

R# Jedi

One simple, naive and somewhat cheating method to make the ReducableList work, is using ReSharper. Just add a IRecudeMixin field and choose Alt+Ins/Delegate members. As a finishing touch you need to initialize the field.

The resulting code is shown below:

    1 namespace Client

    2 {

    3     public class ReducableList<T> : List<T>, IWithReduce<T>

    4     {

    5         public ReducableList()

    6         {

    7             reducer = new ReduceMixin<T>(this);

    8         }

    9 

   10         // Other constructors omitted for brevity

   11 

   12         private readonly ReduceMixin<T> reducer;

   13 

   14         public T Reduce(ReduceDelegate<T> reduceFunction)

   15         {

   16             return reducer.Reduce(reduceFunction);

   17         }

   18     }

   19 }

Well, I know that this is not a viable solution for maintainable software, but it is at least a quick fix and an introduction for the next article, which will show how to use DynamicProxy2 for creating a ReducableList without ReSharper Jedi.

The Component Burden Concept

Important Note: Mort that I am, I have mistaken ReleasePolicy for Component Burden and blogged about the former... As soon as I know what I am writing about, I will update that post.

There is a discussion on the Castle mailing list recently about the Component Burden problem. This is one of the oldest Castle issues (and the oldest that is not resolved by now). Now, the issue got a little bit of momentum by Ayende's post and Hammett's blog entry about it.

In short, the problem is this: If there are non-singleton components created by the IoC-Container that implement IDisposable, someone sometime needs to call Dispose(). However, both someone and sometime pose a problem:

• Someone could be the using code or the container. It has been proposed that a component should dispose it's dependencies itself, but if you think about it, this is problematic because the component cannot know it's dependencies' lifestyles. Just imagine a service configured to run as a singleton being disposed after the very first request it served, because an using component doesn't need it anymore. Additionally there is a simple principle about resource allocation: The code that allocates a resource is responsible for deallocation too. Period. Back when C/C++ was used, every developer adhered to that principle because there was no GC...
• Sometime is dependent on the components lifestyle. A singleton must not be disposed before the container shuts down itself while a transient component must be disposed as soon as possible to free allocated resources.

Now it is clear that the container is fully responsible for both construction and destruction of dependency objects. But how will it know, when the component is not used anymore?

Well, at least one component must be resolved from the container. This component must be released after you are finished using it:

MyComponent c = container.Resolve<MyComponent>(); // Some work container.Release(c);

Do you need to do this even if you're component is not disposable? Yes, because it can have direct or indirect dependencies that are disposable. If you don't release it, the container cannot know that it is now safe to dispose these dependencies, when they are otherwise unused.

Now there is one thing missing: The container must keep track of the dependencies. Assume that your component uses a custom per-web-request-logger that uses a FileStream.

Every time you resolve a component that uses the logger in one web request, the dependency will be fulfilled with the same logger. If you release a component, the container must remember that it used the logger, then check whether the logger is still used by other instances and if not, dispose the logger.

This bookkeeping of dependencies and disposal is called the component burden in the Castle project.

Although I don't know Microkernel's source code for this, I have an idea how to implement such a concept. But this is a topic for another post.

WTF: Overloading and Generics

Recently I discovered some behaviour of .NET that I couldn't explain. I still wonder whether this is an error or "ByDesign". I found it when I tried to call ActiveRecordMediator's Exists method, which threw out an unexpected exception. Minutes later, I stared unbelievingly to the screen: A complete different method overload was called. I sat down and created a short example that doesn't use ActiveRecord, so it can be understood from any .NET-developer:

public interface IString
{
  string Content{ get;}
}

public class DString:IString
{
  private readonly string content;

  public string Content
  {
    get { return content; }
  }

  public DString(string content)
  {
    this.content = content;
  }
}

public static class Class1
{
  public static void Foo(params IString[] bars)
  {
    foreach (IString s in bars)
    {
      Console.WriteLine(s.Content);
    }
  }

  public static void Foo<T>(T bar)
  {
    throw new NotImplementedException();
  }

}

[TestFixture]
public class Tester
{
  [Test]
  public void CallBar()
  {
    Class1.Foo(new DString("baz"));
  }
}

So what would you expect when running the unit test? That's what I got: TestCase 'ClassLibrary2.Tester.CallBar' failed: System.NotImplementedException : The method or operation is not implemented. What happens here? If you add exactly one optional parameter, the compiler somehow thinks, you meant Foo<istring>() instead of Foo(). This happens only if the type parameter is an interface. I tried with string and that worked like expected, calling Foo(). So, is this a bug or a documented behaviour. Do you know ressources that document it? I didn't find any...

From The Book of Ideas: Searchable ActiveRecord-Entities

One of the feature requests that I hear most often from my users that they want a search function for the grids. I then show them a search form where they can specify field and values exactly. Most often, I then hear: "No, I meant a search field. I enter some text and the software shows all relevant entries." I usually answer by asking whether it should also save the climate and overcome poverty or if simple mindreading is sufficient. Ok, but last night I had an idea how searching with a single textfield interface can be accomplished without redeveloping that nasty thing for each view. I use the Castle stack for .Net for my applications, which means that my entities are created with Castle ActiveRecord and displayed in MonoRail web applications. The idea works like this: There is only one who knows which properties must be included into the search and this the developer that created the entities. Therefore we need some mechanism to specify the searchable properties. My idea was adding a Searchable attribute to the ActiveRecord classes' properties that need to be included in such searches:

    [ActiveRecord]
    public class Entity : ActiveRecordBase<Entity>
    {
        private int id;
        private string name;
        private string entityDesc;
        private string internalName;

        [PrimaryKey]
        public int Id
        {
            get { return id; }
            set { id = value; }
        }

        [Searchable][Property]
        public string Name
        {
            get { return name; }
            set { name = value; }
        }

        [Searchable(FriendlyName = "Description")][Property]
        public string EntityDesc
        {
            get { return entityDesc; }
            set { entityDesc = value; }
        }

        [Property]
        public string InternalName
        {
            get { return internalName; }
            set { internalName = value; }
        }
    }

The Searchable takes an optional parameter, FriendlyName that allows to specify a localized or other, more user-friendly name. The attribute can be applied to both properties and relations. If applied to a relation attribute (BelongsTo, HasMany, HasAndBelongsToMany etc.) the properties tagged in that type will be added to the search as well. Then there is a SearchMediator, which will create a NHibernate Criteria Query that will check for every search term whether it can be parsed by the datatype of a tagged property and adds it to the query if this is the case. The code below sketches how the query could be created:

public static T[] Search(string text)
{
    string[] words = text.Split();
    DetachedCriteria criteria = DetachedCriteria.For<T>();
    foreach (string word in words)
    {
        Disjunction group = Expression.Disjunction();
        foreach (PropertyInfo info in typeof (T).GetProperties())
        {
            if (info.GetCustomAttributes(typeof (SearchableAttribute), true).Length > 0)
                group.Add(Expression.Like(info.Name, word, MatchMode.Anywhere));
        }
        criteria.Add(group);
    }
    return ActiveRecordMediator<T>.FindAll(criteria);
}

This can be extended by many means. Examples include:

• Allowing propertyName:searchTerm to narrow search to a specific property by its name or friendly name.
• Allowing other operators than the colon like price>10.00 or name!Foo
• Add globbing to specify whether exact matches or all matches should be found
• Specify the MatchMode behavior for string properties as part of SearchableAttribute

Using that mechanism, I can add searching to almost any entity similar to validation. What do you think, would such a Search component benefit the Castle Framework?

That is only some thoughts. I didn't start implementing anything yet. However, anyone interested in helping is welcome, of course.