The Big Rewrite (1)

It has been a long time since my last blog post. It also has been a long time since I worked on the Castle project either. I don’t want to talk about the reasons however, but I had some ideas regarding ActiveRecord which I want to share.
My ideas have nothing to do with the current version of Castle ActiveRecord. Henry has done a great job pushing an RC out of the door, but AR’s codebase has grown over the years into a state that makes it hard to impossible keeping up with NHibernate’s features.
In the meantime, other changes took place in the world of ORMs that make most AR’s current features obsolete:

• For NHibernate there are two independent mapping libraries allowing to work without XML. I have used confORM recently and it is a pleasure to use. Mapping with attributes is rather clumsy compared to confORM, FluentNHibernate, or NHibernate’s own mapping by code.
• Since NHibernate 2.0 (which is a long time ago), transactions are mandatory. AR uses SessionScope and TransactionScope and a few other scope types which aren’t used widely, making AR even more complicated than naked NHibernate.
• NHibernate has new mapping features. Honestly, I don’t know whether someone has added them to the AR codebase. The last time I tried, the visitor model used for examining the attributes drove me nuts.
• Entity Framework has matured and while EF 4.1 is still a bit behind NHibernate, it is simpler to use than ActiveRecord.

Goals of ActiveRecord vNext

So what are the features I envision for the next AR version? You might guess them from what I don’t like in the current version:

• ActiveRecord API (entity.Save() etc.)
• POCO Support
• Testability
• A single Unit-of-Work concept.
• Abstracting from the used ORM as much as possible reasonable.

For now, I will only highlight each of the design goals in short. I will share more on them in separate posts.

ActiveRecord API

AR vNext will concentrate on one major goal: Providing an easy to use API of the active record pattern. As a user, I want to call Save() or Find() from anywhere in my code without passing around session objects, DAOs or repositories.

POCO Support

The model should consist of plain object classes. The current AR requires inheritance for full support of the active record API. AR vNext will not even contain a base class for models. It will not require to use an attribute either. However, there is a drawback: Even with the full power of C# 4.0, implementing an empty interface is needed to streamline the experience.

Testability

Testing without database access is a must for TDD. Even the use of in-memory-databases requires initialization of the ORM which is far more problematic than getting data from an in-process-call to SQLite.
AR vNext will contain the necessary abstractions and native support for mocking and stubbing data access. Have your code use entity.Save() and fail the test if the wrong entity was saved… all without even loading the ORM’s assemblies.

Single Unit-of-Work Concept

A unit of work is mandatory unless you want to code demo apps. However, we don’t need separate concepts for web apps, service executables, GUIs and CLI programs. The single UoW must include transactions and allow both implicit and explicit usage.

Reasonable Abstractions

You might have noticed that I used ORM in the paragraphs above and not NHibernate. Well, while NH will be used and integrated as default ORM in vNext, I strive to abstract it out of the core API, so that it is possible to plug in other persistence frameworks in the future, which might be EF or RavenDB.
This abstraction will not used at any price. Don’t expect to change a configuration value to move from SQL to NoSQL or something similar. The goal is to factor out ORM-specific features in separate classes, so that you can remove the assembly references and have your compiler tell you what is required to adapt in order to use another ORM.

When can I use it?

Right now.
NuGet: Monastry.ActiveRecord-NHibernate
GIT: https://github.com/mzywitza/Monastry.ActiveRecord

Understand ActiveRecord mapping: Collections of simple objects

Recently I had a post in the Castle user’s group about mapping a collection of enums with ActiveRecord. Since I know it’s just as mapping other simple types, I only referred to the documentation of the Element and ElementType properties of the HasMany attribute.

But, it turned out it is not really clear to many how to map a collection of simple values using HasMany. Therefore I decided to prepare an explanatory post, which is what you are reading now.

Mapping Basics

The problem: I want to hold a collection of enum values, say I have a status enum and want a history of that status persisted. Here is my model:

namespace EnumMapping
{
	[ActiveRecord]
	public class Article
	{
		[PrimaryKey]
		public virtual int Id { get; set; }

		[Property]
		public virtual string Author { get; set; }

		[Property]
		public virtual string Title { get; set; }

		[Property]
		public virtual Status CurrentStatus { get;set;}

		[HasMany(...)]
		public virtual IList StatusHistory
		{
			get { return statusHistory; }
			set { statusHistory = value; }
		}
		private IList statusHistory = new List();

	}

	public enum Status
	{
		None, Planned, InWriting, InEditing, Released 
	}
}

The ellipsis in the HasMany attribute is by design. I will work out in this post what you have to put here, so I let it out to make you understand the post, and not just read the code and copy and paste it.

To understand what information ActiveRecord requires to map this collection, we first will take a look into how these collections are saved in the database:

As expected, we have to tables. The Article table stores all the simple properties, but it cannot store collections. In a relational model, collections are always modeled by foreign key relations. That means we need a second table that stores both the values and the link to the article in question.

Note: You might have noticed that the StatusHistory table does not have a primary key. This is because of the bag semantics we currently have for this collection. More efficient semantics are shown later.

Looking at the StatusHistory table again, we see what we have to tell ActiveRecord to fetch the values for us and populate our List:

What is the table’s name?

 

We take that name and put it into the respective property of the HasMany attribute:

[HasMany(Table="StatusHistory",
	...)]
public virtual IList StatusHistory {...}

As you can see from the code, we are not through yet.

Which column refers to the article?

[HasMany(Table = "StatusHistory",
	ColumnKey = "article",
	...)]
public virtual IList StatusHistory {...}

Which column holds the value?

[HasMany(Table = "StatusHistory",
	ColumnKey = "article",
	Element = "status",
	...)]
public virtual IList StatusHistory {...}

What? There is even more to specify even though we already have all columns included?

Yes. ActiveRecord also needs to know the types of the columns. The foreign key’s column can be inferred from the types primary key, but we need to know what type the value actually has.

[HasMany(Table = "StatusHistory",
	ColumnKey = "article",
	Element = "status",
	ElementType = typeof(Status))]
public virtual IList StatusHistory {...}

That’s it (for now). We now can take that mapping and go for the database.

The Collection Types

I already mentioned the missing primary key on the collection’s table and that the collection uses bag semantics. So what are these semantics? There are a few available in ActiveRecord:

• Bag: All elements are unordered and may appear multiple. Total chaos.
• Set: The elements are unordered, but each one can be present only once. This is the typical relational collection.
• List: Elements are ordered and may be there multiple times.
• Dictionary: Elements have a key that identifies them.

If the collection type is not specified, ActiveRecord assumes that you want bag semantics. After all, that’s what a collection is. Put things in there and get them out. No one cares for ordering or duplication.

Unfortunately, this is quite inefficient with databases:Items in a bag cannot be updated individually. Recall that there is no primary key in the collection table. Since the value can be in the collection more than once, an update or delete would effect all rows with that value. As a result, every time the collection is updated, it is completely deleted from the DB and inserted again.

To get around this, we could use a set. However, this requires us to use another Collection type, Set<Status> from Iesi.Collections. So instead of a set, I will use a list semantic. This is suitable, because a status history has an implicit ordering.

To use the list semantic, the schema must be changed first, because an index column is needed to store the list’s index.

We now don’t have only an additional column, we also have a nice primary key consisting of the combination of the article’s id and the list’s index. So every value in the list can now be updated individually.

Now we must tell ActiveRecord to use list semantics and how to find the index:

[HasMany(Table = "StatusHistory",
	ColumnKey = "article",
	Element = "status",
	ElementType = typeof(Status),
	RelationType = RelationType.List,
	Index = "idx")]
public virtual IList StatusHistory {...}

That’s it.

Understanding ActiveRecord Sessions 1

This article series deals with the internal session keeping of Castle ActiveRecord. It begins in explaining how sessions are managed within ActiveRecord and how the user can influence this behavior.

What happens when Save() is called?

The first part of our journey starts in ActiveRecordBase. Whenever a method is called that requires a NHibernate session, it requests one from the following field:

protected internal static ISessionFactoryHolder holder;

Since that field is marked protected internal static, subclasses of ActiveRecordBase can directly access it to acquire a session. This is done with the following methods:

ISession CreateSession(Type type);
void ReleaseSession(ISession session);
void FailSession(ISession session);

The first method requests a session from the ISessionFactoryHolder, that can be used to issue calls to the session, such as Save(), Load() etc. After the operation was completed, ReleaseSession must be called, preferably in a finally block.

If an exception is raised from NHibernate, the session cannot be used anymore. The ISessionFactoryHolder must be notified through the FailSession-method.

To further understand the inner workings of ActiveRecord, we need to look at the default implementation of ISessionFactoryHolder, which is simply called SessionFactoryHolder.

The SessionFactoryHolder keeps a dictionary which maps the configured ActiveRecord root types to ISessionFactory instances. Whenever CreateSession is called, it fetches the ISessionFactory for that type and creates an ISession.

When ReleaseSession is called the session is flushed and disposed, closing any open connections. If you use the holder to acquire sessions directly, keep in mind that it is necessary to release them or you will get a resource leak. Database sessions are among the most critical resources with regard to leaks.

If an exception is raised, the FailSession method will clear the session so that it doesn't throw again when the session is released.

When ActiveRecord is used without any scopes, all of this happens on a single call of any of the data retrieval or persistence methods (Save(), FindAll() etc.):

1. ActiveRecordBase gets an ISession from the SessionFactoryHolder. ActiveRecordMediator simpy calls a static method on ActiveRecordBase, so there are no differences with respect to session handling.
2. ActiveRecordBase performs the desired database operation.
3. If there is no exception, ReleaseSession is called by ActiveRecordBase. In case of an exception, FailSession is called instead and the exception is rethrown.
4. SessionFactoryHolder closes the ISession instance. Upon the next call, a new session is created.

But what if we need to have a single session span multiple commands? This will be handled in the following articles when we talk about scopes.

WTF: Overloading and Generics

Recently I discovered some behaviour of .NET that I couldn't explain. I still wonder whether this is an error or "ByDesign". I found it when I tried to call ActiveRecordMediator's Exists method, which threw out an unexpected exception. Minutes later, I stared unbelievingly to the screen: A complete different method overload was called. I sat down and created a short example that doesn't use ActiveRecord, so it can be understood from any .NET-developer:

public interface IString
{
  string Content{ get;}
}

public class DString:IString
{
  private readonly string content;

  public string Content
  {
    get { return content; }
  }

  public DString(string content)
  {
    this.content = content;
  }
}

public static class Class1
{
  public static void Foo(params IString[] bars)
  {
    foreach (IString s in bars)
    {
      Console.WriteLine(s.Content);
    }
  }

  public static void Foo<T>(T bar)
  {
    throw new NotImplementedException();
  }

}

[TestFixture]
public class Tester
{
  [Test]
  public void CallBar()
  {
    Class1.Foo(new DString("baz"));
  }
}

So what would you expect when running the unit test? That's what I got: TestCase 'ClassLibrary2.Tester.CallBar' failed: System.NotImplementedException : The method or operation is not implemented. What happens here? If you add exactly one optional parameter, the compiler somehow thinks, you meant Foo<istring>() instead of Foo(). This happens only if the type parameter is an interface. I tried with string and that worked like expected, calling Foo(). So, is this a bug or a documented behaviour. Do you know ressources that document it? I didn't find any...

From The Book of Ideas: Searchable ActiveRecord-Entities

One of the feature requests that I hear most often from my users that they want a search function for the grids. I then show them a search form where they can specify field and values exactly. Most often, I then hear: "No, I meant a search field. I enter some text and the software shows all relevant entries." I usually answer by asking whether it should also save the climate and overcome poverty or if simple mindreading is sufficient. Ok, but last night I had an idea how searching with a single textfield interface can be accomplished without redeveloping that nasty thing for each view. I use the Castle stack for .Net for my applications, which means that my entities are created with Castle ActiveRecord and displayed in MonoRail web applications. The idea works like this: There is only one who knows which properties must be included into the search and this the developer that created the entities. Therefore we need some mechanism to specify the searchable properties. My idea was adding a Searchable attribute to the ActiveRecord classes' properties that need to be included in such searches:

    [ActiveRecord]
    public class Entity : ActiveRecordBase<Entity>
    {
        private int id;
        private string name;
        private string entityDesc;
        private string internalName;

        [PrimaryKey]
        public int Id
        {
            get { return id; }
            set { id = value; }
        }

        [Searchable][Property]
        public string Name
        {
            get { return name; }
            set { name = value; }
        }

        [Searchable(FriendlyName = "Description")][Property]
        public string EntityDesc
        {
            get { return entityDesc; }
            set { entityDesc = value; }
        }

        [Property]
        public string InternalName
        {
            get { return internalName; }
            set { internalName = value; }
        }
    }

The Searchable takes an optional parameter, FriendlyName that allows to specify a localized or other, more user-friendly name. The attribute can be applied to both properties and relations. If applied to a relation attribute (BelongsTo, HasMany, HasAndBelongsToMany etc.) the properties tagged in that type will be added to the search as well. Then there is a SearchMediator, which will create a NHibernate Criteria Query that will check for every search term whether it can be parsed by the datatype of a tagged property and adds it to the query if this is the case. The code below sketches how the query could be created:

public static T[] Search(string text)
{
    string[] words = text.Split();
    DetachedCriteria criteria = DetachedCriteria.For<T>();
    foreach (string word in words)
    {
        Disjunction group = Expression.Disjunction();
        foreach (PropertyInfo info in typeof (T).GetProperties())
        {
            if (info.GetCustomAttributes(typeof (SearchableAttribute), true).Length > 0)
                group.Add(Expression.Like(info.Name, word, MatchMode.Anywhere));
        }
        criteria.Add(group);
    }
    return ActiveRecordMediator<T>.FindAll(criteria);
}

This can be extended by many means. Examples include:

• Allowing propertyName:searchTerm to narrow search to a specific property by its name or friendly name.
• Allowing other operators than the colon like price>10.00 or name!Foo
• Add globbing to specify whether exact matches or all matches should be found
• Specify the MatchMode behavior for string properties as part of SearchableAttribute

Using that mechanism, I can add searching to almost any entity similar to validation. What do you think, would such a Search component benefit the Castle Framework?

That is only some thoughts. I didn't start implementing anything yet. However, anyone interested in helping is welcome, of course.