Raspberry Pi + Mono + EF6 – defeat!

After (or maybe even before) I successfully made EF6 work on Mono I thought it would be cool to try it on Raspberry Pi. Since my project already worked on Mono I thought it should be relatively easy to do. The first thing I had to do was to install Mono on my Raspberry Pi. It is easy and requires running just two commands:

sudo apt-get update
sudo apt-get install mono-runtime

Since my project uses MySQL I also had to install MySQL database. Again it was as easy as executing:

sudo apt-get install mysql-server

After that I used mysql and mysqladmin tools to create a user and a database that would be used by my app. With my Raspberry Pi ready I copied my little app to the Raspberry Pi and ran it using the mono command:

mono EF6MySqlTest.exe

only to see it failing with the following exception:

Missing method .ctor in assembly /home/pi/monotest/MONOTest/EntityFramework.dll, type
System.Runtime.CompilerServices.ExtensionAttribute
Can't find custom attr constructor image: 
 /home/pi/monotest/MONOTest/EntityFramework.dll mtoken: 0x0a00006a

Unhandled Exception: System.TypeLoadException: Could not load
type 'System.Runtime.CompilerServices.ExtensionAttribute' from assembly 'EntityFramework'.
  at EF6MySqlTest.SimpleContext..ctor () [0x00000] in :0
  at EF6MySqlTest.Program.Main (System.String[] args) [0x00000] in :0
[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeLoadException: Could not load
type 'System.Runtime.CompilerServices.ExtensionAttribute' from assembly 'EntityFramework'.
  at EF6MySqlTest.SimpleContext..ctor () [0x00000] in :0
  at EF6MySqlTest.Program.Main (System.String[] args) [0x00000] in :0

The exception startled me a little bit. The very first thing I noticed was that the app failed even before it started executing. I looked at the documentation for the ExtensionAttribute type on the MSDN but I did not find anything extraordinary. Only comparing the documentation for .NET Framework 4.5 with the documentation for .NET Framework 4 gave me the clue. In the .NET Framework 4.5 the ExtensionAttribute type lives in the mscorlib.dll while in the .NET Framework 4 it lived in the System.Core.dll assembly. In general it is OK. CLR has this feature where a type can be moved from one assembly to another as long as its full name does not change and the original assembly contains the TypeForwardedToAttribute assembly level attribute pointing to the assembly the type lives in the new version (this is called type forwarding and you can find more details here). The exception I got just said that the EntityFramework assembly depends on the ExtensionAttribute which could not be found. Putting the information together I inferred what the exception did not say – namely that the ExtensionAttribute type could not be found in the mscorlib.dll assembly. This was just one step away from solving my problem. If the mscorlib.dll did not contain the ExtensionAttribute type then the type had to be in the System.Core.dll assembly which in turn meant that the version of the .NET Framework supported by Mono packages I installed was 4 and my application was compiled targeting version 4.5. This was not a big deal. Entity Framework 6 supports both .NET Framework 4.5 and .NET Framework 4 so I went back and recompiled my app so that it targets .NET Framework 4, copied it to my Raspberry Pi started again and… failed again. This time the exception looked like this:

Unhandled Exception: 
System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.Entity.Config.DbConfigurationManager 
---> System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.SqlTypes.SqlDecimal 
---> System.OverflowException: Can't convert to SqlDecimal, Out of range
  at System.Data.SqlTypes.SqlDecimal..ctor (Byte bPrecision, Byte bScale, Boolean fPositive, Int32 data1, Int32 data2, Int32 data3, Int32 data4) [0x00000] in :0
  at System.Data.SqlTypes.SqlDecimal..cctor () [0x00000] in :0
  --- End of inner exception stack trace ---
  at System.Data.DataColumn..ctor (System.String columnName, System.Type dataType, System.String expr, MappingType type) [0x00000] in :0
  at System.Data.DataColumn..ctor (System.String columnName, System.Type dataType) [0x00000] in :0
  at System.Data.Common.DbProviderFactoriesConfigurationHandler.CreateDataSet () [0x00000] in :0

  [... omitted for brevity ...]

  --- End of inner exception stack trace ---
  at System.Data.Entity.DbContext.InitializeLazyInternalContext (IInternalConnection internalConnection, System.Data.Entity.Infrastructure.DbCompiledModel model) [0x00000] in :0
  at System.Data.Entity.DbContext..ctor (System.String nameOrConnectionString) [0x00000] in :0
  at EF6MySqlTest.SimpleContext..ctor () [0x00000] in :0
  at EF6MySqlTest.Program.Main (System.String[] args) [0x00000] in :0

The good thing was that the previous exception was gone and that I could see some familiar Entity Framework methods in the stack trace. This means that this time the application at least tried running. But then I got to the System.OverflowException: Can't convert to SqlDecimal, Out of range exception thrown from System.Data.SqlTypes.SqlDecimal..ctor and started feeling a bit uneasy. This did not look like a bug in my code. On the contrary, it looked like some problem in Mono which I most likely would not be able to fix or work around. To confirm that it was unrelated to my app or EF6 I played a bit and came up with the following program that duplicated the issue (yes, it is just one line of code and it probably could be simplified even more):

public static void Main()
{
  Console.WriteLine(DbProviderFactories.GetFactoryClasses().Rows.Count);
}

Running the above code resulted in the same exception I saw when running my app:

Unhandled Exception: System.TypeInitializationException: 
An exception was thrown by the type initializer for System.Data.SqlTypes.SqlDecimal —>
System.OverflowException: Can’t convert to SqlDecimal, Out of range
 at System.Data.SqlTypes.SqlDecimal..ctor (Byte bPrecision, Byte bScale, Boolean fPositive, Int32 data1, Int32 data2, Int32 data3, Int32 data4) [0x00000] in :0
 at System.Data.SqlTypes.SqlDecimal..cctor () [0x00000] in :0

I looked around a little bit and found two bugs (bug report 1 and bug report 2) that looked similar to what I was seeing. One of them even contained a link to a patch with the fix which made me think that this might be fixed in a newer version of Mono. Soon I found this page which mentions a Debian Experimental version which is 3.0.6 as opposed to 2.10.8 installed on the Raspberry Pi. I tried to follow the steps listed on the page but unfortunately I could not install the experimental version due to missing package dependencies:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
mono-complete : Depends: mono-devel (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-mcs (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-gmcs (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-dmcs (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-csharp-shell (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-2.0-gac (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-2.0-service (= 2.10.8.1-5) but it is not going to be installed
Depends: mono-4.0-service (= 2.10.8.1-5) but it is not going to be installed
Depends: monodoc-base (= 2.10.8.1-5) but it is not going to be installed
Depends: monodoc-manual (= 2.10.8.1-5) but it is not going to be installed
Depends: libmono-cil-dev (= 2.10.8.1-5) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

I am not a Linux guru but really wanted to make it work (or at least understand why it did not want to work) so I asked a question on stackoverflow and wrote an email to Mirco Bauer who seems to overlook Mono 3.0 on Debian. Unfortunately I have not received any answers which would help me resolve the problem so I gave up and forgot about this when suddenly after a couple weeks I got a response from Mirco. He wrote:

Mono has no working armhf port, but there is now work done to create an official armhf port. The only option for Mono is armel right now.

I have to admit that when I read the message I did not really understand it. So I did some research and found this post. It enlightened me and made me realize that the bug report I looked at before and the Raspberry Pi downloads page already had the answer I needed. I should have just tried understanding some details included there instead of ignoring them :) . To put it simply
- the Raspbian “wheezy” is an armhf distro (and – as per email from Mirco – “Mono has no working armhf port”) while the Soft-float Debian “wheezy” is an armel image. The difference between the images is how they handle floating point operations. This gave me some hope. I downloaded the Soft-float Debian “wheezy” image and quickly prep my Raspberry Pi by installing mono and MySQL. One of the very first thing I did was to run the repro app to verify if the OverflowException is gone. Indeed this time instead of the exception the app wrote 7 (I wish it had been 42 but 7 is still better than OverflowException). This started looking promising so I tried to run the “proper” app. I got a few uninteresting exceptions (like no permissions to access registry etc.) but then I got this:

Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.Entity.ModelConfiguration.Conventions.Sets.V2ConventionSet ---> System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.Entity.ModelConfiguration.Conventions.Sets.V1ConventionSet ---> System.TypeLoadException: Could not load type 'System.Data.Entity.ModelConfiguration.Conventions.KeyAttributeConvention' from assembly 'EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.
  --- End of inner exception stack trace ---
  at System.Data.Entity.ModelConfiguration.Conventions.Sets.V2ConventionSet..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---
  at System.Data.Entity.DbModelBuilder.SelectConventionSet (DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.DbModelBuilder..ctor (System.Data.Entity.ModelConfiguration.Configuration.ModelConfiguration modelConfiguration, DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.DbModelBuilder..ctor (DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.LazyInternalContext.CreateModelBuilder () [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.LazyInternalContext.CreateModel (System.Data.Entity.Internal.LazyInternalContext internalContext) [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.RetryLazy`2[System.Data.Entity.Internal.LazyInternalContext,System.Data.Entity.Infrastructure.DbCompiledModel].GetValue (System.Data.Entity.Internal.LazyInternalContext input) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.Entity.ModelConfiguration.Conventions.Sets.V2ConventionSet ---> System.TypeInitializationException: An exception was thrown by the type initializer for System.Data.Entity.ModelConfiguration.Conventions.Sets.V1ConventionSet ---> System.TypeLoadException: Could not load type 'System.Data.Entity.ModelConfiguration.Conventions.KeyAttributeConvention' from assembly 'EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.
  --- End of inner exception stack trace ---
  at System.Data.Entity.ModelConfiguration.Conventions.Sets.V2ConventionSet..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---
  at System.Data.Entity.DbModelBuilder.SelectConventionSet (DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.DbModelBuilder..ctor (System.Data.Entity.ModelConfiguration.Configuration.ModelConfiguration modelConfiguration, DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.DbModelBuilder..ctor (DbModelBuilderVersion modelBuilderVersion) [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.LazyInternalContext.CreateModelBuilder () [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.LazyInternalContext.CreateModel (System.Data.Entity.Internal.LazyInternalContext internalContext) [0x00000] in <filename unknown>:0
  at System.Data.Entity.Internal.RetryLazy`2[System.Data.Entity.Internal.LazyInternalContext,System.Data.Entity.Infrastructure.DbCompiledModel].GetValue (System.Data.Entity.Internal.LazyInternalContext input) [0x00000] in <filename unknown>:0

The most important thing here is Could not load type 'System.Data.Entity.ModelConfiguration.Conventions.KeyAttributeConvention' from assembly 'EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.. I looked at the KeyAttributeConvention and the class is very simple. There is actually hardly anything that could break here. Since the only dependency this class has is the KeyAttribute type it had to be the thing that caused problems. Again, I tried to isolate the issue so I added the following line to my test app:

Console.WriteLine(typeof(System.ComponentModel.DataAnnotations.KeyAttribute).Name);

After I did this the test app failed to compile complaining that it cannot resolve the KeyAttribute type and recommended adding references to an assembly containing a definition of this type even though I did provide a reference to System.ComponentModel.DataAnnotations.dll. This was weird. I decided to list all the types from the System.ComponentModel.DataAnnotations.dll assembly to make sure that the KeyAttribute is there so I replaced the line referring to the KeyAttribute with:

foreach(var t in typeof(ScaffoldColumnAttribute).Assembly.GetTypes())
{
    Console.WriteLine(t.FullName);
}

After running the app I looked at the type names and could not find the KeyAttribute type. This is unfortunate since it means that it’s impossible to run EF6 on the stable version (2.10.8) of Mono I installed. I thought the attribute might have been added in a newer version so I tried installing the experimental version. Unfortunately this did not really work. This time I did not get any errors but I got the message mono-complete is already the newest version and nothing was installed.
This was an interesting journey and I am a bit disappointed I was not able to achieve what I wanted especially because the goal seemed so close. The bright side is that I learnt quite a lot of interesting things and I believe it will be possible to run EF6 on the Raspberry Pi when either Mono works on armhf or there is a newer version of Mono for armel available. I will give it a try for sure in the future!

Entity Framework 6 on Mono

I had been wanting to try Entity Framework 6 with Mono for a long time and finally after a check-in I made on Friday I was able to spare some time to do this. I decided to go with MySQL and dotConnect for MySQL from Devart. After skimming the tutorial on using Devart’s providers with EF6 I opened Visual Studio 2012, created a new project, installed the EF6 alpha3 package from NuGet and thought I was almost done. The only problem was that the dead simple (one entity) EF6 based app I created did not work. I did not expect the exception I got and quickly realized why the tutorial I was following started with “Entity Framework 6 Alpha 2 support is implemented…”. Things got a little better when I went back from EF6 Alpha 3 to EF6 Alpha 2 but still did not work. I found that I was missing one of the dlls and then that I used wrong revision number in the version of the provider assemblies (interestingly the version of the assembly containing the provider factory was 7.6.217.0 while the version of the assembly containing provider services was 7.6.217.6 – I expected both assemblies to have the same version). The last time I used MySQL was… some time ago (OK, it was in the 20th century) and it was the first time I used it with Code First (kinda obvious). Even after setting all the versions correctly the app still did not want to run. I was able to get the edmx file with the EdmxWriter (which is a good sign meaning that the provider is configured correctly and working) but then the app would die because it could not create the database. After a quick search I found that I need to create an empty database using MySQL command prompt and then EF should be able to add tables to this database. Indeed – after I created the empty database that app did not throw and I was able to add entities to the database and then read them. With this I was ready to move to the Mono world. So, I installed the latest version of Mono and started playing a little bit with it. mcs.exe (the compiler) worked just fine. However since I could build my app with Visual Studio and ran it using mono.exe I decided not to use the mcs.exe. Unfortunately the app would not quite work. It would start and then throw System.Configuration.ConfigurationErrorsException: Failed to find or load the registered .Net Framework Data Provider .... Which basically means that the ADO.NET/EF provider I used was not registered. I think the reason was that I installed Mono after I installed the provider and therefore the entry in the machine.config was missing. I added the following entry:

  <system.data>
    <DbProviderFactories>
      <remove invariant="Devart.Data.MySql" />
      <add name="dotConnect for MySQL" invariant="Devart.Data.MySql" 
           description="Devart dotConnect for MySQL" 
           type="Devart.Data.MySql.MySqlProviderFactory, Devart.Data.MySql, Version=7.6.217.0, Culture=neutral, PublicKeyToken=09af7300eec23701" />
    </DbProviderFactories>
  </system.data>

to my app.config file and I was able to get a little bit further. (Just in case – you can just add the “add” line from the above to the Mono’s machine.config in the provider section and it should work as well). Now when I ran my app I got this:

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1&amp; schemaCollection) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems (IEnumerable`1 xmlReaders, IEnumerable`1sourceFilePaths) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init (IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest&amp; providerManifest, System.Data.Common.DbProviderFactory&amp; providerFactory, System.String&amp; providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2&amp; cachedCTypeFunction) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor (IEnumerable`1 xmlReaders) [0x00000] in :0
  at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection (System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo&amp; providerInfo) [0x00000] in :0
  at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff (System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.ModelMatches (System.Xml.Linq.XDocument model) [0x00000] in :0
  at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel (System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Database.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.CreateDatabaseIfNotExists`1[System.Data.Entity.DbContext].InitializeDatabase (System.Data.Entity.DbContext context) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[EF6MySqlTest.SimpleContext].b__b () [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformInitializationAction (System.Action action) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization () [0x00000] in :0
  at System.Data.Entity.Internal.LazyInternalContext.b__4 (System.Data.Entity.Internal.InternalContext c) [0x00000] in :0
  at System.Data.Entity.Internal.RetryAction`1[System.Data.Entity.Internal.InternalContext].PerformAction (System.Data.Entity.Internal.InternalContext input) [0x00000] in :0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1&amp; schemaCollection) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems (IEnumerable`1 xmlReaders, IEnumerable`1sourceFilePaths) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init (IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest&amp; providerManifest, System.Data.Common.DbProviderFactory&amp; providerFactory, System.String&amp; providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2&amp; cachedCTypeFunction) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor (IEnumerable`1 xmlReaders) [0x00000] in :0
  at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection (System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo&amp; providerInfo) [0x00000] in :0
  at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff (System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.ModelMatches (System.Xml.Linq.XDocument model) [0x00000] in :0
  at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel (System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Database.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.CreateDatabaseIfNotExists`1[System.Data.Entity.DbContext].InitializeDatabase (System.Data.Entity.DbContext context) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[EF6MySqlTest.SimpleContext].b__b () [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformInitializationAction (System.Action action) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization () [0x00000] in :0
  at System.Data.Entity.Internal.LazyInternalContext.b__4 (System.Data.Entity.Internal.InternalContext c) [0x00000] in :0
  at System.Data.Entity.Internal.RetryAction`1[System.Data.Entity.Internal.InternalContext].PerformAction (System.Data.Entity.Internal.InternalContext input) [0x00000] in :0

WTF? The same app worked just fine on .NET Framework! Obviously there was a bug somewhere but where? Mono? EF? .NET Framework? I followed the advice given by John Robins from Wintellect (http://www.wintellect.com/cs/blogs/jrobbins/default.aspx) during the “Debugging Windows Applications” training I took (too) many years ago and prayed that the bug was in the code base I own (because “in this case you can fix it”). I looked at some tutorials on debugging Mono apps but quickly felt it was too much to learn. Then I installed WinDbg but without symbols I could see only this:

ChildEBP RetAddr  Args to Child              
WARNING: Stack unwind information not available. Following frames may be wrong.
0022f5ec 76ccbbf7 00000000 77e8f3b0 ffffffff ntdll!KiFastSystemCallRet
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\msvcrt.dll - 
0022f600 758a36dc 00000001 0022f644 758a3372 kernel32!ExitProcess+0x15
0022f60c 758a3372 00000001 172bd79c 016b6698 msvcrt!exit+0x32
0022f644 758a36bb 00000001 00000000 00000000 msvcrt!dup+0x2a9
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Mono-3.0.9\bin\mono-2.0.dll - 
0022f658 65a62b3a 00000001 65bba06c 0546e928 msvcrt!exit+0x11
0022f678 65a61fe6 05af88c0 016b6698 00000000 mono_2_0!mono_invoke_unhandled_exception_hook+0x6a
0022f7e8 65aa7aba 00000000 00000000 00000000 mono_2_0!mono_print_thread_dump_from_ctx+0xaa6
0022f898 01550e8f 0022f8b0 05af88c0 03cd75ac mono_2_0!mono_x86_throw_exception+0xca
0022f968 03cd6a64 03d6f230 01c53cc0 03ee0d58 0x1550e8f
0022f9a8 03cd6968 01c53cc0 03ee0d90 0022f9bc 0x3cd6a64
0022f9c8 01bc19f5 01c53cc0 02b8e090 0022f9dc 0x3cd6968
0022f9e8 01bc1984 01c53cc0 02b8e090 0022f9fc 0x1bc19f5
0022fa08 01bc1935 01c53cc0 00538df0 0022fa1c 0x1bc1984
0022fa28 01bc18c5 01c593c0 02b8e090 0022fa3c 0x1bc1935
0022fa48 01bc17be 01c593c0 016bd7a8 0022fa5c 0x1bc18c5
0022fa68 01bc1740 01c593c0 02bc38f8 00000004 0x1bc17be
0022faa8 01bc1654 01c593c0 02b8e060 0022fabc 0x1bc1740
0022fba8 65b5340a 002c73d8 00000001 0022fbf8 0x1bc1654
0022fbb8 659cc21c 00000000 0022fc38 00000000 mono_2_0!mono_runtime_class_init+0x1a
0022fbf8 65b52e42 002c81b0 00000000 0022fc38 mono_2_0!mono_jit_compile_method+0x10c
0022fc18 65b55682 002c81b0 00000000 0022fc38 mono_2_0!mono_runtime_invoke+0x42
0022fc48 65a386b2 002c81b0 00539e00 00000000 mono_2_0!mono_runtime_exec_main+0xd2
*** ERROR: Module load completed but symbols could not be loaded for image00400000
0022fec8 004014bd 00000002 002c1710 00000000 mono_2_0!mono_main+0x1562
0022ff28 004010bb 00000002 002c16e8 002c1bb8 image00400000+0x14bd
0022ff68 004012a8 00000001 00000000 00000000 image00400000+0x10bb
0022ff88 76cbed6c 7ffdd000 0022ffd4 7701377b image00400000+0x12a8
0022ff94 7701377b 7ffdd000 7e6b10f1 00000000 kernel32!BaseThreadInitThunk+0x12
0022ffd4 7701374e 00401290 7ffdd000 00000000 ntdll!RtlInitializeExceptionChain+0xef
0022ffec 00000000 00401290 7ffdd000 00000000 ntdll!RtlInitializeExceptionChain+0xc2

Maybe there are people on this planet who can make sense out of it but I am not there yet. Without symbols I could do not much. So, I ended up sync’ing my source to the changeset around the date we shipped (git reset --hard 9a52aa5f) and this time WinDbg showed this:

This should never be null, since if we were not able to resolve, we should have never reached to this point
---- Assert Long Message ----
   at System.Diagnostics.DefaultTraceListener.Fail(System.String message, System.String detailMessage)
   at System.Diagnostics.TraceListener.Fail(System.String message)
   at System.Diagnostics.DefaultTraceListener.Fail(System.String message)
   at System.Diagnostics.TraceImpl.Fail(System.String message)
   at System.Diagnostics.TraceImpl.Assert(Boolean condition, System.String message)
   at System.Diagnostics.Debug.Assert(Boolean condition, System.String message)
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1 ByRef schemaCollection)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init(IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest ByRef providerManifest, System.Data.Common.DbProviderFactory ByRef providerFactory, System.String ByRef providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2 ByRef cachedCTypeFunction)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor(IEnumerable`1 xmlReaders)
   at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection(System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo ByRef providerInfo)
   at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff(System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations)
   at System.Data.Entity.Internal.InternalContext.ModelMatches(System.Xml.Linq.XDocument model)
   at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel(System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata)
   at System.Data.Entity.Internal.InternalContext.CompatibleWithModel(Boolean throwIfNoMetadata)
   at System.Data.Entity.Database.CompatibleWithModel(Boolean throwIfNoMetadata)
   at System.Data.Entity.CreateDatabaseIfNotExists`1[[System.Data.Entity.DbContext, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].InitializeDatabase(System.Data.Entity.DbContext context)
   at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[[EF6MySqlTest.SimpleContext, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].b__b()
   at System.Data.Entity.Internal.InternalContext.PerformInitializationAction(System.Action action)
   at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization()
   at System.Data.Entity.Internal.LazyInternalContext.b__4(System.Data.Entity.Internal.InternalContext c)
   at System.Data.Entity.Internal.RetryAction`1[[System.Data.Entity.Internal.InternalContext, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].PerformAction(System.Data.Entity.Internal.InternalContext input)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeDatabaseAction(System.Action`1 action)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeDatabase()
   at System.Data.Entity.Internal.InternalContext.Initialize()
   at System.Data.Entity.Internal.InternalContext.GetEntitySetAndBaseTypeForType(System.Type entityType)
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Initialize()
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].get_InternalContext()
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].ActOnSet(System.Action action, EntityState newState, System.Object entity, System.String methodName)
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Add(System.Object entity)
   at System.Data.Entity.DbSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Add(EF6MySqlTest.Customer entity)
   at EF6MySqlTest.Program.Main(System.String[] args)

Apparently my prayer was heard and this line proves it: at System.Diagnostics.Debug.Assert(Boolean condition, System.String message). This is an Assert in the EF code, so looks like a bug in the EF. After looking at the EntityKeyElement.Validate method I could easily find the Assert:

Debug.Assert(
    property != null,
    "This should never be null, since if we were not able to resolve, we should have never reached to this point");

This was a good find (bug filed: https://entityframework.codeplex.com/workitem/1038) but now – how this could have happened? After poking around and using less than scientific methods to debug this I finally concluded that the error is in the invalid SSDL file stored in the __MigrationHistory table. Here is the most interesting fragment:

<EntityType Name=""HistoryRow"" p5:IsSystem=""true"" xmlns:p5=""http://schemas.microsoft.com/ado/2012/10/edm/migrations"">
  <Key p5:IsSystem=""true"">
    <PropertyRef Name=""MigrationId"" p5:IsSystem=""true"" />
    <PropertyRef Name=""ContextKey"" p5:IsSystem=""true"" />
  </Key>
  <Property Name=""MigrationId"" Type=""varchar"" MaxLength=""255"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""ContextKey"" Type=""varchar"" MaxLength=""512"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""Model"" Type=""longblob"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""ProductVersion"" Type=""varchar"" MaxLength=""32"" Nullable=""false"" p5:IsSystem=""true"" />
</EntityType>

It does not look unusual but according to the schema for the SSDL the Key element must not have any attributes (be it in empty or non-empty namespace). Why this thing worked on the .NET Framework and not on Mono? Apparently the Xsd validation in Mono works a bit differently than on the .NET Framework. When processing PropertyRef elements the Xsd validator on .NET Framework reported they were valid whereas Mono claims they are not. I am not sure who is right in this case and, honestly, I don’t care too much. The major bug is in the EF – we should not produce invalid SSDL artifacts (bug filed: https://entityframework.codeplex.com/workitem/1038). This can be fixed either by not writing the {http://schemas.microsoft.com/ado/2012/10/edm/migrations}IsSystem attribute or by allowing attributes in non-empty and non-EF schema on the element. Since I already had a bootleg of entityframework.dll (and I believe the bug is actually in the schema for SSDL) I decided to fix the schema. In the System.Data.Resources.SSDLSchema_3.xsd file I changed the TEntityKeyElement to look like this (added the xs:anyAttribute line):

<xs:complexType name="TEntityKeyElement">
  <xs:sequence>
    <xs:element name="PropertyRef" type="edm:TPropertyRef" minOccurs="1" maxOccurs="unbounded" />
    <xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded" />          
  </xs:sequence>
  <!-- added to allow attributes on the element-->
  <xs:anyAttribute namespace="##other" processContents="lax" />
</xs:complexType>

This almost did the trick – I went past the NullReferenceException… but only to hit an exception with “Failed for unknown reasons” along with “SqlServer” in the message:

Could not load signature of System.Data.Entity.SqlServer.SqlProviderServices:GetExecutionStrategy due to: Failed for unknown reasons.

Unhandled Exception:
System.TypeInitializationException: An exception was thrown by the type initializer for ExtentPlaceholderCreator ---&gt; System.TypeLoadException: Could not load type 'System.Data.Entity.SqlServer.SqlProviderServices' from assembly 'EntityFramework.SqlServer, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.
  at (wrapper managed-to-native) System.MonoType:GetPropertiesByName (System.MonoType,string,System.Reflection.BindingFlags,bool,System.Type)
...

Hello??? Anyone there??? I am using MySQL. Any reason you want to load the SqlServer provider? (Yes, the provider was installed by default when I installed the EF package and then, somewhere on my way (must have been after making the app work on .NET Framework), I removed it manually but there should not be a reason to have this provider in the project (and potentially deployed) since it is not being used in my app). This seems like a bug to me but it might have been fixed since alpha2 (a similar issue like this has already been fixed http://entityframework.codeplex.com/workitem/587). Just in case I filed a bug for this https://entityframework.codeplex.com/workitem/1039. To make EF happy I did add the provider for SqlServer and… TADA the app worked.
Just in case – here is the “app” (not even worth putting on github)


namespace EF6MySqlTest
{
  public class Customer
  {
    public int Id { get; set; }
    public string Name { get; set; }
  }

  public class SimpleContext : DbContext
  {
    public SimpleContext()
      : base ("name=MySqlTest")
    {}
    
    public DbSet<Customer> Customers { get; set; }
  }

  class Program
  {
    static void Main(string[] args)
    {
      using (var ctx = new SimpleContext())
      {
        ctx.Customers.Add(new Customer() { Name = "moozzyk" });
        ctx.SaveChanges();
      }

      using (var ctx = new SimpleContext())
      {
        foreach (var c in ctx.Customers)
        {
           Console.WriteLine(c.Name);
        }
      }
    }
  }
}

and the config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <!-- For more information on Entity Framework configuration, visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
    <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
  </configSections>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
  </startup>
  <entityFramework>
    <providers>
      <provider invariantName="Devart.Data.MySql" type="Devart.Data.MySql.Entity.MySqlEntityProviderServices, Devart.Data.MySql.Entity, Version=7.6.217.6, Culture=neutral, PublicKeyToken=09af7300eec23701">
        <migrationSqlGenerator type="Devart.Data.MySql.Entity.Migrations.MySqlEntityMigrationSqlGenerator, Devart.Data.MySql.Entity, Version=7.6.217.6, Culture=neutral, PublicKeyToken=09af7300eec23701" />
      </provider>
    </providers>
  </entityFramework>
  
  <system.data>
    <DbProviderFactories>
      <remove invariant="Devart.Data.MySql" />
      <add name="dotConnect for MySQL" invariant="Devart.Data.MySql" 
           description="Devart dotConnect for MySQL" 
           type="Devart.Data.MySql.MySqlProviderFactory, Devart.Data.MySql, Version=7.6.217.0, Culture=neutral, PublicKeyToken=09af7300eec23701" />
    </DbProviderFactories>
  </system.data>
  
  <connectionStrings>
    <add name="MySqlTest" connectionString="server=localhost;user id=moozzyk;password=XXXXXXXXXXXXX;database=TestDb;persist security info=True" providerName="Devart.Data.MySql" />
  </connectionStrings>
</configuration>

To sum up – it was a pretty bumpy road but I am glad I took it. I found a few real issues we should be able to fix before we ship the final version of EF6 and therefore the experience of using EF6 and Mono should be much smoother.

My First Encounter with Custom Conventions in Entity Framework 6

I was not really involved in the work related to custom conventions in EF6 and had not had a chance to try this feature so when I saw this question on stackoverflow I thought it would be a good opportunity to try it out and learn something new. The question was – how do I configure foreign keys using custom conventions? Entity Framework has a built-in convention which configures a property whose name is equal to a corresponding navigation property name suffixed with “Id” as a foreign key property. However if the model uses a different naming convention the built-in convention will not work. In the question from stackoverflow the convention for foreign key properties was “Id” + {NavigationPropertyName}. Since it was used consistently across the model it really did make sense to use custom conventions instead of configuring each foreign key property manually (i.e. using ForeignKey attribute or .HasForeignKey() method). So, I looked at the custom conventions specification and started reading about lightweight conventions. It looked simple – one line of code should be enough. Unfortunately I could not find a good way to configure foreign keys with this approach – .Properties() allows only configuring primitive properties and foreign keys need to be configured on navigation properties. .Entities() is great for configuring entity level settings but does not really allow to configure a property if you don’t know its name (and it did not seem possible to configure navigation properties this way either). Therefore I started looking at the Configuration-based Conventions. They are no longer a single line of code as the lightweight conventions and require creating a class implementing the IConfigurationConvention interface containing the logic for the convention. In return they allow for much more flexibility. The IConfigurationConvention interface is generic and requires two types – TMemberInfo and TConfiguration. TMemberInfo can be either Type or PropertyInfo and defines whether the IConfigurationConvention.Apply() method should be invoked for types or properties. The second parameter (TConfiguration) describes what we would like to configure and can be for instance a PropertyConfiguration, StructuralTypeConfiguration, PrimitivePropertyConfiguration etc. (the full list can be found in the feature specification). One of the configurations on the list is NavigationPropertyConfiguration. This seemed to be exactly what I needed to be able to configure foreign keys. Now the plan was simple – create a convention that for each navigation property will try finding a property named “Id” + {NavigationPropertyName} and if such a property exists configure it as a foreign key property. This is the convention I came up with:

public class NavigationPropertyConfigurationConvention
    : IConfigurationConvention<PropertyInfo, NavigationPropertyConfiguration>
{
    public void Apply(
                 PropertyInfo propertyInfo, 
                 Func<NavigationPropertyConfiguration> configuration)
    {
        var foreignKeyProperty = 
            propertyInfo.DeclaringType.GetProperty("Id" + propertyInfo.Name);

        if (foreignKeyProperty != null && configuration().Constraint == null)
        {
            var fkConstraint = new ForeignKeyConstraintConfiguration();
            fkConstraint.AddColumn(foreignKeyProperty);

            configuration().Constraint = fkConstraint;
        }           
    }
}

While not a single liner the convention is short and easy to follow. I created a very simple model to test it out:

public class Entity
{
    public int Id { get; set; }

    public ICollection<RelatedEntity> RelatedEntities { get; set; }
}

public class RelatedEntity
{
    public int Id { get; set; }

    public int IdPrincipal { get; set; }

    public Entity Principal { get; set; }
}

public class MyContext : DbContext
{
    public DbSet<Entity> Entities { get; set; }
    public DbSet<RelatedEntity> RelatedEntities { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        modelBuilder.Conventions.Add<NavigationPropertyConfigurationConvention>();
    }
}

Note that the convention had to be registered in the OnModelCreating() method.
To see that the convention was applied I set a break point in the convention an started my test app. Also to verify it worked I just dumped the edmx using edmx writer and checked my AssociationType (notice the PropertyRef Name under the Dependent element):

<Association Name="RelatedEntity_Principal">
  <End Role="Entity" Type="Self.Entity" Multiplicity="1">
    <OnDelete Action="Cascade" />
  </End>
  <End Role="RelatedEntity" Type="Self.RelatedEntity" Multiplicity="*" />
  <ReferentialConstraint>
    <Principal Role="Entity">
      <PropertyRef Name="Id" />
    </Principal>
    <Dependent Role="RelatedEntity">
      <PropertyRef Name="IdPrincipal" />
    </Dependent>
  </ReferentialConstraint>
</Association>

You may also want to check this interesting post on conventions from Rowan.

Hacking Linq to Entities with Expressions Part 1: Clean Generic Repository

The repository pattern is intended to create an abstraction layer between the data access layer and the business logic layer of an application and is often used with Entity Framework. To avoid creating repository classes specific to each entity type it is a common practice to create a generic repository class that can be used for any entity. However, most examples I have seen could not really be used easily for any entity. For instance the repository type requires to provide a generic type for the key (e.g. class Repository<TEntity, TKey>) which should not really be required as the type of the entity is provided. Another thing to look at it is the GetById() method. It’s interesting at least for a couple of reasons:

  • key properties of different entity types may have different types (e.g. string key properties vs. int key properties)
  • key properties of different entity types may have different names

I have seen several ways of solving the above problems, for instance: enforcing all entities to be derived from a base (possibly generic) entity type containing the key property (it does not solve the problem of entity types with different key property names since the names of all key properties in the whole model will be the same) or passing a lambda expression/query to the GetById() method (feels wrong to me since the GetById() method should just take the value of the key for which to return the entity and not query, property name and whatnot). I thought a little bit on this I concluded that it should be possible to create a generic repository type without any additional overhead since we already have all the information that is needed. We know the entity type – it is the generic parameter to the repository type. We are able to reason about the entity type (i.e. figure out what the key property are) because we do have the context and – as a result – we can access all the metadata. Finally – for the GetById() we have the value of the key since it is provided by the user. The only obstacle here is to create the right query to send to the database but this can be easily solved by creating the query dynamically with Expression Trees.
** EDIT **
As pointed out by Florim below in the comments there is a better option than building a dynamic query – namely DbSet.Find() method. Not it is simpler (it does not require building the dynamic query) but also it may save a trip to the database if the entity is available locally. I am leaving the rest of the post as is to justify the “Hacking Linq to Entities with Expressions” title.
** EDIT END **
Let’s start from finding the key property – given the entity type TEntity (the generic entity type of the repository) and a valid DbContext (derived) instance (passed as a parameter to the constructor of the repository type) we can find the key property as follows:

private PropertyInfo GetKeyProperty(DbContext context)
{
    if (_keyProperty == null)
    {
        var edmEntityType = 
            ((IObjectContextAdapter)context)
                .ObjectContext
                .MetadataWorkspace
                .GetItems<EntityType>(DataSpace.CSpace)
                .Single(e => e.Name == typeof(TEntity).Name);

        _keyProperty = 
            typeof(TEntity)
                .GetProperty(
                    edmEntityType.KeyMembers.Single().Name, 
                    BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);

        if (_keyProperty == null)
        {
            throw new InvalidOperationException("Key property not found.");
        }
    }

    return _keyProperty;
}

Building the filter (i.e. the e => e.{keyProperty} == value) using Expression Trees is just a few lines of code:

private IQueryable<TEntity> Filter<TKey>(
    IQueryable<TEntity> dbSet,
    PropertyInfo keyProperty,
    TKey value)
{
    var entityParameter = Expression.Parameter(typeof(TEntity), "e");

    var lambda =
        Expression.Lambda<Func<TEntity, bool>>(
            Expression.Equal(
                Expression.Property(entityParameter, keyProperty),
                // no cast required if the passed value is of the 
                // same type as the key property
                typeof(TKey) == keyProperty.PropertyType ?
                    (Expression)Expression.Constant(value) :
                    (Expression)Expression.Convert(
                        Expression.Constant(value), keyProperty.PropertyType)),
                entityParameter);

    return dbSet.Where(lambda);
}

And finally we will connect the dots and create the GetById() method:

public TEntity GetById<TKey>(TKey value)
{
    return Filter(
        _context.Set<TEntity>(),
        GetKeyProperty(_context), value).SingleOrDefault();
}

Yes, the GetById() is generic. This is to avoid the value to be of the object type. Note that this does not add any overhead since the generic type does not have to be provided when invoking this method – the compiler is able to infer the type from the value of the parameter. In addition the Filter method will add a cast if the type of the passed value is different from the type of the key property (which will result in an exception at runtime if the provided value cannot be cast to the type of the key property).
For completeness here is the generic repository class (it does not include the Add, Delete etc. methods as they are not as interesting to me as the GetById() method):

public class Repository<TEntity> where TEntity : class
{
    private readonly DbContext _context;

    // for brevity composite keys are not supported
    private PropertyInfo _keyProperty;

    public Repository(DbContext context)
    {
        _context = context;
    }

    public TEntity GetById<TKey>(TKey value)
    {
        return Filter(
            _context.Set<TEntity>(),
            GetKeyProperty(_context), value).SingleOrDefault();
    }

    private IQueryable<TEntity> Filter<TKey>(
        IQueryable<TEntity> dbSet,
        PropertyInfo keyProperty,
        TKey value)
    {
        var entityParameter = Expression.Parameter(typeof(TEntity), "e");

        var lambda =
            Expression.Lambda<Func<TEntity, bool>>(
                Expression.Equal(
                    Expression.Property(entityParameter, keyProperty),
                    // no cast required if the passed value is of the
                    // same type as the key property
                    typeof(TKey) == keyProperty.PropertyType ?
                        (Expression)Expression.Constant(value) :
                        (Expression)Expression.Convert(
                            Expression.Constant(value), keyProperty.PropertyType)),
                    entityParameter);

        return dbSet.Where(lambda);
    }

    private PropertyInfo GetKeyProperty(DbContext context)
    {
        if (_keyProperty == null)
        {
            var edmEntityType =
                ((IObjectContextAdapter)context)
                    .ObjectContext
                    .MetadataWorkspace
                    .GetItems<EntityType>(DataSpace.CSpace)
                    .Single(e => e.Name == typeof(TEntity).Name);

            _keyProperty =
                typeof(TEntity)
                    .GetProperty(
                        edmEntityType.KeyMembers.Single().Name,
                        BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);

            if (_keyProperty == null)
            {
                throw new InvalidOperationException("Key property not found.");
            }
        }

        return _keyProperty;
    }

    // other "less interesting" methods
}

… and an example of how to use it. For the following, simple model with entities having keys with different names and of different types:

public class Customer
{
    public string CustomerId { get; set; }
    
    // ...
}

public class Order
{
    public Guid OrderId { get; set; }

    // ...
}

public class Item
{
    public int ItemId { get; set; }

    // ...
}

public class Context : DbContext
{
    public DbSet<Customer> Customers { get; set; }
    public DbSet<Order> Orders { get; set; }
    public DbSet<Item> Items { get; set; }
}

The entities can be retrieved by id as simple as:

using (var ctx = new Context())
{
    Console.WriteLine(
        new Repository<Customer>(ctx)
            .GetById("ALFKI").CustomerId);

    Console.WriteLine(
        new Repository<Order>(ctx)
            .GetById(new Guid("00000000-0000-0000-C000-000000000046")).OrderId);

    Console.WriteLine(
        new Repository<Item>(ctx)
            .GetById((byte)1).ItemId);
}

As you can see the code is clean – no extraneous information is provided – just the type of the entity and the value of the key (Yeah, the cast to byte is not needed – it is just to test that the logic in the dynamically built filter works)

Entity Framework 6 and Model/Database First Work Flows

Entity Framework 6 Alpha 2 has shipped. It has some new cool features (like custom conventions or automatic discovery of entity configurations) and a few other improvements (like improved queries for Linq Enumerable.Contains or changing the default isolation level when creating a SqlServer new database with CodeFirst). Most of the new features and many improvements are CodeFirst related or CodeFirst only. Still there are people who would prefer using a designer to draw a model and create the database or create a model from an existing database and tweak it. The latest version of the Entity Framework Designer which shipped in VS2012 supports only EF5 so it does not seem like it could handle EF6. However after seeing a question on the Entity Framework team blog a couple days ago I thought it would be interesting to really see if this is the case and what it would take to be able to actually use ModelFirst and DatabaseFirst work flows with EF6. In general I thought it might be possible – artifacts have not changed since EF5. As well as most APIs. The two fundamental changes to EF6 are changes to the provider model and all the “new” types that previously lived in System.Data.Entity.dll. New provider model should not be a big concern here – we care about the results here and not about how they are achieved. So, as long as the designer is able to create the database correctly (model first approach) or the edmx file from the database (database first approach) EF6 runtime should be able to use those. Changes to types seemed more worrisome – not only types themselves changed and have new identities but in many cases namespaces changed as well. Luckily the designer now uses T4 templates to generate the code from the edmx file so it is just a pure textual transformation. I expected that I would need to change the T4 templates a bit to make the code compile with EF6 but it should be easy. After all this mental work-out I decided to try it out. I opened VS 2012, created a new project, added a new ADO.NET Entity Framework model, removed references to System.Data.Entity.dll and EntityFramework.dll (5.0.0.0) and added a reference to the EF6 Alpha2 package using NuGet. Then I created a model from an existing database. The project compiled without errors. I added a few lines of code to bring some entities from the database and everything worked. Adding new entities worked as well. Finally I deleted my model and created a new model to try the Model First approach. Similarly I had to remove references to System.Data.Entity.dll and EntityFramework.dll (5.0.0.0) but other than that everything just worked. While what I did was not a very comprehensive test and using VS2012 for EF6 projects is in general not supported I am pretty confident it will work and should be sufficient until a version of the designer that supports EF6 ships.
(Yes, I am a bit disappointed with how easy it was. I hoped this would be a report from a battlefield where I was able to achieve my goal by using a hack here or adding a few lines of code there and maybe even producing a VSIX as a side effect. On the other hand I am happy that even though the post is a little boring the experience for users is much nicer. This is more important).

Safe Git Clean with MSBuild

I like to clean my source tree once in a while. Unfortunately msbuild {whateverproject}.csproj /t:Clean often falls short – for more complicated solutions with multiple projects some temporary files generated during the build almost always are left on the disk. Removing these should not be a big deal given that the important files are under source control – I can simply remove everything that is not tracked. With git you do that with git clean -xfd command and the files are gone. Usually, this is the (sad) moment when I realize that there was that file you forgot to add to the index or this directory that was not supposed to be added to the index but had some “temporary” data I have been using for a couple of weeks now. To save myself a grief I came up with an MSBuild target that saves everything that is going to be deleted to a backup directory before the clean up. This way even if I missed something not everything is lost – I can go to the backup directory and restore the file.
The target itself turned out quite simple. The biggest problem was to make git commands work with MSBuild. It’s possible to run git commands with Exec task. However Exec task does not capture results of the command. The only way I found to capture the results was to redirect the command output to a file and then read the file contents to an ItemGroup with ReadLinesFromFile task. Not ideal for sure but seems to work.
Using the target is easy as the target takes only two optional parameters – BackupDir and DeleteBackupDir. BackupDir is the path to a directory where files that are not tracked should be saved. If the value is not provided files will be saved to a subfolder of the %TEMP% folder. DeleteBackupDir tells whether to delete the BackupDir if one exists – ‘false’ is the default value. An exemplary command would be:
MSBuild {projectfile} /t:SafeGitClean /p:BackupDir=C:\temp\backup /p:DeleteBackupDir=false
The target itself looks as follows:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <Target Name="SafeGitClean">
    <PropertyGroup>
      <BackupDir Condition="'$(BackupDir)' == ''" >$(Temp)\SafeGitClean$([System.Guid]::NewGuid())</BackupDir>
      <DeleteBackupDir Condition="'$(DeleteBackupDir)' == ''">false</DeleteBackupDir>
      <TempFile>$(Temp)\SafeGitClean_UntrackedFiles$([System.Guid]::NewGuid())</TempFile>
    </PropertyGroup>
    
    <Message Text="$(BackupDir)" />

    <RemoveDir 
      Condition="'$(DeleteBackupDir)' == 'true' And Exists('$(BackupDir)')" 
      Directories="$(BackupDir)" />
    
    <Exec Command="git ls-files --other &gt; $(TempFile)" /> 

    <ReadLinesFromFile File="$(TempFile)">
      <Output TaskParameter="Lines" ItemName="UntrackedFile" />
    </ReadLinesFromFile>
        
    <ItemGroup>
      <UntrackedFileFixed Include="@(UntrackedFile->Replace('/', '\'))" />
    </ItemGroup>
    
    <Delete Files="$(TempFile)" TreatErrorsAsWarnings="true" />
      
    <Copy 
      SourceFiles="@(UntrackedFileFixed)" 
      DestinationFiles="@(UntrackedFileFixed->'$(BackupDir)\%(Identity)')" 
      OverwriteReadOnlyFiles="true" />
      
    <Exec Command="git clean -xfd" />      
  </Target>

</Project>

You can also find it in my github repo: https://github.com/moozzyk/MSBuild-Tasks

Pawel Kadluczka

Entity Framework 6 and pre-generated views

Entity Framework 6 is here. Even though it is in a very early stage it already looks exciting – a lot of improvements in Migrations (multi-tenant migrations, migrations history table customizations), Async, DI for resolving dependencies, code based configuration. Most of it (including features shipped in EF5 – e.g. enums) is running on both .NET Framework 4 and .NET Framework 4.5. In addition trying all of this is as simple as 1, 2, 3 – signed nightly builds are available on a nuget feed. We also take contributions and are thankful to everyone who has already contributed. There is one thing in EF6 that is missing however – the ability to create pre-generated views. I would love it to stay this way but unfortunately views are still one of the problematic areas in EF6. We see some promising activities around views and I hope this will help resolve or at least relieve the problem but for now the solution is still to pre-generate views. So, how do you pre-generate views in EF6? In the previous versions of EF you would either use EdmGen or EF Power Tools. Heck, you could even use my T4 templates. The problem is that all these tools are using System.Data.Entity.Design.dll to generate views and this code was not open sourced. Also, the code generated by System.Data.Entity.Design.dll will not work (without modifications) for EF6. So, it seems it is not possible to pre-generate views on EF6 then… But wait, EF6 is open source isn’t it? Why not make the code that is needed to create views public to enable view generation? It’s one option but there is also a second option – hack the system. While I strongly believe the first option is the right thing to do in the long run for now I went with the second option. There is one main reason for this – making some random functions public to make stuff work is less then ideal. It would be much better to add a nice(r), small API for view generation that could be used by tools that need to generate views. Therefore I decided to create a T4 template for generating views which, at the moment, is using reflection to get what it needs. I treat it just as a prototype (that’s one of the reasons why only C# version exists at the moment) and I hope it will help me define the right API for view generation. When I get to this point I will be able to remove the reflection hacks and just use the API. There is one more thing about the template itself. Since it is not possible to use System.Data.Entity.Design.dll the code needs to be generated by the template itself. It’s a bit more work but allows for much more flexibility. For instance, view generators based on System.Data.Entity.Design.dll were prone to the “No logical space left to create more user strings” error caused by the number of strings in the generated code that could be so big that it reached the .NET metadata format limit on the number of user string characters. This error would prevent an application from starting. This problem is now solved – the template creates an xml file that contains actual view definitions and saves this file in the assembly as an embedded resource. When the EF stack requests views the code generated by the template loads views from the embedded xml file. Using the template is not much different from using the templates for EF5 as it is too published on Visual Studio Code Gallery. First, if you have not already, setup the nuget feed containing EF6 assemblies. Create a new project and add the EF6 nuget package (make sure to select “Include Prelease” in the dropdown at the top of the window) from the feed you created. Now you can start writing your app. Once you have something that compiles right-click on your project and select Add→New Item (Ctrl+Shift+A). In the “Add New Item” window select “Online” on the left. You may want to filter by EF or EF6. Select the “EF6 CodeFirst View Generation T4 Template for C#”. Change the name of the .tt file so that it starts with the name of your context and press the “Add” button:

Once it’s done you should see the template and two new files added to your project – one of the files is the embedded xml resource file containing views and the second is the C# files used to load views from the first file:

If you need to uninstall the templates go to Tools→Extensions and Updates… select the template from the list and click the “Uninstall” button.

That’s it for now. Use EF6, enjoy the template and report bugs for both…

Tagged , , ,

MSBuild Zip task without external dependencies

I have seen quite a few build systems in my life. Most of them were very complicated with a ton of crap a lot of tools, dependencies, perl scripts, batch files, nested targets files and God knows what else. Figuring out how something worked (or, more often, why something did not work) was time consuming and very frustrating. One of the reasons for this was people were adding some stuff to the build system but no one has ever removed anything. What was even more annoying was that new dependencies oftentimes added tens of new files just to enable one small thing. Not only the enlistments were huge (how about ~200GB without QA tests?) but also configuring the machine to be able to build all this and run the tests was a sort of black magic. Because of all this I always felt bad about the fact that MSBuild did not have a Zip task available out-of-the-box. I feel that not having this one thing is the first step to having a build system that everyone hates. Yes, I know there wasn’t a Zip library in .NET Framework until now. Yes, I know there are third party Zip libraries out there. Yes, I know about MSBuild Community Tasks. Yes, I know MSBuild can somehow zip files internally as it can create VSIX files which are zip files… oops – this probably was not the best example. Anyways, not having a built-in Zip tasks means that you need to add some dependencies to your build system to be able to build your project. This will lead to a build system that no one wants to touch to not break anything. What about sharing just a single project? Like for instance my Code First view gen templates? Is it OK to tell people – “you can build it on your own – here is the project, but first you need to install this and this and this or it won’t build”? I don’t think it’s OK, but before I could not help much. Fortunately, a Zip library was finally added to .NET Framework 4.5. This allowed me creating my own Zip task. How is this task different from, for instance, the Zip task from the MSBuild community tasks? I created it as an inline task. As a resull it is just a small text file I can import to my projects. It can be checked in to my source control. It does not require any additional components being installed or present on the machine apart from what’s already there. If I need to know what the task is doing I can see the source without Reflector. I can easily change the task without having to recompile half of my build system just to be able to build what I actually want to build. (See the Disclaimer at the bottom of the page). The task looks just like this:

<UsingTask TaskName="Zip" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
    <ParameterGroup>
      <InputFileNames ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
      <OutputFileName ParameterType="System.String" Required="true" />
      <OverwriteExistingFile ParameterType="System.Boolean" Required="false" />
	</ParameterGroup>
    <Task>
      <Reference Include="System.IO.Compression" />
      <Using Namespace="System.IO.Compression" />
      <Code Type="Fragment" Language="cs">
      <![CDATA[        
        const int BufferSize = 64 * 1024;

        var buffer = new byte[BufferSize];
        var fileMode = OverwriteExistingFile ? FileMode.Create : FileMode.CreateNew;

        using (var outputFileStream = new FileStream(OutputFileName, fileMode))
        {
          using (var archive = new ZipArchive(outputFileStream, ZipArchiveMode.Create))
          {
            foreach (var inputFileName in InputFileNames.Select(f => f.ItemSpec))
            {
              var archiveEntry = archive.CreateEntry(Path.GetFileName(inputFileName));

              using (var fs = new FileStream(inputFileName, FileMode.Open))
              {
                using (var zipStream = archiveEntry.Open())
                {
                  int bytesRead = -1;
                  while ((bytesRead = fs.Read(buffer, 0, BufferSize)) > 0)
                  {
                    zipStream.Write(buffer, 0, bytesRead);
                  }
                }
              }
            }
          }
        }        
      ]]>
      </Code>
    </Task>
  </UsingTask>

Using the task is simple. Put the task to a separate file and import the file to the csproj file. In fact the file is available in my github repo – https://github.com/moozzyk/MSBuild-Tasks. Once you import the file to the project you just invoke the task as you would invoke any other task – for example (this is an actual except from one of my csproj files):


  <Import Project="common.tasks" />
  
  <Target Name="BeforeBuild">
    <ItemGroup>
      <FilesToZip Include="$(ProjectDir)\PayloadUnzipped\*.*" />
    </ItemGroup>
    <Zip 
      InputFileNames="@(FilesToZip)"
      OutputFileName="$(ProjectDir)$(TargetZipFile)"
      OverwriteExistingFile="true" />
  </Target>

That’s pretty much it. Works for me and hopefull will work for you.

Disclaimer:
I am not trying to diminish MSBuild Community Tasks or claim that inline tasks will solve all problems of this world. I am trying to say that for small simple tasks inline tasks can be just much more convenient.

Entity Framework Code First View Generation Templates Updated

Everyone fights to be on the first page of the Google search results. But sometimes it’s not cool. One of the cases when it’s not cool is when you introduce a bug that causes a link to your blog to pop up on the first page of the Google search results. Can it be worse? How about the link to your blog being not only on the first page of the Google search results but also *the only* link on the Google search results. Can it be even worse? How about the only result not only in Google but in Bing as well (Hey http://bingiton.com, it’s a tie: ). Sure, it will add some traffic to your blog but it’s a bad kind of traffic. Desperate people looking for a solution to a problem that seemingly can be solved by only one guy on this planet. Now, I feel unique. Unfortunately in a bad sense. Why? Because a bug that was in T4 templates for generating views for CodeFirst apps made all the above a true story. When the templates were used on Visual Studio 2012 the user would get the an exception saying: “The default target Entity Framework version requires the edmx schema version 2.0.0.0 or lower. The specified schema is version 3.0.0.0. To avoid this warning specify the target Entity Framework version explicitly.” (now Google and Bing should show two results ;) ). I noticed this the first time when I wanted to show the integration of Visual Studio and Visual Studio Gallery to my sister. Then it was reported by a reader as a comment to the first post on view generation and code first I wrote some time ago. Then I saw it more and more often in search engine terms in the stats of this blog. Today I finally found some time to fix the bug and update the templates to the Visual Studio Gallery. I tested the fix on Visual Studio 2012 (C# and VB.NET projects, both targeting .NET Framework 4.5 and .NET Framework 4) and on Visual Studio 2010 (C# and VB.NET project, targeting .NET Framework 4) and did not get the exception anymore. The new templates have version 1.0.1. If you installed version 1.0.0 you probably need to uninstall the old templates (Tools → Extensions and Updates) and install new ones. I have not tried installing new templates without uninstalling old ones – it may or may not work. If you hit any new problems let me know.

Using exisiting enum types in Entity Framework 5

When we first showed long awaited support for enum types in the Microsoft Entity Framework June 2011 CTP (which you should not be using) we received a lot of feedback. One of the areas we got lots of comments in were restrictions around mapping O-Space (i.e. CLR) enum types to C-Space (i.e. EDM) enum types. The requirements were that the full name of the C-Space enum type must match the full name of the O-Space enum type and that enum members of both types had to be an exact match. These two rules basically prevented from using already existing enum types. While I think that in most EF scenarios people will use enum types they create rather than re-use existing enum types, there was really no reason to block scenarios where existing enum types were being used. So, we went ahead and relaxed the above rules. Firstly, when trying to match enum types we only compare type name and ignore namespaces. Secondly, members are no longer required to match exactly but any member on the EDM enum type must have a counterpart in the CLR enum type (note that this is unidirectional i.e. members from the CLR enum type don’t have to exist in the EDM enum type). The latter effectively allows an EDM enum type that has no member whatsoever to be mapped to a CLR enum type that has some members (* – disclaimer: see at the bottom). After introducing the changes using existing enum types became easy. When using CodeFirst approach the entity will just have a property of the desired enum type e.g.:

    public class MyEntity
    {
        public int Id { get; set; }
        public System.DayOfWeek Day { get; set; }
    }

That’s it – it should “just work”. But what about ModelFirst or DatabaseFirst? When you look at the “Add Enum Type” dialog:

Add Enum Type Dialog

Add Enum Type Dialog

you will see the “Reference External Type” checkbox at the bottom. If you want to use an existing enum type just check this checkbox and enter the fully qualified name of the enum type in the box below the checkbox. Remember that the name of the enum type being created (“Enum Type Name” box) must match the name of the CLR enum type. Underlying types must also match. If you now close the dialog and look at the EDMX (you need to open the .edmx file with Xml Editor) you will see the following enum type definition (I copied the namespace definition to make the xml fragment valid):

<EnumType Name="DayOfWeek" cg:ExternalTypeName="System.DayOfWeek" xmlns:cg="http://schemas.microsoft.com/ado/2006/04/codegeneration" />

The interesting fact about the definition above is that the type has no members (we don’t need them). But the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute is even more interesting – it contains the name of the CLR type we entered when adding a new enum type to the model. As the namespace uri indicates this attribute is used by code generation. When the code is generated whenever the enum type is referenced the templates will use the value from the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute rather than the value from the Name attribute. Here is the result:


//------------------------------------------------------------------------------
// <auto-generated>
//    This code was generated from a template.
//
//    Manual changes to this file may cause unexpected behavior in your application.
//    Manual changes to this file will be overwritten if the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

namespace ConsoleApplication1
{
    using System;
    using System.Collections.Generic;
    
    public partial class MyEntity
    {
        public int Id { get; set; }
        public System.DayOfWeek Day { get; set; }
    }
}

To make it clear – the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute is not used at runtime. It exists purely to support code generation. Another caveat is that the attribute works only in DbContext based templates – which are the default code generators in EF5 and Visual Studio 2012.
Using existing enum types in Entity Framework 5 should be now easy!

* – as reported by Paweł Madurski on stackoverflow there is a bug where EF still checks for enum members when using Contains with an array of enum values (or to be more precise where a query requires creating the DbConstantExpression instance an enum value). This is tracked on Entity Framework codeplex page: http://entityframework.codeplex.com/workitem/623

Follow

Get every new post delivered to your Inbox.

Join 137 other followers