Using exisiting enum types in Entity Framework 5

When we first showed long awaited support for enum types in the Microsoft Entity Framework June 2011 CTP (which you should not be using) we received a lot of feedback. One of the areas we got lots of comments in were restrictions around mapping O-Space (i.e. CLR) enum types to C-Space (i.e. EDM) enum types. The requirements were that the full name of the C-Space enum type must match the full name of the O-Space enum type and that enum members of both types had to be an exact match. These two rules basically prevented from using already existing enum types. While I think that in most EF scenarios people will use enum types they create rather than re-use existing enum types, there was really no reason to block scenarios where existing enum types were being used. So, we went ahead and relaxed the above rules. Firstly, when trying to match enum types we only compare type name and ignore namespaces. Secondly, members are no longer required to match exactly but any member on the EDM enum type must have a counterpart in the CLR enum type (note that this is unidirectional i.e. members from the CLR enum type don’t have to exist in the EDM enum type). The latter effectively allows an EDM enum type that has no member whatsoever to be mapped to a CLR enum type that has some members (* – disclaimer: see at the bottom). After introducing the changes using existing enum types became easy. When using CodeFirst approach the entity will just have a property of the desired enum type e.g.:

    public class MyEntity
    {
        public int Id { get; set; }
        public System.DayOfWeek Day { get; set; }
    }

That’s it – it should “just work”. But what about ModelFirst or DatabaseFirst? When you look at the “Add Enum Type” dialog:

Add Enum Type Dialog
Add Enum Type Dialog

you will see the “Reference External Type” checkbox at the bottom. If you want to use an existing enum type just check this checkbox and enter the fully qualified name of the enum type in the box below the checkbox. Remember that the name of the enum type being created (“Enum Type Name” box) must match the name of the CLR enum type. Underlying types must also match. If you now close the dialog and look at the EDMX (you need to open the .edmx file with Xml Editor) you will see the following enum type definition (I copied the namespace definition to make the xml fragment valid):

<EnumType Name="DayOfWeek" cg:ExternalTypeName="System.DayOfWeek" xmlns:cg="http://schemas.microsoft.com/ado/2006/04/codegeneration" />

The interesting fact about the definition above is that the type has no members (we don’t need them). But the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute is even more interesting – it contains the name of the CLR type we entered when adding a new enum type to the model. As the namespace uri indicates this attribute is used by code generation. When the code is generated whenever the enum type is referenced the templates will use the value from the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute rather than the value from the Name attribute. Here is the result:


//------------------------------------------------------------------------------
// <auto-generated>
//    This code was generated from a template.
//
//    Manual changes to this file may cause unexpected behavior in your application.
//    Manual changes to this file will be overwritten if the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

namespace ConsoleApplication1
{
    using System;
    using System.Collections.Generic;
    
    public partial class MyEntity
    {
        public int Id { get; set; }
        public System.DayOfWeek Day { get; set; }
    }
}

To make it clear – the {http://schemas.microsoft.com/ado/2006/04/codegeneration}ExternalTypeName attribute is not used at runtime. It exists purely to support code generation. Another caveat is that the attribute works only in DbContext based templates – which are the default code generators in EF5 and Visual Studio 2012.
Using existing enum types in Entity Framework 5 should be now easy!

* – as reported by Paweł Madurski on stackoverflow there is a bug where EF still checks for enum members when using Contains with an array of enum values (or to be more precise where a query requires creating the DbConstantExpression instance an enum value). This is tracked on Entity Framework codeplex page: http://entityframework.codeplex.com/workitem/623

Yes, you can! (create mutifile zip files in .NET Framework (finally))

I recently have been working on moving some projects from an old legacy build system to a new shiny build system. After understanding intricacies of the old build system I was able to isolate and remove 99% stuff we did not really need and comment out the remaining 1% we needed but could not make it work easily. I felt proud of myself because I was even able to build something in the new build system (OK, maybe the stuff we did not need was actually 98.9%?). So, I started uncommenting things we needed but were not as straightforward to fix. One of the first things I came across was the need of zipping files after building. I thought I would just use the MSBuild Zip task and call it done. Unfortunately it turned out that out of the box MSBuild does not have a Zip task. I could find it in MSBuild community tasks but for certain reasons we could not take a dependency on these. We also did not have a zip-like command line tool we could take a dependency on. One of the last sunny days around Seattle this year started looking gloomy to me. I asked folks on our team and they had a few ideas like using PowerShell (we don’t really need it anywhere else so again I wanted to avoid it), leveraging a third party library we could use to create a task or finally using GZipStream. I liked the idea of using GZipStream – I could create and inline MSBuild task which would be clean and simple. However gzip only supports compressing one file and I needed to zip several files so, I could not use it. I spent more time looking for something suitable and thought I found it – ZipPackage!. Again, my hopes were quickly dashed. This guy needs a metadata file containing content types of the included files. This file is then added to the archive. But I needed just a POZF™ (Plain Old Zip File) without any additional crap inside! Afternoon was getting really dark. I started looking at the option of using a third party library. It would add complexity to the build system. I would have to build tools before building projects. Suddenly, I had this feeling that instead of having a lightweight, easy and fast build we will have a new version of what we had before. Each time I wanted to start working on this my hands refused and decided to open a browser and check the news or something along these lines. So, I gave one more shot at finding a better solution but I could only find that there is an MSBuild Zip task in the community tasks and that the GZipStream basically can compress only one file and that the ZipPackage requires some metadata. I was loosing hope when suddenly on Xth page of results (where X > 5) I found this – the ZipArchive class. The ZipArchive is a new class in the .NET Framework 4.5 that allows creating zip files containing not only multiple files but also the whole folder hierarchy! In just one second all the clouds were gone. If there had not been for pierogi I would have probably stayed playing with this class until my MSBuild problem was resolved. On the other hand I don’t like when pierogi are too hot so I stayed a few more minutes just to see if it was easy to create a zip file. Here is what I came up with (I checked only briefly but could not find an example how to create a new archive on msdn):

public static void Main()
{
    Zip(
        new string[] 
        { 
            @"C:\Temp\file1.txt", 
            @"C:\temp\file2.txt", 
            @"C:\Source\ConsoleApplication1\ConsoleApplication1\Program.cs" 
        },
        @"c:\temp\out.zip");
}

static void Zip(string[] inputFileNames, string outputFileName)
{
    using (var outputFileStream = new FileStream(outputFileName, FileMode.Create))
    {
        using (var archive = new ZipArchive(outputFileStream, ZipArchiveMode.Create))
        {
            foreach (var inputFile in inputFileNames)
            {
                AddFileToArchive(archive, inputFile);
            }
        }
    }
}

static void AddFileToArchive(ZipArchive archive, string inputFile)
{
    const int BufferSize = 64 * 1024;

    // relies on inputFile being an absolute path 
    // - Substring(3) removes :\ from the path
    var archiveEntry = archive.CreateEntry(inputFile.Substring(3));

    using (var fs = new FileStream(inputFile, FileMode.Open))
    {
        using (var zipStream = archiveEntry.Open())
        {
            byte[] buffer = new byte[BufferSize];
            int bytesRead = -1;
            while ((bytesRead = fs.Read(buffer, 0, BufferSize)) &gt; 0)
            {
                zipStream.Write(buffer, 0, bytesRead);
            }
        }
    }
}

I can open the archive I created in the Windows Explorer, browse and open files. No additional metadata files are added to the archive – it’s a POZF™. So – yes, you now can create a multifile zip archives in .NET Framework without using third party libraries. And now creating an inline MSBuild Zip task that fits my new build system is easy. This makes me happy!

Pawel Kadluczka

Digital Clock on Arduino (Uno) with Nokia LCD Display

When I heard about Arduino a couple years ago (yes, I was a bit late to the party) I immediately fell in love with it. The main reason was that it allowed me to try things that otherwise would require a lot of time and effort from a total electronics noob like me. With Arduino, a breadboard, a few LEDs and a couple of cables I was ready to write my very first program – controlling the LED with Arduino. It was fun! But I wanted to do something more interesting. I figured that while flashing a LED is fun displaying something on a screen must be even more fun. Soon I found that it was possible to buy a cheap Nokia 5110 LCD displays on ebay. I had some doubts whether I would be able to make it work but figured out that even if I wouldn’t I could always use it as a key chain or a similar geeky gadget. Indeed, when the display arrived I did have some problems to make it work. First, I was not sure how to connect the display to Arduino. Even though I found a few websites that showed how to do that I had hard time to make it work. I also tried a couple of sample programs that in theory should display something on the display but in practice they did not. Eventually, I found that one of the cables were connected incorrectly and the sequence to initialize the display did not work (at least on my display). Seeing something on the screen was a huge step forward and I finally could think about using the display for my own purpose. I decided that I could do a digital clock as it seemed a reasonably sized project which would still require understanding how the display really works. First I wanted to “design” fonts for the clock. For this I just used graph paper and this reminded me designing sprites for my C-64 in the late eighties. After designing all digits I had to understand how to encode these so that they could be drawn easily on the screen. I found the datasheet for the PCD8544 which is a controller for the display. This was the first time in my life I had to read a datasheet for an electronic component but I was able to understand what the most important things and the datasheet turned out to be an interesting read. Soon, I found what I wanted – the display has 6 rows and 84 columns. Each column in a row has 8 pixels. To draw something on a screen first you need to set a pointer (row and column) and then write a byte (8 bits) to draw the column. When you draw a column the pointer automatically advances to the next column – this is handy since you don’t really have to set the pointer all the time if you draw something like a bitmap. The mechanics of drawing on the screen surprised me. Firstly, I found it kind of similar to advanced graphics modes on 8-bit computers. Secondly, it was interesting to see that the programmer’s convenience totally lost with the low level design of the controller (or may be I am just spoiled by higher level APIs which allow me just to pass x and y to draw a point). After figuring out how drawing looks like encoding my digits was relatively simple. Now that I was able to draw digits on the screen it was the time to implement the clock. Initially I wanted to use the loop function but I quickly realized that it would not really work for a clock. I did not know how long drawing would take and therefore the clock would not work correctly. I could calculate how much time drawing took and then cut the wait time accordingly but it didn’t feel like the right solution. Obviously, there was one more (and I believe the only correct for this kind of problem) solution – interrupts. The only problem was to set everything up correctly. I spent some time reading Arduino datasheet but things did not really want to work. Fortunately I found this page which helped tremendously. After setting up interrupts things went smoothly and soon I had my digital clock up and running. I actually ended up using both the loop function and the interrupt. The interrupt just increases the timer while all the drawing takes place in the loop function. This way I don’t have to worry that drawing takes too much time and when a new interrupt happens the previous one is still being handled. That’s pretty much it. Here is a short video showing the clock in action:

If you want to try my project here is how I connected the Nokia 5110 display to Arduino:

Arduino Uno               Nokia 5110 Display
3.3V   ------------------ 1-VCC
PIN #7 ------------------ 3-SCE
PIN #6 ------------------ 4-RST
PIN #5 ------------------ 5-D/C
PIN #4 ------------------ 6-DNK(MOSI) (SDIN)
PIN #3 ------------------ 7-SCLK

I also posted the sketch on my github.

One more thing I always wanted to do but never really have had time to do was to add some buttons to enable setting the clock – at the moment the clock always starts at 00:00. Maybe one day I will come back to this project again (like I did today just to publish it) and will add the buttons…

Pawel Kadluczka

Unit testing XSD schemas

Once in a while a new task no-one is really eager to work on pops-up. From my experience in teams that are not focusing on or use extensively Xml related technologies most (if not all) tasks that have anything to do with XSD schemas belong to this group. This was the case in our team recently and I ended up to be a “volunteer” since the schedule was tight and I previously worked on the managed Xml team. So, I started refreshing my rusted XSD skills and soon I got something that more or less worked. It was a good starting point but then I asked myself – “how do I test this”. I needed something lightweight that would fit in our unit tests. I briefly searched the Internet but could not find anything that would be suitable.  As the old saying goes, necessity is mother of invention, so I came up with my own way of testing the schema. I like it because it contains just 3 small (less than 30 lines total) helper methods, one helper schema and most of the unit tests are just 2-3 lines. The tests actually also helped me come up with a better design I originally had. Note, I don’t know if this is the “right” approach or if it would scale for bigger schemas. I only know that for the schema I had to write it worked fine.

So, let’s say we need to write a schema for Xml files that have a structure like this:

<Settings>
    <ServiceProvider Type="typeName">
      <Setting Name="Setting1" Value="Value1" />
      <Setting Name="Setting2" Value="Value2" />
    </ServiceProvider>

   <Factory Type="typeName">
     <Setting Name="Setting1" Value="Value1" />
     <Setting Name="Setting2" Value="Value2" />
     <Setting Name="Setting3" Value="Value3" />
   </Factory>
 </Settings>

and that both ServiceProvider and Factory elements are optional.

First we need to create a starting schema. For new schemas I usually create a sample Xml file, open it in Visual Studio and use Xml → Create Schema. The schema created by the VS is not really usable but gives me something I can iterate on. The main problem with the generated schema is that all the types are defined inline. This makes it hard to test – ideally we would like to test each type separately. Generating inline types leads to another problem – each element has its own type even if the same element is used repeatedly (let alone cases where the same types are used for different elements or where inheritance is involved). The key to testing a schema is to have simple types. The simpler the type the easier it is to test. Once a type is tested it can be used as a building block to build more complicated types but it won’t require any more comprehensive testing as part of the more complicated type. For the Xml structure above we can identify three types:

  • Setting (for Setting element)
  • ServiceTypeInitializer (a common type for ServiceProvider and Factory elements)
  • Settings (for Settings element)

The problem with unit testing all these types in separation is that the schema itself should not allow any but Settings element as the document element. Fortunately for testing purposes we can create a helper schema that will allow document elements of types that are normally not allowed to be document elements. We will conditionally add this helper schema to the schema set used for validating the input Xml. Why the helper schema needs to be added conditionally? The tested schema should not allow any but the Settings element as the document element. So, when testing the Settings element we must not add the helper schema to the schema set to make sure that this is the only element allowed as the document element. Let’s see how this looks like in practice. Here is the schema created by refactoring the initial schema created by Visual Studio:


<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="Settings" type="Settings_Type" />

  <xs:complexType name="Settings_Type">
    <xs:sequence>
      <xs:element name="ServiceProvider" type="ServiceTypeInitializer_Type" minOccurs="0" maxOccurs="1" />
      <xs:element name="Factory" type="ServiceTypeInitializer_Type" minOccurs="0" maxOccurs="1" />
    </xs:sequence>
  </xs:complexType>

  <xs:complexType name="ServiceTypeInitializer_Type">
    <xs:sequence>
      <xs:element maxOccurs="unbounded" name="Setting" type="Setting_Type" />
    </xs:sequence>
    <xs:attribute name="Type" type="xs:string" use="required" />
  </xs:complexType>

  <xs:complexType name="Setting_Type">
    <xs:attribute name="Name" type="xs:string" use="required" />
    <xs:attribute name="Value" type="xs:string" use="required" />
  </xs:complexType>
</xs:schema>

Now let’s create the helper schema that will allow testing each of the types separately:


<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Setting" type="Setting_Type" />
  <xs:element name="ServiceTypeInitializer" type="ServiceTypeInitializer_Type" />
</xs:schema>

(In the above schema the Settings element is not present since it’s already allowed at the top level by the other schema). After creating the helper schema we need a function that will validate Xml documents against our schemas:


private static IEnumerable<ValidationEventArgs> RunValidation(string inputXml, bool includeHelperSchema)
{
    var schemaSet = new XmlSchemaSet();
    schemaSet.Add(schemaUnderTest);

    if (includeHelperSchema)
    {
        schemaSet.Add(helperSchema);
    }

    var readerSettings = new XmlReaderSettings()
    {
        Schemas = schemaSet,
        ValidationType = ValidationType.Schema,
        ValidationFlags = XmlSchemaValidationFlags.ReportValidationWarnings,
    };

    var events = new List<ValidationEventArgs>();
    readerSettings.ValidationEventHandler += (s, e) => { events.Add(e); };

    using (var reader = XmlReader.Create(new StringReader(inputXml), readerSettings))
    {
        while (reader.Read())
            ;
    }

    return events;
}

There are two interesting points here. First we need to turn on reporting validation warnings. This is because XmlSchemaSet has a nasty behavior where no error is reported if the document element of the validated Xml document is in different namespace that the targetNamespace of the schema. This may result in accepting documents that are not being validated at all. Turning on reporting warnings is the first step to catch this condition. The second interesting point is that schema validation will throw exceptions for validation errors but not for warnings. Again, to catch the condition where the expected and actual namespaces don’t match we have to set XmlReaderSettings.ValidationEventHandler which will be invoked for both validation errors and warnings. Other than that the method is pretty straightforward – we create an XmlSchemaSet instance and add the schema under test and conditionally the helper schema. Then we create an XmlReaderSettings object and set it up for schema validation. We use the reader settings to create a validating XmlReader. Finally we read the input xml with the validating reader – all errors and warnings are reported by invoking the validation event handler we set.
With the test driver method ready we can start writing test cases. We write test cases for each type starting from “leaf” types (i.e. types that are defined using only pre-defined schema types) moving to more complex types. If a type contains an element of a type that has already been tested we just test that schema accepts an Xml with the simplest child element of that type and, if the type is mandatory, the Xml is rejected if it does not contain the element. If there are multiple elements of the same type we just write test cases to test the type itself and not test cases to test all the possible elements of that type (they will be tested when testing their parent type). If there was a hierarchy we would write test cases for the base type and then test cases just for what was added (or removed – in case of derivation by restriction) in the derived type. The test cases themselves are simple – in most cases a hardcoded minimal Xml document is validated using the validation method we created and we check whether expected errors are reported or that there are no errors for valid Xml documents. Some examples:


[Fact]
public void Schema_accepts_minimal_valid_Xml()
{
    Assert.True(!RunValidation("<Settings />", false).Any());
}

[Fact]
public void Schema_rejects_Setting_Type_without_Name()
{
    var error = 
        RunValidation(@"<Setting Value=""ABC"" />", true)
        .Single();

    Assert.Equal(XmlSeverityType.Error, error.Severity);
    Assert.Equal(
        "The required attribute 'Name' is missing.",
        error.Message);
}

An exemplary test suite using XUnit can be found on my github. The Readme contains details about requirements, setting up the environment, building and running tests. If you just want to see what’s most interesting (i.e. the code) you can find it here

Pawel Kadluczka

Entity Framework Code First View Generation Templates On Visual Studio Code Gallery

Some time ago I created T4 templates for creating pre-generated views for Entity Framework Code First applications. I wanted to make them available as soon as possible so I just uploaded them as a zip file to one of my sites and provided a link. This worked as a short-term solution but long-term I wanted something better. Something that would not require manual work. Something that would integrate with Visual Studio seamlessly. Something that is actually called Visual Studio Code Gallery. And it happened yesterday. Yesterday I published the templates on the Visual Studio Code Galery.

Using the templates

First you need to download the templates. You can do it directly from Visual Studio. Right click on your project and select Add -> New Item (Ctrl+Shift+A). In the Add New Item dialog go to “Online templates”:

Add New Item - Online Templates

and search for “EF Views”. This should make the “EF CodeFirst View Generation T4 Template for C#/VB” show up (note: only the template for the language of the current project will show up).

Add New Item - Search Templates

Change the name of the file at the bottom to {Context}.Views.tt where {Context} is the name of the class derived from DbContext you want to create pre-generated views for.
Click “Add” to add the template to your project. Wait for the views to be generated (note: for bigger models view generation may take an extended amount of time).

You can also install templates manually just by downloading vsix files from Visual Studio Code Gallery and pressing “Enter”. Here are direct links to the templates:

Once you installed the templates you can find them in the “Code” category. Right click on your project and select Add -> New Item (Ctrl+Shift+A). In the “Add New Item” dialog go to the “Code” section:

Add New Item - Using Installed Templates

If needed the templates can be uninstalled from Extension Manager (Tools -> Extension Manager):

Unistalling Templates

Happy coding.

Pawel Kadluczka