Category Archives: .NET Framework

Support for Store Functions (TVFs and Stored Procs) in Code First (Entity Framework 6.1)

See what’s new in Beta here

Until Entity Framework 6.1 was released store functions (i.e. Table Valued Functions and Stored Procedures) could be used in EF only when doing Database First. There were some workarounds which made it possible to invoke store functions in Code First apps but you still could not use TVFs in Linq queries which was one of the biggest limitations. In EF 6.1 the mapping API was made public which (along with some additional tweaks) made it possible to use store functions in your Code First apps. Note, that it does not mean that things will start working automagically once you upgrade to EF6.1. Rather, it means that it is now possible to help EF realize that it actually is capable of handling store functions even when Code First approach is being used. Sounds exciting doesn’t it? So, probably the question you have is:

How do I do that?

To understand how store functions could be enabled for Code First in EF 6.1 let’s take a look first at how they work in the Database First scenario. In Database First you define methods that are driving the execution of store functions in your context class (typically these methods are generated for you when you create a model from the database). You use these methods in your app by calling them directly or, in case of TVFs, in LINQ queries. One thing that is worth mentioning is that these methods need to follow certain conventions otherwise EF won’t be able to use them. Apart from methods defined in your context class store functions must also be specified in the artifacts describing the model – SSDL, CSDL and MSL (think: edmx). At runtime these artifacts are loaded to MetadataWorkspace object which contains all the information about the model.
In Code First the model is being built from the code when the application starts. Types are discovered using reflection and are configured with fluent API in the OnModelCreating method, attributes and/or conventions. The model is then loaded to the MetadataWorkspace (similarly to what happens in the Database First approach) and once this is done both – Code First and Database First operate in the same way. Note that the model becomes read-only after it has been loaded the MetadataWorkspace.
Because Database First and Code First converge at the MetadataWorkspace level enabling discovery of store functions in Code First along with additional model configuration should suffice to add general support for store functions in Code First. Model configuration (and therefore store function discovery) has to happen before the model is loaded to the MetadataWorkspace otherwise the metadata will be sealed and it will be impossible to tweak the model. There are three ways we can configure the model in Code First – configuration attributes, fluent API and conventions. Attributes are not rich enough to configure store functions. Fluent API does not have access to mapping. This leaves conventions. Indeed a custom model convention seems ideal – it gives you access to the model which in EF 6.1 not only contains conceptual and store models but also modifiable mapping information. So, we could create a convention which discovers methods using reflection, then configures store and conceptual models accordingly and defines the mapping. Methods mapped to store functions will have to meet some specific requirements imposed by Entity Framework. The requirements for methods mapped to table valued functions are the following:

  • return type must be an IQueryable<T> where T is a type for which a corresponding EDM type exists – i.e. is either a primitive type that is supported by EF (for instance int is fine while uint won’t work) or a non-primitive type (enum/complex type/entity type) that has been configured (either implicitly or explicitly) in your model
  • method parameters must be of scalar (i.e. primitive or enum) types mappable to EF types
  • methods must have the DbFunctionAttribute whose the first argument is the conceptual container name and the second argument is the function name. The container name is typically the name of the DbContext derived class however if you are unsure you can use the following code snippet to get the name:
    Console.WriteLine(
        ((IObjectContextAdapter) ctx).ObjectContext.MetadataWorkspace
            .GetItemCollection(DataSpace.CSpace)
            .GetItems<EntityContainer>()
            .Single()
            .Name);
    
  • the name of the method, the value of the DbFunction.FunctionName and the queryString name passed to the CreateQuery call must all be the same
  • in some cases TVF mapping may require additional details – a column name and/or the name of the database schema. You can specify them using the DbFunctionDetailsAttribute. The column name is required if the method is mapped to a TVF that returns a collection of primitive values. This is needed because EF requires providing the name of the column containing the values and there is no way of inferring this information from the code and therefore it has to be provided externally by setting the ResultColumnName property of the DbFunctionDetails attribute to the name of the column returned by the function. The database schema name needs to be specified if the schema of the TVF being mapped is different from the default schema name passed to the convention constructor and can be done by setting the DatabaseSchema property of the DbFunctionDetailsAttribute.

The requirements for methods mapped to stored procedures are less demanding and are the following:

  • the return type has to be ObjectResult<T> where T, similarly to TVFs, is a type that can be mapped to an EDM type
  • you can also specify the name of the database schema if it is different from the default name by setting the DatabaseSchema property of the DbFunctionDetailsAttribute. (Because of how the result mapping works for stored procedures setting the ResultColumnName property has no effect)

The above requirements were mostly about method signatures but the bodies of the methods are important too. For TVFs you create a query using the ObjectContext.CreateQuery method while stored procedures just use ObjectContext.ExecuteFunction method. Below you can find examples for both TVFs and stored procedures (also notice how parameters passed to store functions are created). In addition the methods need to be members of the DbContext derived type which itself is the generic argument of the convention.
Currently only the simplest result mapping where names of the columns returned from the database match the names of the names of the properties of the target type (except for mapping to scalar results) is supported. This is actually a limitation in the EF Code First where more complicated mappings would currently be ignored in most cases even though they are valid from the MSL perspective. There is a chance of having more complicated mappings enabled in EF 6.1.1 if appropriate changes are checked in by the time EF 6.1.1 ships. From here there should be just one step to enabling stored procedures returning multiple resultsets in Code First.
Now you probably are a bit tired of all this EF mumbo-jumbo and would like to see

The Code

To see the custom convention in action create a new (Console Application) project. Once the project has been created add the EntityFramework.CodeFirstStoreFunctions NuGet package. You can add it either from the Package Manager Console by executing

Install-Package EntityFramework.CodeFirstStoreFunctions -Pre 

command or using the UI – right click the References in the solution explorer and select “Manage NuGet Packages”, then when the dialog opens make sure that the “Include Prerelease” option in the dropdown at the top of the dialog is selected and use “storefunctions” in the search box to find the package. Finally click the “Install” button to install the package.

Code First Store Functions NuGet


Installing EntityFramework.CodeFirstStoreFunctions from UI

After the package has been installed copy and paste the code snippet from below to your project. This code demonstrates how to enable store functions in Code First.

public class Customer
{
    public int Id { get; set; }

    public string Name { get; set; }

    public string ZipCode { get; set; }
}

public class MyContext : DbContext
{
    static MyContext()
    {
        Database.SetInitializer(new MyContextInitializer());
    }

    public DbSet<Customer> Customers { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        modelBuilder.Conventions.Add(new FunctionsConvention<MyContext>("dbo"));
    }

    [DbFunction("MyContext", "CustomersByZipCode")]
    public IQueryable<Customer> CustomersByZipCode(string zipCode)
    {
        var zipCodeParameter = zipCode != null ?
            new ObjectParameter("ZipCode", zipCode) :
            new ObjectParameter("ZipCode", typeof(string));

        return ((IObjectContextAdapter)this).ObjectContext
            .CreateQuery<Customer>(
                string.Format("[{0}].{1}", GetType().Name, 
                    "[CustomersByZipCode](@ZipCode)"), zipCodeParameter);
    }

    public ObjectResult<Customer> GetCustomersByName(string name)
    {
        var nameParameter = name != null ?
            new ObjectParameter("Name", name) :
            new ObjectParameter("Name", typeof(string));

        return ((IObjectContextAdapter)this).ObjectContext.
            ExecuteFunction<Customer>("GetCustomersByName", nameParameter);
    }
}

public class MyContextInitializer : DropCreateDatabaseAlways<MyContext>
{
    public override void InitializeDatabase(MyContext context)
    {
        base.InitializeDatabase(context);

        context.Database.ExecuteSqlCommand(
            "CREATE PROCEDURE [dbo].[GetCustomersByName] @Name nvarchar(max) AS " +
            "SELECT [Id], [Name], [ZipCode] " +
            "FROM [dbo].[Customers] " +
            "WHERE [Name] LIKE (@Name)");

        context.Database.ExecuteSqlCommand(
            "CREATE FUNCTION [dbo].[CustomersByZipCode](@ZipCode nchar(5)) " +
            "RETURNS TABLE " +
            "RETURN " +
            "SELECT [Id], [Name], [ZipCode] " +
            "FROM [dbo].[Customers] " + 
            "WHERE [ZipCode] = @ZipCode");
    }

    protected override void Seed(MyContext context)
    {
        context.Customers.Add(new Customer {Name = "John", ZipCode = "98052"});
        context.Customers.Add(new Customer { Name = "Natasha", ZipCode = "98210" });
        context.Customers.Add(new Customer { Name = "Lin", ZipCode = "98052" });
        context.Customers.Add(new Customer { Name = "Josh", ZipCode = "90210" });
        context.Customers.Add(new Customer { Name = "Maria", ZipCode = "98074" });
        context.SaveChanges();
    }
}

class Program
{
    static void Main()
    {
        using (var ctx = new MyContext())
        {
            const string zipCode = "98052";
            var q = ctx.CustomersByZipCode(zipCode)
                .Where(c => c.Name.Length > 3);
            //Console.WriteLine(((ObjectQuery)q).ToTraceString());
            Console.WriteLine("TVF: CustomersByZipCode('{0}')", zipCode);
            foreach (var customer in q)
            {
                Console.WriteLine("Id: {0}, Name: {1}, ZipCode: {2}", 
                    customer.Id, customer.Name, customer.ZipCode);
            }

            const string name = "Jo%";
            Console.WriteLine("\nStored procedure: GetCustomersByName '{0}'", name);
            foreach (var customer in ctx.GetCustomersByName(name))
            {
                Console.WriteLine("Id: {0}, Name: {1}, ZipCode: {2}", 
                    customer.Id, customer.Name, customer.ZipCode);   
            }
        }
    }
}

In the code above I use a custom initializer to initialize the database and create a table-valued function and a stored procedure (in a real application you would probably use Code First Migrations for this). The initializer also populates the database with some data in the Seed method. The MyContext class is a class derived from the DbContext class and contains two methods that are mapped to store functions created in the initializer. The context class contains also the OnModelCreating method where we register the convention which will do all the hard work related to setting up our store functions. The Main method contains code that invokes store functions created when initializing the database. First, we use the TVF. Note, that we compose the query on the function which means that the whole query will be translated to SQL and executed on the database side. If you would like to see this you can uncomment the line which prints the SQL query in the above snippet and you will see the exact query that will be sent to the database:

SELECT
    [Extent1].[Id] AS [Id],
    [Extent1].[Name] AS [Name],
    [Extent1].[ZipCode] AS [ZipCode]
    FROM [dbo].[CustomersByZipCode](@ZipCode) AS [Extent1]
    WHERE ( CAST(LEN([Extent1].[Name]) AS int)) > 3

(Back to the code) Next we execute the query and display results. Once we are done with the TVF we invoke the stored procedure. This is just an invocation because you cannot build queries on top of results returned by stored procedures. If you need any query-like (or other) logic it must be inside the stored procedure itself and otherwise you end up having a Linq query that is being run against materialized results. That’s pretty much the whole app. Just in case I am pasting the output the app produces below:

TVF: CustomersByZipCode('98052')
Id: 1, Name: John, ZipCode: 98052

Stored procedure: GetCustomersByName 'Jo%'
Id: 1, Name: John, ZipCode: 98052
Id: 4, Name: Josh, ZipCode: 90210
Press any key to continue . . .

Note that in both examples the return types are based on entity types. As I hinted above you can also use complex and scalar types for your results. Take a look at the End-to-End tests in the project itself – all scenarios are tested there.

That’s about what’s in alpha, so you may ask:

what’s next?

If you look at the code there are a few TODOs in the code. One of the most important is the support for nullable parameters. I am also thinking of removing the limitation where the method name in your DbContext derived class must ultimately match the name of the TVF in the database. If the workitem 2192 is resolved for the next version of EF I will be able to add support for non-default mapping. In addition I think it is very close from workitem 2192 to supporting stored procedures returning multiple resultsets. Not sure how useful it would be but it would be cool to see support for this feature which currently is kind of a dead feature because it is supported neither by CodeFirst nor by EF tooling.

Anything else?
The project is open source and is hosted on codeplex. You can get the sources from here. Try it and let me know what you think.

Advertisements

Second Level Cache for EF 6.1

See what’s new in Beta here.

Currently Entity Framework does not natively support second level caching. For pre-EF6 versions you could use EF Caching Provider Wrapper but due to changes to the EF provider model in EF6 it does not work with newest versions of EF. In theory it would be possible to recompile the old caching provider against EF6 but, unfortunately, this would not be sufficient. A number of new features breaking some assumptions made in the old caching provider (e.g. support for passing an open connection when instantiating a new context) have been introduced in EF6 resulting in the old caching provider not working correctly in these scenarios. Also dependency injection and Code-based Configuration introduced in EF6 simplify registering wrapping providers which was rather cumbersome (especially for Code First) in the previous versions. To fill the gap I created the Second Level Cache for EF 6.1 project. It’s a combination of a wrapping provider and a transaction handler which work together on caching query results and keeping the cache in a consistent state. In a typical scenario query results are cached by the wrapping caching provider and are invalidated by the transaction handler if an entity set on which cached query results depend was modified (note that data modification in EF always happens in a transaction hence the need of using transaction handler).
Using the cache is easy but requires a couple steps. First you need to install the EntityFramework.Cache NuGet package. There are two ways to do this from Visual Studio. One way to do this is to use the UI – right click the References node in the Solution Explorer and select the Manage NuGet Packages option. This will open a dialog you use to install NuGet packages. Since the project is currently in the alpha stage you need to select “Include Prelease” in the drop down at the top. Then enter “EntityFramework.Cache” in the search window and, once the package appears, click the “Install” button.

AAAA


Installing EntityFramework.Cache from UI


You can also install the package from the Package Manager Console. Open the Package Manager Console (Tools → NuGet Package Manager → Package Manager Console) and execute the following command:
Install-Package EntityFramework.Cache –Pre
(-Pre allows installing pre-release packages).
Note that the package depends on Entity Framework 6.1. If you don’t have Entity Framework 6.1 in your project it will be automatically installed when you install the EntityFramework.Cache package.
Once the package is installed you need to tell EF to use caching by configuring the caching provider and the transaction handler. You do this by creating a configuration class derived from the DbConfiguration class. In the constructor of your DbConfiguration derived class you need to set the transaction interceptor and the Loaded event handler which will be responsible for replacing the provider. Here is an example of how to setup caching which uses the built-in in-memory cache implementation and the default caching policy.

public class Configuration : DbConfiguration
{
  public Configuration()
  {
    var transactionHandler = new CacheTransactionHandler(new InMmemoryCache());

    AddInterceptor(transactionHandler);

    Loaded +=
      (sender, args) => args.ReplaceService<DbProviderServices>(
        (s, _) => new CachingProviderServices(s, transactionHandler, 
          new DefaultCachingPolicy()));
  }
}

The default caching policy used above is part of the project and allows caching all query results regardless of their size or entity sets used to obtain the results. Both, sliding and absolute, expiration times in the default caching policy are set to maximum values therefore items will be cached until an entity set used to obtain the results depended on is modified. If the default caching policy is not suitable for your needs you can create a custom caching policy in which you can limit what will be cached. To create a custom policy you just need to derive a class from the CachingPolicy class, implement the abstract methods, and pass the policy to the CachingProviderServices during registration. When implementing a custom caching policy there is one thing to be aware of – the expired entries are removed from the cache lazily. It means that an entry will be removed by the caching provider only when the caching provider tries to read the entry and finds it is expired. This is not extremely helpful (especially because since the item is expired the provider will query the database to get fresh results which will be then put to the cache – so effectively the expired entry will be replaced with a new entry) but lets the user decide what the best strategy of cleaning the cache in their case is. For instance, in the InMemoryCache implementation included in the project I created a Purge method. This method could be periodically called to remove stale cache entries.
The project also includes a simple implementation of a cache which caches query results in memory. This is just a sample implementation and if you would like to use a different caching mechanism you are free to do so – you just need to implement the ICache interface. This interface is a slightly modified version of the interface that shipped with the original EF Caching Provider Wrapper which should make moving existing apps using the old caching solution to EF6 easier.

As the old saying goes “A program is worth a 1000 words“, so let’s take a look at the second level cache in action. Here is a complete sample app which is using the cache:

public class Airline
{
  [Key]
  public string Code { get; set; }

  public string Name { get; set; }

  public virtual ICollection<Aircraft> Aircraft { get; set; }
}

public class Aircraft
{
  public int Id { get; set; }

  public string EquipmentCode { get; set; }

  public virtual Airline Airline { get; set; }
}

public class AirTravelContext : DbContext
{
  static AirTravelContext()
  {
    Database.SetInitializer(new DropCreateDatabaseIfModelChanges<AirTravelContext>());  
  }

  public DbSet<Airline> Airlines { get; set; }

  public DbSet<Aircraft> Aircraft { get; set; }
}

public class Configuration : DbConfiguration
{
  public Configuration()
  {
    var transactionHandler = new CacheTransactionHandler(Program.Cache);

    AddInterceptor(transactionHandler);

    Loaded +=
      (sender, args) => args.ReplaceService<DbProviderServices>(
        (s, _) => new CachingProviderServices(s, transactionHandler));
  }
}
  
class Program
{
  internal static readonly InMemoryCache Cache = new InMemoryCache();

  private static void Seed()
  {
    using (var ctx = new AirTravelContext())
    {
      ctx.Airlines.Add(
        new Airline
        {
          Code = "UA",
          Name = "United Airlines",
          Aircraft = new List<Aircraft>
          {
            new Aircraft {EquipmentCode = "788"},
            new Aircraft {EquipmentCode = "763"}
          }
        });

      ctx.Airlines.Add(
        new Airline
        {
          Code = "FR",
          Name = "Ryan Air",
          Aircraft = new List<Aircraft>
          {
            new Aircraft {EquipmentCode = "738"},
          }
        });

      ctx.SaveChanges();
    }
  }

  private static void RemoveData()
  {
    using (var ctx = new AirTravelContext())
    {
      ctx.Database.ExecuteSqlCommand("DELETE FROM Aircraft");
      ctx.Database.ExecuteSqlCommand("DELETE FROM Airlines");
    }
  }

  private static void PrintAirlinesAndAircraft()
  {
    using (var ctx = new AirTravelContext())
    {
      foreach (var airline in ctx.Airlines.Include(a => a.Aircraft))
      {
        Console.WriteLine("{0}: {1}", airline.Code, 
          string.Join(", ", airline.Aircraft.Select(a => a.EquipmentCode)));
      }
    }
  }

  private static void PrintAirlineCount()
  {
    using (var ctx = new AirTravelContext())
    {
      Console.WriteLine("Airline Count: {0}", ctx.Airlines.Count());
    }
  }

  static void Main(string[] args)
  {
    // populate and print data
    Console.WriteLine("Entries in cache: {0}", Cache.Count);
    RemoveData();
    Seed();
    PrintAirlinesAndAircraft();

    Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
    // remove data bypassing cache
    RemoveData();
    // not cached - goes to the database and counts airlines
    PrintAirlineCount();
    // prints results from cache
    PrintAirlinesAndAircraft();
    Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
    // repopulate data - invalidates cache
    Seed();
    Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
    // print data
    PrintAirlineCount();
    PrintAirlinesAndAircraft();
    Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
  }
}

The app is pretty simple – a couple of entities, a context class, code base configuration (should look familiar), a few methods and the Main() method which drives the execution. One method that is worth mentioning is the RemoveData() method. It removes the data from the tables using the SqlExecuteMethod() therefore bypassing the entire Entity Framework update pipeline including the caching provider. This is to show that the cache really works but is at the same type a kind of warning – if you bypass EF you will need to make sure the cache is in a consistent state or you can get incorrect results. Running the sample app results in the following output:

Entries in cache: 0
FR: 738
UA: 788, 763
Entries in cache: 3

Removing data directly from the database
Airline Count: 0
FR: 738
UA: 788, 763

Entries in cache: 4

Entries in cache: 2
Airline Count: 2
FR: 738
UA: 788, 763

Entries in cache: 4
Press any key to continue . . .

Let’s analyze what’s happening here. On line 1 we just report that the cache is empty (no entries in the cache). Seems correct – no queries have been sent to the database so far. Then we remove any stale data from the database, seed the database, and print the contents of the database (lines 2 and 3). Printing the content of the database requires querying the database which should result in some entries in the cache. Indeed on line 4 three entries are reported to be in the cache. Why three if we sent just one query? If you peek at the cache with the debugger you will see that two of these entries are for the HistoryContext so three seems correct. Now (line 6) we delete all the data in the database so when we query the database for the airline count (line 7) the result is 0. However, even though we don’t have any data in the database, we can still print data on lines 8 and 9. This data comes from the cache – note that when we deleted data we used ExecuteSqlCommand() method which bypassed the EF update pipeline and, as a result, no cache entries were invalidated. On line 11 we again report the number of items in the cache. This time the cache contains four entries – two entries for the HistoryContext related queries, one entry for the data that was initially cached and one entry for the result of the query where we asked for the number of airlines in the database. Now we add the data to the database again and again print the number of items in the cache on line 13. This time the number of entries is two. This is because inserting the data to the database invalidated cached results for queries that used either Airlines or Aircraft entity sets and as a result only the results related to the HistoryContext remained in the cache. We again query the database for the number of airlines (line 14) and print all the data from the database. The results for both queries are added to the cache and we end up having four entries in the cache again (line 18).
That’s pretty much it. Try it out and let me what you think (or report bugs). You can also look at and play with the code – it is available on the project website on codeplex.
Note that this version is an alpha version and there is still some work remainig before it can be called done – most notably support for async methods, adding some missing overrides for the ADO.NET wrapping classes. I am also considering adding support for TransactionScope.

What changed in the EF Tooling in Visual Studio 2013 (and Visual Studio 2012 Out Of Band)

The recently shipped version of Visual Studio 2013 contains a new version of EF Tooling (a standalone version for Visual Studio 2012 is also available for download). The main goal of this release was to teach the EF designer how to talk to both EF5 and EF6. It was quite an undertaking given how EF6 differs from EF5 (binary incompatible, no correspondence between provider models and metadata workspace, EF6 runtime assemblies no longer in GAC just to name a few differences) and required changes to the existing functionality. Some of these changes may be confusing to people who used previous versions of EF tooling. The goal of this post is to take a look what has changed, why, and how it works now.
Before EF6 things were reasonably straightforward – from the EF Tooling standpoint there could be only one version of the EF runtime installed on the box – EF1, EF4 or EF5. In addition it was easy to determine the version of EF just by looking at the installed version of the .NET Framework – .NET Framework 3.5 meant EF1, .NET Framework 4 meant EF4 and .NET Framework 4.5 meant EF5. Note that the designer did not need to distinguish between EF4.x versions (treated as EF4) or EF5 for .NET Framework 4 (also treated as EF4) since they were built on top of the core EF runtime version residing in GAC. The installed (GAC’ed) version of the EF runtime automatically implied the latest supported version of Csdl, Ssdl, Msl and Edmx (‘schemas’ as we internally call them) since each version of the core EF runtime introduced a new version of schemas. But then happened EF6 – no longer in GAC, works on .NET Framework 4 and .NET Framework 4.5 (also meaning you could have v3 schemas on .NET Framework 4), binary incompatible with earlier versions of EF, some types moved to new/different namespaces, etc. The only way for tooling to be able to handle all of this was to break a number of assumptions made in the designer and, in some cases, change how the designer works. Below is a list of things that changed the most with a short explanation why they changed and what to expect:

  • For Model First the code is no longer generated when adding an empty model to the project. This is probably the most confusing change (funnily it was internally reported to me as a bug multiple times even before we shipped the new EF Tooling the first time in Visual Studio 2013 RC). The reason for this change is simple – since some types in EF6 were moved to different namespaces the code we used to generate for EF5 no longer works with EF6 (and vice versa). In some cases (e.g. you don’t have a reference to any EF runtime in your project) it is just impossible to tell whether the user is going it need EF5 or EF6 code. The most interesting part is that the code for we used to add when creating an empty model was is in reality not that useful before the database was created – yes, you could write code against the generated context/entities but you would not be really able to run it without the database. Therefore, we changed the workflow a bit and now the code generation templates are added when you create the database from model instead of when you add an empty model to your project because the wizard will make you select the version of EF you would like to use in your project, if there already isn’t one.
  • Code Generation Strategy disabled for EF6 models. Again this is partially related to changes to types and namespaces. Code generated for EF5 would not work with EF6. In addition the code we generate for ObjectContext based context and EntityObject based entities for EF5 (and earlier) applications is generated using System.Data.Entity.Design.dll. System.Data.Entity.Design.dll is part of the .NET Framework and does not fit into the EF6 shipping model. Therefore instead of updating System.Data.Entity.Design.dll we decided to support only T4 template based code generation for EF6. EF Tooling ships with T4 templates for generating DbContext based context and POCO entities. If, for some reason, you absolutely need ObjectContext based context the EF6 templates were posted on VS Gallery. For EF5 (or earlier) applications the Code Generation Strategy drop down is not blocked and you can choose between T4 and LegacyObjectContext (yes, we changed the option names as well because they were a bit ridiculous in VS2012 where ‘None’ basically meant T4 and ‘Default’ meant ObjectContext but the ‘None’ option was default).
  • Impossible to select the version of Entity Framework. To determine the applicable version of EF the designer looks for references to EntityFramework.dll and System.Data.Entity.dll in the project (note that EntityFramework.dll wins over System.Data.Entity.dll when determining the version). If it cannot find any, the user has to select the version in the wizard. Otherwise the latest version of EF found in the project is being used and the selection will not be possible (radio buttons will be disabled or the wizard page will not be shown at all).
  • Using third party providers. In the simple days of EF5 (and earlier) EF providers were registered globally and you could get one basically anytime and anywhere. Those days are long gone. In EF6 EF providers don’t have a global registration point. Rather, they are registered in the config file or using code based configuration. When you select a connection in the wizard the designer tries to find a reference to a corresponding provider in your project. If it cannot find one you won’t be able to use EF6 – instead you will be asked to close the wizard, install the provider and restart the wizard (note that this does not apply to the provider for Sql Server – EF Tooling contains EF6 provider for Sql Server which in the majority of cases make things easier for the user).
  • Retargeting. Depending on the referenced version of Entity Framework in the project changing the target .NET Framework version for the project may but may not change the edmx file. If the project references System.Data.Entity.dll and does not reference EF6 retargeting will result in upgrading/downgrading the version of the edmx file to match the latest supported version of the schema for the given .NET Framework and, as a result, EF runtime version. If the project contains a reference to EF6 toggling the target between .NET Framework 4 and .NET Framework 4.5 will not result in changing the edmx version since EF6 is supported on both versions and understands v3 schemas. Targeting .NET Framework 3.5 should always downgrade to v1 since EF1 is the only version available on .NET Framework 3.5. One caveat to retargeting is that you should always update your NuGet packages after retargeting. This is because EF (and potentially other packages) has a different assembly for .NET Framework 4 and for .NET Framework 4.5 (one of the reasons is that Async is not natively supported on .NET Frramework 4) and even though they are both shipped in the same NuGet package, NuGet does not replace project references after changing the target .NET Framework version. This leads either to missing functionality (e.g. Async not available on .NET Framework 4.5 (when using EF6 for .NET Framework 4 on .NET Framework 4.5)) or weird build errors saying you don’t have reference to EntityFramework.dll (only running MsBuild /v:diag will tell you that the reference you have was ignored since the referenced EntityFramework.dll was built for a later version of .NET Framework than the one the project is currently targeting (happens when using EF6 for .NET Framework 4.5 on .NET Framework 4)).
  • Using EF6 and EF5/EF4 in one project. Initially we did not want to support this scenario at all. However it is not only pretty easy to end up in such a situation but supporting this would also be helpful for people wanting to gradually move legacy, bigger projects from EF5 to EF6, so finally we decided to do what we can to support mixed EF versions in one project. The most important thing the designer has to do to handle this scenario correctly is to ensure that the right provider is being used for a given edmx file. Also, the version of the edmx file needs to be in sync with the referenced version of the EF runtime. Seems easy at first but there are some cases where things may behave a bit weird. A canonical example is when you have a project targeting .NET Framework 4 and you don’t have a reference to any Entity Framework runtime. When you create a new model it will create a v2 edmx since this was the latest supported schema version in .NET Framework 4. Now you create a database from your model and select EF6. Since EF6 supports v3 schemas the version of your edmx file will be changed to v3. Because the number of dimensions for versioning scenarios ended up being bigger than we originally expected and had a number of exceptions on top of that it was initially hard to tell what the outcome for each combination should be. To address this I created a set of rules which were much easier to understand and to follow when working on versioning scenarios. You can find these rules here.
  • In-the-box packages are not the latest available. EF Tooling contains EF6 NuGet packages that will be added to the project when adding a new model if needed. However, since EF Tooling ships with Visual Studio which has a different cadence that EF the included packages may not be the latest ones. In fact in case of EF6 we fixed a few (mostly performance related) bugs after Visual Studio 2013 was locked down. As a result version 6.0.1 of EF was shipped on the same day as Visual Studio 2013 but the version of EF that ships with Visual Studio 2013 is 6.0.0 and does not contain fixes included in 6.0.1. Moreover 6.0.2 is on its way and again – it will be a runtime only release. Updating EF package is as easy as running the following command from Package Manager Console (Tools → Library Package Manager → Package Manager Console)
    Update-Package EntityFramework

T4 Templates for Generating Views for EF4/EF5 Database/Model First apps

After we received a comment on the EF team blog asking why the templates for generating views for Edmx based apps we posted years ago were not updated for EF5 Rowan asked me if I it would be possible to update the template so that it works for EF5. I actually remember looking at this template before I shipped my templates generating views for Code First apps and one of the first things I noticed was that the way it handled versions was robust but not very flexible. Therefore, when we shipped EF5, the templates could not be used with apps using v3 of the EF artifacts (edmx). I thought a bit more about this when I was working on the templates for CodeFirst and concluded that robustness is not really very important here. If someone wants views they will have to provide a valid model (be it Code First model or Edmx) or the APIs used for generating views will throw an exception. Since the exception will be displayed in the error pane (and should be understandable for any person who is trying to use pre-generated views) there was no need to sacrifice flexibility for robustness. Also, the expierence of using the template was not very good. It was posted as a .zip file attached to a blog post. Not only wasn’t it discoverable but you would also have to take a number of extra steps to be able to use it. Visual Studio Gallery makes it much easier to find and use extensions like this, so I thought it was not a bad idea to revamp the template and ship it on the VS Gallery (especially given that I already have all the infrastructure needed to create a shippable .VSIX files containing a T4 templates). The only thing I was missing was the template itself. Luckily(?) I was investigating a bug which I thought could be worked around by using pre-generated views. After I finished up coding the template it turned out that this is not the case (i.e. you cannot use pre-generated views if you have QueryViews in your mapping) but at least I had a T4 template for generating views for Database/Model First apps. Now I only had to marry the infrastructure for building .vsix with the template and call it done (I went a bit further and even created a VB.Net version of the template). Since yesterday the templates are available on Visual Studio Gallery – the C# version can be found here and the VB.Net version can be found here. Using the templates is simple. In Visual Studio right-click on your project and select Add→New Item (Ctrl+Shift+A). In the “Add New Item” window select “Online” on the left. You may want to filter by EF5 or edmx. Select the “EF4/EF5 Model/Db First View Gen .tt for {C#|VB.Net}”. Change the name of the .tt file so that it starts with the name of your edmx file (i.e. if your edmx file is called “MyModel.edmx” name the template “MyModel.Views.tt”) and press the “Add” button:

Add New Item - Edmx Views

Add New Item – Edmx Views


and that’s pretty much it – the template and a class containing views should now be added to your project:
Project with views

Project with views


If you want to enforce re-generating views (e.g. after the edmx file changed) just right click the .tt file and select “Run Custom Tool”.
If you need to uninstall the template go to Tools→Extensions and Updates… select the template from the list and click the “Uninstall” button.

EF Designer now supports EF6

Some time ago I showed how to hack the EF Designer shipped with Visual Studio 2012 to work with EF6. Those hacks should be no longer needed – last week we shipped Entity Framework 6 Beta1 which for the first time contains not only EF6 runtime but also contains tooling. The new EF Designer works with Visual Studio 2012 and replaces the designer that was originally shipped with Visual Studio 2012. Almost all the work we did was about adding support for EF6 – being able to handle new metadata types, using the new provider model, creating EF6 specific T4 templates etc. This work has not been completed yet and some scenarios like support for EF6 on .NET Framework 4 or using non-SqlServer (or SqlServerCE) providers when using EF6 are still not enabled but if you want to use model first/database first workflows with EF6 the Beta version is currently your best option. Note that even though this release is mostly about adding support for EF6 we are not abandoning EF5. For EF5 the EF Designer Beta 1 version supports all the scenarios that were supported by the previous versions. You choose the target version of Entity Framework in the wizard when adding a new model to your project (note that in some cases the choice is limited – e.g. if your project already has a reference to System.Data.Entity.dll you won’t be able to select EF6).

It’s not a June CTP kind of thing
EF6 runtime is no longer shipped as part of the .NET Framework and we did not have to do hacks we had to do when shipping tooling in June 2011 CTP. It means that it is easy to install the EF Designer Beta 1 but it is also easy to uninstall it and go back to the original designer shipped with Visual Studio 2012 – just go to Programs and Features, uninstall the new designer and repair Visual Studio and things should work as they did before.

How to get it?
Just go to the download center and install the msi.

We want your feedback
Since the new EF Designer is easy to install and uninstall and it supports all the scenarios supported by the old one I would like to encourage everyone to try it out and let us know about your experience (and issues). This is only a Beta version and we already know it has some rough edges but it may also have bugs we don’t not know about yet. Try it out and if you see something start a new discussion or create a new work item.

EF6 CodeFirst View Generation T4 Template for C# updated for EF6 Beta1

Entity Framework 6 Beta 1 has shipped. The good news is that among (many) others it contains a few changes to view generation. There is now a public API on the StorageMappingItemCollection type that can return views for a given StorageMappingItemCollection instance. The bad news is that some of the changes we introduced in this area in the Beta version are breaking. This means that views created with the EF6 CodeFirst View Generation T4 Template for C# I created for the EF6 Alpha will no longer work with the Beta version and need to be recreated. The main changes (apart from adding the public API) to view generation in Beta are:

Because of the above breaking changes I updated the EF6 CodeFirst View Generation T4 Template for C# to work with EF6 Beta 1. Note that the new version uses the new public API for view generation and therefore it requires that your project has a reference to the EntityFramework.dll shipped with EF6 Beta 1. Also, you cannot generate views with the new version and re-use them in an EF6 Alpha project.
Installation steps have not changed comparing to the previous version and you can found them in this post. You shall not have to uninstall the previous version – the new vsix should replace the old one.

Entity Framework 6 on Mono

I had been wanting to try Entity Framework 6 with Mono for a long time and finally after a check-in I made on Friday I was able to spare some time to do this. I decided to go with MySQL and dotConnect for MySQL from Devart. After skimming the tutorial on using Devart’s providers with EF6 I opened Visual Studio 2012, created a new project, installed the EF6 alpha3 package from NuGet and thought I was almost done. The only problem was that the dead simple (one entity) EF6 based app I created did not work. I did not expect the exception I got and quickly realized why the tutorial I was following started with “Entity Framework 6 Alpha 2 support is implemented…”. Things got a little better when I went back from EF6 Alpha 3 to EF6 Alpha 2 but still did not work. I found that I was missing one of the dlls and then that I used wrong revision number in the version of the provider assemblies (interestingly the version of the assembly containing the provider factory was 7.6.217.0 while the version of the assembly containing provider services was 7.6.217.6 – I expected both assemblies to have the same version). The last time I used MySQL was… some time ago (OK, it was in the 20th century) and it was the first time I used it with Code First (kinda obvious). Even after setting all the versions correctly the app still did not want to run. I was able to get the edmx file with the EdmxWriter (which is a good sign meaning that the provider is configured correctly and working) but then the app would die because it could not create the database. After a quick search I found that I need to create an empty database using MySQL command prompt and then EF should be able to add tables to this database. Indeed – after I created the empty database that app did not throw and I was able to add entities to the database and then read them. With this I was ready to move to the Mono world. So, I installed the latest version of Mono and started playing a little bit with it. mcs.exe (the compiler) worked just fine. However since I could build my app with Visual Studio and ran it using mono.exe I decided not to use the mcs.exe. Unfortunately the app would not quite work. It would start and then throw System.Configuration.ConfigurationErrorsException: Failed to find or load the registered .Net Framework Data Provider .... Which basically means that the ADO.NET/EF provider I used was not registered. I think the reason was that I installed Mono after I installed the provider and therefore the entry in the machine.config was missing. I added the following entry:

  <system.data>
    <DbProviderFactories>
      <remove invariant="Devart.Data.MySql" />
      <add name="dotConnect for MySQL" invariant="Devart.Data.MySql" 
           description="Devart dotConnect for MySQL" 
           type="Devart.Data.MySql.MySqlProviderFactory, Devart.Data.MySql, Version=7.6.217.0, Culture=neutral, PublicKeyToken=09af7300eec23701" />
    </DbProviderFactories>
  </system.data>

to my app.config file and I was able to get a little bit further. (Just in case – you can just add the “add” line from the above to the Mono’s machine.config in the provider section and it should work as well). Now when I ran my app I got this:

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1&amp; schemaCollection) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems (IEnumerable`1 xmlReaders, IEnumerable`1sourceFilePaths) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init (IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest&amp; providerManifest, System.Data.Common.DbProviderFactory&amp; providerFactory, System.String&amp; providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2&amp; cachedCTypeFunction) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor (IEnumerable`1 xmlReaders) [0x00000] in :0
  at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection (System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo&amp; providerInfo) [0x00000] in :0
  at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff (System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.ModelMatches (System.Xml.Linq.XDocument model) [0x00000] in :0
  at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel (System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Database.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.CreateDatabaseIfNotExists`1[System.Data.Entity.DbContext].InitializeDatabase (System.Data.Entity.DbContext context) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[EF6MySqlTest.SimpleContext].b__b () [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformInitializationAction (System.Action action) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization () [0x00000] in :0
  at System.Data.Entity.Internal.LazyInternalContext.b__4 (System.Data.Entity.Internal.InternalContext c) [0x00000] in :0
  at System.Data.Entity.Internal.RetryAction`1[System.Data.Entity.Internal.InternalContext].PerformAction (System.Data.Entity.Internal.InternalContext input) [0x00000] in :0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema () [0x00000] in :0
  at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1&amp; schemaCollection) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems (IEnumerable`1 xmlReaders, IEnumerable`1sourceFilePaths) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor (IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init (IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest&amp; providerManifest, System.Data.Common.DbProviderFactory&amp; providerFactory, System.String&amp; providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2&amp; cachedCTypeFunction) [0x00000] in :0
  at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor (IEnumerable`1 xmlReaders) [0x00000] in :0
  at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection (System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo&amp; providerInfo) [0x00000] in :0
  at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff (System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.ModelMatches (System.Xml.Linq.XDocument model) [0x00000] in :0
  at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel (System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.Database.CompatibleWithModel (Boolean throwIfNoMetadata) [0x00000] in :0
  at System.Data.Entity.CreateDatabaseIfNotExists`1[System.Data.Entity.DbContext].InitializeDatabase (System.Data.Entity.DbContext context) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[EF6MySqlTest.SimpleContext].b__b () [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformInitializationAction (System.Action action) [0x00000] in :0
  at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization () [0x00000] in :0
  at System.Data.Entity.Internal.LazyInternalContext.b__4 (System.Data.Entity.Internal.InternalContext c) [0x00000] in :0
  at System.Data.Entity.Internal.RetryAction`1[System.Data.Entity.Internal.InternalContext].PerformAction (System.Data.Entity.Internal.InternalContext input) [0x00000] in :0

WTF? The same app worked just fine on .NET Framework! Obviously there was a bug somewhere but where? Mono? EF? .NET Framework? I followed the advice given by John Robins from Wintellect (http://www.wintellect.com/cs/blogs/jrobbins/default.aspx) during the “Debugging Windows Applications” training I took (too) many years ago and prayed that the bug was in the code base I own (because “in this case you can fix it”). I looked at some tutorials on debugging Mono apps but quickly felt it was too much to learn. Then I installed WinDbg but without symbols I could see only this:

ChildEBP RetAddr  Args to Child              
WARNING: Stack unwind information not available. Following frames may be wrong.
0022f5ec 76ccbbf7 00000000 77e8f3b0 ffffffff ntdll!KiFastSystemCallRet
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\msvcrt.dll - 
0022f600 758a36dc 00000001 0022f644 758a3372 kernel32!ExitProcess+0x15
0022f60c 758a3372 00000001 172bd79c 016b6698 msvcrt!exit+0x32
0022f644 758a36bb 00000001 00000000 00000000 msvcrt!dup+0x2a9
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Mono-3.0.9\bin\mono-2.0.dll - 
0022f658 65a62b3a 00000001 65bba06c 0546e928 msvcrt!exit+0x11
0022f678 65a61fe6 05af88c0 016b6698 00000000 mono_2_0!mono_invoke_unhandled_exception_hook+0x6a
0022f7e8 65aa7aba 00000000 00000000 00000000 mono_2_0!mono_print_thread_dump_from_ctx+0xaa6
0022f898 01550e8f 0022f8b0 05af88c0 03cd75ac mono_2_0!mono_x86_throw_exception+0xca
0022f968 03cd6a64 03d6f230 01c53cc0 03ee0d58 0x1550e8f
0022f9a8 03cd6968 01c53cc0 03ee0d90 0022f9bc 0x3cd6a64
0022f9c8 01bc19f5 01c53cc0 02b8e090 0022f9dc 0x3cd6968
0022f9e8 01bc1984 01c53cc0 02b8e090 0022f9fc 0x1bc19f5
0022fa08 01bc1935 01c53cc0 00538df0 0022fa1c 0x1bc1984
0022fa28 01bc18c5 01c593c0 02b8e090 0022fa3c 0x1bc1935
0022fa48 01bc17be 01c593c0 016bd7a8 0022fa5c 0x1bc18c5
0022fa68 01bc1740 01c593c0 02bc38f8 00000004 0x1bc17be
0022faa8 01bc1654 01c593c0 02b8e060 0022fabc 0x1bc1740
0022fba8 65b5340a 002c73d8 00000001 0022fbf8 0x1bc1654
0022fbb8 659cc21c 00000000 0022fc38 00000000 mono_2_0!mono_runtime_class_init+0x1a
0022fbf8 65b52e42 002c81b0 00000000 0022fc38 mono_2_0!mono_jit_compile_method+0x10c
0022fc18 65b55682 002c81b0 00000000 0022fc38 mono_2_0!mono_runtime_invoke+0x42
0022fc48 65a386b2 002c81b0 00539e00 00000000 mono_2_0!mono_runtime_exec_main+0xd2
*** ERROR: Module load completed but symbols could not be loaded for image00400000
0022fec8 004014bd 00000002 002c1710 00000000 mono_2_0!mono_main+0x1562
0022ff28 004010bb 00000002 002c16e8 002c1bb8 image00400000+0x14bd
0022ff68 004012a8 00000001 00000000 00000000 image00400000+0x10bb
0022ff88 76cbed6c 7ffdd000 0022ffd4 7701377b image00400000+0x12a8
0022ff94 7701377b 7ffdd000 7e6b10f1 00000000 kernel32!BaseThreadInitThunk+0x12
0022ffd4 7701374e 00401290 7ffdd000 00000000 ntdll!RtlInitializeExceptionChain+0xef
0022ffec 00000000 00401290 7ffdd000 00000000 ntdll!RtlInitializeExceptionChain+0xc2

Maybe there are people on this planet who can make sense out of it but I am not there yet. Without symbols I could do not much. So, I ended up sync’ing my source to the changeset around the date we shipped (git reset --hard 9a52aa5f) and this time WinDbg showed this:

This should never be null, since if we were not able to resolve, we should have never reached to this point
---- Assert Long Message ----
   at System.Diagnostics.DefaultTraceListener.Fail(System.String message, System.String detailMessage)
   at System.Diagnostics.TraceListener.Fail(System.String message)
   at System.Diagnostics.DefaultTraceListener.Fail(System.String message)
   at System.Diagnostics.TraceImpl.Fail(System.String message)
   at System.Diagnostics.TraceImpl.Assert(Boolean condition, System.String message)
   at System.Diagnostics.Debug.Assert(Boolean condition, System.String message)
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.EntityKeyElement.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaEntityType.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.Validate()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.Schema.ValidateSchema()
   at System.Data.Entity.Core.EntityModel.SchemaObjectModel.SchemaManager.ParseAndValidate(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, SchemaDataModelOption dataModel, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.AttributeValueNotification providerManifestTokenNotification, System.Data.Entity.Core.EntityModel.SchemaObjectModel.ProviderManifestNeeded providerManifestNeeded, IList`1 ByRef schemaCollection)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader.LoadItems(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection+Loader..ctor(IEnumerable`1 xmlReaders, IEnumerable`1 sourceFilePaths, Boolean throwOnError)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection.Init(IEnumerable`1 xmlReaders, IEnumerable`1 filePaths, Boolean throwOnError, System.Data.Entity.Core.Common.DbProviderManifest ByRef providerManifest, System.Data.Common.DbProviderFactory ByRef providerFactory, System.String ByRef providerManifestToken, System.Data.Entity.Core.Common.Utils.Memoizer`2 ByRef cachedCTypeFunction)
   at System.Data.Entity.Core.Metadata.Edm.StoreItemCollection..ctor(IEnumerable`1 xmlReaders)
   at System.Data.Entity.Utilities.XDocumentExtensions.GetStoreItemCollection(System.Xml.Linq.XDocument model, System.Data.Entity.Infrastructure.DbProviderInfo ByRef providerInfo)
   at System.Data.Entity.Migrations.Infrastructure.EdmModelDiffer.Diff(System.Xml.Linq.XDocument sourceModel, System.Xml.Linq.XDocument targetModel, Nullable`1 includeSystemOperations)
   at System.Data.Entity.Internal.InternalContext.ModelMatches(System.Xml.Linq.XDocument model)
   at System.Data.Entity.Internal.ModelCompatibilityChecker.CompatibleWithModel(System.Data.Entity.Internal.InternalContext internalContext, System.Data.Entity.Internal.ModelHashCalculator modelHashCalculator, Boolean throwIfNoMetadata)
   at System.Data.Entity.Internal.InternalContext.CompatibleWithModel(Boolean throwIfNoMetadata)
   at System.Data.Entity.Database.CompatibleWithModel(Boolean throwIfNoMetadata)
   at System.Data.Entity.CreateDatabaseIfNotExists`1[[System.Data.Entity.DbContext, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].InitializeDatabase(System.Data.Entity.DbContext context)
   at System.Data.Entity.Internal.InternalContext+c__DisplayClassc`1[[EF6MySqlTest.SimpleContext, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].b__b()
   at System.Data.Entity.Internal.InternalContext.PerformInitializationAction(System.Action action)
   at System.Data.Entity.Internal.InternalContext.PerformDatabaseInitialization()
   at System.Data.Entity.Internal.LazyInternalContext.b__4(System.Data.Entity.Internal.InternalContext c)
   at System.Data.Entity.Internal.RetryAction`1[[System.Data.Entity.Internal.InternalContext, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].PerformAction(System.Data.Entity.Internal.InternalContext input)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeDatabaseAction(System.Action`1 action)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeDatabase()
   at System.Data.Entity.Internal.InternalContext.Initialize()
   at System.Data.Entity.Internal.InternalContext.GetEntitySetAndBaseTypeForType(System.Type entityType)
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Initialize()
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].get_InternalContext()
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].ActOnSet(System.Action action, EntityState newState, System.Object entity, System.String methodName)
   at System.Data.Entity.Internal.Linq.InternalSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Add(System.Object entity)
   at System.Data.Entity.DbSet`1[[EF6MySqlTest.Customer, EF6MySqlTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Add(EF6MySqlTest.Customer entity)
   at EF6MySqlTest.Program.Main(System.String[] args)

Apparently my prayer was heard and this line proves it: at System.Diagnostics.Debug.Assert(Boolean condition, System.String message). This is an Assert in the EF code, so looks like a bug in the EF. After looking at the EntityKeyElement.Validate method I could easily find the Assert:

Debug.Assert(
    property != null,
    "This should never be null, since if we were not able to resolve, we should have never reached to this point");

This was a good find (bug filed: https://entityframework.codeplex.com/workitem/1038) but now – how this could have happened? After poking around and using less than scientific methods to debug this I finally concluded that the error is in the invalid SSDL file stored in the __MigrationHistory table. Here is the most interesting fragment:

<EntityType Name=""HistoryRow"" p5:IsSystem=""true"" xmlns:p5=""http://schemas.microsoft.com/ado/2012/10/edm/migrations"">
  <Key p5:IsSystem=""true"">
    <PropertyRef Name=""MigrationId"" p5:IsSystem=""true"" />
    <PropertyRef Name=""ContextKey"" p5:IsSystem=""true"" />
  </Key>
  <Property Name=""MigrationId"" Type=""varchar"" MaxLength=""255"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""ContextKey"" Type=""varchar"" MaxLength=""512"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""Model"" Type=""longblob"" Nullable=""false"" p5:IsSystem=""true"" />
  <Property Name=""ProductVersion"" Type=""varchar"" MaxLength=""32"" Nullable=""false"" p5:IsSystem=""true"" />
</EntityType>

It does not look unusual but according to the schema for the SSDL the Key element must not have any attributes (be it in empty or non-empty namespace). Why this thing worked on the .NET Framework and not on Mono? Apparently the Xsd validation in Mono works a bit differently than on the .NET Framework. When processing PropertyRef elements the Xsd validator on .NET Framework reported they were valid whereas Mono claims they are not. I am not sure who is right in this case and, honestly, I don’t care too much. The major bug is in the EF – we should not produce invalid SSDL artifacts (bug filed: https://entityframework.codeplex.com/workitem/1038). This can be fixed either by not writing the {http://schemas.microsoft.com/ado/2012/10/edm/migrations}IsSystem attribute or by allowing attributes in non-empty and non-EF schema on the element. Since I already had a bootleg of entityframework.dll (and I believe the bug is actually in the schema for SSDL) I decided to fix the schema. In the System.Data.Resources.SSDLSchema_3.xsd file I changed the TEntityKeyElement to look like this (added the xs:anyAttribute line):

<xs:complexType name="TEntityKeyElement">
  <xs:sequence>
    <xs:element name="PropertyRef" type="edm:TPropertyRef" minOccurs="1" maxOccurs="unbounded" />
    <xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded" />          
  </xs:sequence>
  <!-- added to allow attributes on the element-->
  <xs:anyAttribute namespace="##other" processContents="lax" />
</xs:complexType>

This almost did the trick – I went past the NullReferenceException… but only to hit an exception with “Failed for unknown reasons” along with “SqlServer” in the message:

Could not load signature of System.Data.Entity.SqlServer.SqlProviderServices:GetExecutionStrategy due to: Failed for unknown reasons.

Unhandled Exception:
System.TypeInitializationException: An exception was thrown by the type initializer for ExtentPlaceholderCreator ---&gt; System.TypeLoadException: Could not load type 'System.Data.Entity.SqlServer.SqlProviderServices' from assembly 'EntityFramework.SqlServer, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.
  at (wrapper managed-to-native) System.MonoType:GetPropertiesByName (System.MonoType,string,System.Reflection.BindingFlags,bool,System.Type)
...

Hello??? Anyone there??? I am using MySQL. Any reason you want to load the SqlServer provider? (Yes, the provider was installed by default when I installed the EF package and then, somewhere on my way (must have been after making the app work on .NET Framework), I removed it manually but there should not be a reason to have this provider in the project (and potentially deployed) since it is not being used in my app). This seems like a bug to me but it might have been fixed since alpha2 (a similar issue like this has already been fixed http://entityframework.codeplex.com/workitem/587). Just in case I filed a bug for this https://entityframework.codeplex.com/workitem/1039. To make EF happy I did add the provider for SqlServer and… TADA the app worked.
Just in case – here is the “app” (not even worth putting on github)


namespace EF6MySqlTest
{
  public class Customer
  {
    public int Id { get; set; }
    public string Name { get; set; }
  }

  public class SimpleContext : DbContext
  {
    public SimpleContext()
      : base ("name=MySqlTest")
    {}
    
    public DbSet<Customer> Customers { get; set; }
  }

  class Program
  {
    static void Main(string[] args)
    {
      using (var ctx = new SimpleContext())
      {
        ctx.Customers.Add(new Customer() { Name = "moozzyk" });
        ctx.SaveChanges();
      }

      using (var ctx = new SimpleContext())
      {
        foreach (var c in ctx.Customers)
        {
           Console.WriteLine(c.Name);
        }
      }
    }
  }
}

and the config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <!-- For more information on Entity Framework configuration, visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
    <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
  </configSections>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
  </startup>
  <entityFramework>
    <providers>
      <provider invariantName="Devart.Data.MySql" type="Devart.Data.MySql.Entity.MySqlEntityProviderServices, Devart.Data.MySql.Entity, Version=7.6.217.6, Culture=neutral, PublicKeyToken=09af7300eec23701">
        <migrationSqlGenerator type="Devart.Data.MySql.Entity.Migrations.MySqlEntityMigrationSqlGenerator, Devart.Data.MySql.Entity, Version=7.6.217.6, Culture=neutral, PublicKeyToken=09af7300eec23701" />
      </provider>
    </providers>
  </entityFramework>
  
  <system.data>
    <DbProviderFactories>
      <remove invariant="Devart.Data.MySql" />
      <add name="dotConnect for MySQL" invariant="Devart.Data.MySql" 
           description="Devart dotConnect for MySQL" 
           type="Devart.Data.MySql.MySqlProviderFactory, Devart.Data.MySql, Version=7.6.217.0, Culture=neutral, PublicKeyToken=09af7300eec23701" />
    </DbProviderFactories>
  </system.data>
  
  <connectionStrings>
    <add name="MySqlTest" connectionString="server=localhost;user id=moozzyk;password=XXXXXXXXXXXXX;database=TestDb;persist security info=True" providerName="Devart.Data.MySql" />
  </connectionStrings>
</configuration>

To sum up – it was a pretty bumpy road but I am glad I took it. I found a few real issues we should be able to fix before we ship the final version of EF6 and therefore the experience of using EF6 and Mono should be much smoother.

Entity Framework 6 and pre-generated views

The version for EF6 RTM is now available.

(If you are interested in pre-generated views in EF6 take also a look at this .)

Entity Framework 6 is here. Even though it is in a very early stage it already looks exciting – a lot of improvements in Migrations (multi-tenant migrations, migrations history table customizations), Async, DI for resolving dependencies, code based configuration. Most of it (including features shipped in EF5 – e.g. enums) is running on both .NET Framework 4 and .NET Framework 4.5. In addition trying all of this is as simple as 1, 2, 3 – signed nightly builds are available on a nuget feed. We also take contributions and are thankful to everyone who has already contributed. There is one thing in EF6 that is missing however – the ability to create pre-generated views. I would love it to stay this way but unfortunately views are still one of the problematic areas in EF6. We see some promising activities around views and I hope this will help resolve or at least relieve the problem but for now the solution is still to pre-generate views. So, how do you pre-generate views in EF6? In the previous versions of EF you would either use EdmGen or EF Power Tools. Heck, you could even use my T4 templates. The problem is that all these tools are using System.Data.Entity.Design.dll to generate views and this code was not open sourced. Also, the code generated by System.Data.Entity.Design.dll will not work (without modifications) for EF6. So, it seems it is not possible to pre-generate views on EF6 then… But wait, EF6 is open source isn’t it? Why not make the code that is needed to create views public to enable view generation? It’s one option but there is also a second option – hack the system. While I strongly believe the first option is the right thing to do in the long run for now I went with the second option. There is one main reason for this – making some random functions public to make stuff work is less then ideal. It would be much better to add a nice(r), small API for view generation that could be used by tools that need to generate views. Therefore I decided to create a T4 template for generating views which, at the moment, is using reflection to get what it needs. I treat it just as a prototype (that’s one of the reasons why only C# version exists at the moment) and I hope it will help me define the right API for view generation. When I get to this point I will be able to remove the reflection hacks and just use the API. There is one more thing about the template itself. Since it is not possible to use System.Data.Entity.Design.dll the code needs to be generated by the template itself. It’s a bit more work but allows for much more flexibility. For instance, view generators based on System.Data.Entity.Design.dll were prone to the “No logical space left to create more user strings” error caused by the number of strings in the generated code that could be so big that it reached the .NET metadata format limit on the number of user string characters. This error would prevent an application from starting. This problem is now solved – the template creates an xml file that contains actual view definitions and saves this file in the assembly as an embedded resource. When the EF stack requests views the code generated by the template loads views from the embedded xml file. Using the template is not much different from using the templates for EF5 as it is too published on Visual Studio Code Gallery. First, if you have not already, setup the nuget feed containing EF6 assemblies. Create a new project and add the EF6 nuget package (make sure to select “Include Prelease” in the dropdown at the top of the window) from the feed you created. Now you can start writing your app. Once you have something that compiles right-click on your project and select Add→New Item (Ctrl+Shift+A). In the “Add New Item” window select “Online” on the left. You may want to filter by EF or EF6. Select the “EF6 CodeFirst View Generation T4 Template for C#”. Change the name of the .tt file so that it starts with the name of your context and press the “Add” button:

Once it’s done you should see the template and two new files added to your project – one of the files is the embedded xml resource file containing views and the second is the C# files used to load views from the first file:

If you need to uninstall the templates go to Tools→Extensions and Updates… select the template from the list and click the “Uninstall” button.

That’s it for now. Use EF6, enjoy the template and report bugs for both…

Tagged , , ,

MSBuild Zip task without external dependencies

I have seen quite a few build systems in my life. Most of them were very complicated with a ton of crap a lot of tools, dependencies, perl scripts, batch files, nested targets files and God knows what else. Figuring out how something worked (or, more often, why something did not work) was time consuming and very frustrating. One of the reasons for this was people were adding some stuff to the build system but no one has ever removed anything. What was even more annoying was that new dependencies oftentimes added tens of new files just to enable one small thing. Not only the enlistments were huge (how about ~200GB without QA tests?) but also configuring the machine to be able to build all this and run the tests was a sort of black magic. Because of all this I always felt bad about the fact that MSBuild did not have a Zip task available out-of-the-box. I feel that not having this one thing is the first step to having a build system that everyone hates. Yes, I know there wasn’t a Zip library in .NET Framework until now. Yes, I know there are third party Zip libraries out there. Yes, I know about MSBuild Community Tasks. Yes, I know MSBuild can somehow zip files internally as it can create VSIX files which are zip files… oops – this probably was not the best example. Anyways, not having a built-in Zip tasks means that you need to add some dependencies to your build system to be able to build your project. This will lead to a build system that no one wants to touch to not break anything. What about sharing just a single project? Like for instance my Code First view gen templates? Is it OK to tell people – “you can build it on your own – here is the project, but first you need to install this and this and this or it won’t build”? I don’t think it’s OK, but before I could not help much. Fortunately, a Zip library was finally added to .NET Framework 4.5. This allowed me creating my own Zip task. How is this task different from, for instance, the Zip task from the MSBuild community tasks? I created it as an inline task. As a resull it is just a small text file I can import to my projects. It can be checked in to my source control. It does not require any additional components being installed or present on the machine apart from what’s already there. If I need to know what the task is doing I can see the source without Reflector. I can easily change the task without having to recompile half of my build system just to be able to build what I actually want to build. (See the Disclaimer at the bottom of the page). The task looks just like this:

<UsingTask TaskName="Zip" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
    <ParameterGroup>
      <InputFileNames ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
      <OutputFileName ParameterType="System.String" Required="true" />
      <OverwriteExistingFile ParameterType="System.Boolean" Required="false" />
	</ParameterGroup>
    <Task>
      <Reference Include="System.IO.Compression" />
      <Using Namespace="System.IO.Compression" />
      <Code Type="Fragment" Language="cs">
      <![CDATA[        
        const int BufferSize = 64 * 1024;

        var buffer = new byte[BufferSize];
        var fileMode = OverwriteExistingFile ? FileMode.Create : FileMode.CreateNew;

        using (var outputFileStream = new FileStream(OutputFileName, fileMode))
        {
          using (var archive = new ZipArchive(outputFileStream, ZipArchiveMode.Create))
          {
            foreach (var inputFileName in InputFileNames.Select(f => f.ItemSpec))
            {
              var archiveEntry = archive.CreateEntry(Path.GetFileName(inputFileName));

              using (var fs = new FileStream(inputFileName, FileMode.Open))
              {
                using (var zipStream = archiveEntry.Open())
                {
                  int bytesRead = -1;
                  while ((bytesRead = fs.Read(buffer, 0, BufferSize)) > 0)
                  {
                    zipStream.Write(buffer, 0, bytesRead);
                  }
                }
              }
            }
          }
        }        
      ]]>
      </Code>
    </Task>
  </UsingTask>

Using the task is simple. Put the task to a separate file and import the file to the csproj file. In fact the file is available in my github repo – https://github.com/moozzyk/MSBuild-Tasks. Once you import the file to the project you just invoke the task as you would invoke any other task – for example (this is an actual except from one of my csproj files):


  <Import Project="common.tasks" />
  
  <Target Name="BeforeBuild">
    <ItemGroup>
      <FilesToZip Include="$(ProjectDir)\PayloadUnzipped\*.*" />
    </ItemGroup>
    <Zip 
      InputFileNames="@(FilesToZip)"
      OutputFileName="$(ProjectDir)$(TargetZipFile)"
      OverwriteExistingFile="true" />
  </Target>

That’s pretty much it. Works for me and hopefull will work for you.

Disclaimer:
I am not trying to diminish MSBuild Community Tasks or claim that inline tasks will solve all problems of this world. I am trying to say that for small simple tasks inline tasks can be just much more convenient.

Yes, you can! (create mutifile zip files in .NET Framework (finally))

I recently have been working on moving some projects from an old legacy build system to a new shiny build system. After understanding intricacies of the old build system I was able to isolate and remove 99% stuff we did not really need and comment out the remaining 1% we needed but could not make it work easily. I felt proud of myself because I was even able to build something in the new build system (OK, maybe the stuff we did not need was actually 98.9%?). So, I started uncommenting things we needed but were not as straightforward to fix. One of the first things I came across was the need of zipping files after building. I thought I would just use the MSBuild Zip task and call it done. Unfortunately it turned out that out of the box MSBuild does not have a Zip task. I could find it in MSBuild community tasks but for certain reasons we could not take a dependency on these. We also did not have a zip-like command line tool we could take a dependency on. One of the last sunny days around Seattle this year started looking gloomy to me. I asked folks on our team and they had a few ideas like using PowerShell (we don’t really need it anywhere else so again I wanted to avoid it), leveraging a third party library we could use to create a task or finally using GZipStream. I liked the idea of using GZipStream – I could create and inline MSBuild task which would be clean and simple. However gzip only supports compressing one file and I needed to zip several files so, I could not use it. I spent more time looking for something suitable and thought I found it – ZipPackage!. Again, my hopes were quickly dashed. This guy needs a metadata file containing content types of the included files. This file is then added to the archive. But I needed just a POZF™ (Plain Old Zip File) without any additional crap inside! Afternoon was getting really dark. I started looking at the option of using a third party library. It would add complexity to the build system. I would have to build tools before building projects. Suddenly, I had this feeling that instead of having a lightweight, easy and fast build we will have a new version of what we had before. Each time I wanted to start working on this my hands refused and decided to open a browser and check the news or something along these lines. So, I gave one more shot at finding a better solution but I could only find that there is an MSBuild Zip task in the community tasks and that the GZipStream basically can compress only one file and that the ZipPackage requires some metadata. I was loosing hope when suddenly on Xth page of results (where X > 5) I found this – the ZipArchive class. The ZipArchive is a new class in the .NET Framework 4.5 that allows creating zip files containing not only multiple files but also the whole folder hierarchy! In just one second all the clouds were gone. If there had not been for pierogi I would have probably stayed playing with this class until my MSBuild problem was resolved. On the other hand I don’t like when pierogi are too hot so I stayed a few more minutes just to see if it was easy to create a zip file. Here is what I came up with (I checked only briefly but could not find an example how to create a new archive on msdn):

public static void Main()
{
    Zip(
        new string[] 
        { 
            @"C:\Temp\file1.txt", 
            @"C:\temp\file2.txt", 
            @"C:\Source\ConsoleApplication1\ConsoleApplication1\Program.cs" 
        },
        @"c:\temp\out.zip");
}

static void Zip(string[] inputFileNames, string outputFileName)
{
    using (var outputFileStream = new FileStream(outputFileName, FileMode.Create))
    {
        using (var archive = new ZipArchive(outputFileStream, ZipArchiveMode.Create))
        {
            foreach (var inputFile in inputFileNames)
            {
                AddFileToArchive(archive, inputFile);
            }
        }
    }
}

static void AddFileToArchive(ZipArchive archive, string inputFile)
{
    const int BufferSize = 64 * 1024;

    // relies on inputFile being an absolute path 
    // - Substring(3) removes :\ from the path
    var archiveEntry = archive.CreateEntry(inputFile.Substring(3));

    using (var fs = new FileStream(inputFile, FileMode.Open))
    {
        using (var zipStream = archiveEntry.Open())
        {
            byte[] buffer = new byte[BufferSize];
            int bytesRead = -1;
            while ((bytesRead = fs.Read(buffer, 0, BufferSize)) &gt; 0)
            {
                zipStream.Write(buffer, 0, bytesRead);
            }
        }
    }
}

I can open the archive I created in the Windows Explorer, browse and open files. No additional metadata files are added to the archive – it’s a POZF™. So – yes, you now can create a multifile zip archives in .NET Framework without using third party libraries. And now creating an inline MSBuild Zip task that fits my new build system is easy. This makes me happy!

Pawel Kadluczka