Currently Entity Framework does not natively support second level caching. For pre-EF6 versions you could use EF Caching Provider Wrapper but due to changes to the EF provider model in EF6 it does not work with newest versions of EF. In theory it would be possible to recompile the old caching provider against EF6 but, unfortunately, this would not be sufficient. A number of new features breaking some assumptions made in the old caching provider (e.g. support for passing an open connection when instantiating a new context) have been introduced in EF6 resulting in the old caching provider not working correctly in these scenarios. Also dependency injection and Code-based Configuration introduced in EF6 simplify registering wrapping providers which was rather cumbersome (especially for Code First) in the previous versions. To fill the gap I created the Second Level Cache for EF 6.1 project. It’s a combination of a wrapping provider and a transaction handler which work together on caching query results and keeping the cache in a consistent state. In a typical scenario query results are cached by the wrapping caching provider and are invalidated by the transaction handler if an entity set on which cached query results depend was modified (note that data modification in EF always happens in a transaction hence the need of using transaction handler).
Using the cache is easy but requires a couple steps. First you need to install the EntityFramework.Cache NuGet package. There are two ways to do this from Visual Studio. One way to do this is to use the UI – right click the References node in the Solution Explorer and select the Manage NuGet Packages option. This will open a dialog you use to install NuGet packages. Since the project is currently in the alpha stage you need to select “Include Prelease” in the drop down at the top. Then enter “EntityFramework.Cache” in the search window and, once the package appears, click the “Install” button.
Installing EntityFramework.Cache from UI
You can also install the package from the Package Manager Console. Open the Package Manager Console (Tools → NuGet Package Manager → Package Manager Console) and execute the following command:
Install-Package EntityFramework.Cache –Pre
(-Pre allows installing pre-release packages).
Note that the package depends on Entity Framework 6.1. If you don’t have Entity Framework 6.1 in your project it will be automatically installed when you install the EntityFramework.Cache package.
Once the package is installed you need to tell EF to use caching by configuring the caching provider and the transaction handler. You do this by creating a configuration class derived from the DbConfiguration class. In the constructor of your DbConfiguration derived class you need to set the transaction interceptor and the Loaded event handler which will be responsible for replacing the provider. Here is an example of how to setup caching which uses the built-in in-memory cache implementation and the default caching policy.
public class Configuration : DbConfiguration
{
public Configuration()
{
var transactionHandler = new CacheTransactionHandler(new InMmemoryCache());
AddInterceptor(transactionHandler);
Loaded +=
(sender, args) => args.ReplaceService<DbProviderServices>(
(s, _) => new CachingProviderServices(s, transactionHandler,
new DefaultCachingPolicy()));
}
}
The default caching policy used above is part of the project and allows caching all query results regardless of their size or entity sets used to obtain the results. Both, sliding and absolute, expiration times in the default caching policy are set to maximum values therefore items will be cached until an entity set used to obtain the results depended on is modified. If the default caching policy is not suitable for your needs you can create a custom caching policy in which you can limit what will be cached. To create a custom policy you just need to derive a class from the CachingPolicy class, implement the abstract methods, and pass the policy to the CachingProviderServices during registration. When implementing a custom caching policy there is one thing to be aware of – the expired entries are removed from the cache lazily. It means that an entry will be removed by the caching provider only when the caching provider tries to read the entry and finds it is expired. This is not extremely helpful (especially because since the item is expired the provider will query the database to get fresh results which will be then put to the cache – so effectively the expired entry will be replaced with a new entry) but lets the user decide what the best strategy of cleaning the cache in their case is. For instance, in the InMemoryCache implementation included in the project I created a Purge method. This method could be periodically called to remove stale cache entries.
The project also includes a simple implementation of a cache which caches query results in memory. This is just a sample implementation and if you would like to use a different caching mechanism you are free to do so – you just need to implement the ICache interface. This interface is a slightly modified version of the interface that shipped with the original EF Caching Provider Wrapper which should make moving existing apps using the old caching solution to EF6 easier.
As the old saying goes “A program is worth a 1000 words“, so let’s take a look at the second level cache in action. Here is a complete sample app which is using the cache:
public class Airline
{
[Key]
public string Code { get; set; }
public string Name { get; set; }
public virtual ICollection<Aircraft> Aircraft { get; set; }
}
public class Aircraft
{
public int Id { get; set; }
public string EquipmentCode { get; set; }
public virtual Airline Airline { get; set; }
}
public class AirTravelContext : DbContext
{
static AirTravelContext()
{
Database.SetInitializer(new DropCreateDatabaseIfModelChanges<AirTravelContext>());
}
public DbSet<Airline> Airlines { get; set; }
public DbSet<Aircraft> Aircraft { get; set; }
}
public class Configuration : DbConfiguration
{
public Configuration()
{
var transactionHandler = new CacheTransactionHandler(Program.Cache);
AddInterceptor(transactionHandler);
Loaded +=
(sender, args) => args.ReplaceService<DbProviderServices>(
(s, _) => new CachingProviderServices(s, transactionHandler));
}
}
class Program
{
internal static readonly InMemoryCache Cache = new InMemoryCache();
private static void Seed()
{
using (var ctx = new AirTravelContext())
{
ctx.Airlines.Add(
new Airline
{
Code = "UA",
Name = "United Airlines",
Aircraft = new List<Aircraft>
{
new Aircraft {EquipmentCode = "788"},
new Aircraft {EquipmentCode = "763"}
}
});
ctx.Airlines.Add(
new Airline
{
Code = "FR",
Name = "Ryan Air",
Aircraft = new List<Aircraft>
{
new Aircraft {EquipmentCode = "738"},
}
});
ctx.SaveChanges();
}
}
private static void RemoveData()
{
using (var ctx = new AirTravelContext())
{
ctx.Database.ExecuteSqlCommand("DELETE FROM Aircraft");
ctx.Database.ExecuteSqlCommand("DELETE FROM Airlines");
}
}
private static void PrintAirlinesAndAircraft()
{
using (var ctx = new AirTravelContext())
{
foreach (var airline in ctx.Airlines.Include(a => a.Aircraft))
{
Console.WriteLine("{0}: {1}", airline.Code,
string.Join(", ", airline.Aircraft.Select(a => a.EquipmentCode)));
}
}
}
private static void PrintAirlineCount()
{
using (var ctx = new AirTravelContext())
{
Console.WriteLine("Airline Count: {0}", ctx.Airlines.Count());
}
}
static void Main(string[] args)
{
// populate and print data
Console.WriteLine("Entries in cache: {0}", Cache.Count);
RemoveData();
Seed();
PrintAirlinesAndAircraft();
Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
// remove data bypassing cache
RemoveData();
// not cached - goes to the database and counts airlines
PrintAirlineCount();
// prints results from cache
PrintAirlinesAndAircraft();
Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
// repopulate data - invalidates cache
Seed();
Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
// print data
PrintAirlineCount();
PrintAirlinesAndAircraft();
Console.WriteLine("\nEntries in cache: {0}", Cache.Count);
}
}
The app is pretty simple – a couple of entities, a context class, code base configuration (should look familiar), a few methods and the Main() method which drives the execution. One method that is worth mentioning is the RemoveData() method. It removes the data from the tables using the SqlExecuteMethod() therefore bypassing the entire Entity Framework update pipeline including the caching provider. This is to show that the cache really works but is at the same type a kind of warning – if you bypass EF you will need to make sure the cache is in a consistent state or you can get incorrect results. Running the sample app results in the following output:
Entries in cache: 0 FR: 738 UA: 788, 763 Entries in cache: 3 Removing data directly from the database Airline Count: 0 FR: 738 UA: 788, 763 Entries in cache: 4 Entries in cache: 2 Airline Count: 2 FR: 738 UA: 788, 763 Entries in cache: 4 Press any key to continue . . .
Let’s analyze what’s happening here. On line 1 we just report that the cache is empty (no entries in the cache). Seems correct – no queries have been sent to the database so far. Then we remove any stale data from the database, seed the database, and print the contents of the database (lines 2 and 3). Printing the content of the database requires querying the database which should result in some entries in the cache. Indeed on line 4 three entries are reported to be in the cache. Why three if we sent just one query? If you peek at the cache with the debugger you will see that two of these entries are for the HistoryContext so three seems correct. Now (line 6) we delete all the data in the database so when we query the database for the airline count (line 7) the result is 0. However, even though we don’t have any data in the database, we can still print data on lines 8 and 9. This data comes from the cache – note that when we deleted data we used ExecuteSqlCommand() method which bypassed the EF update pipeline and, as a result, no cache entries were invalidated. On line 11 we again report the number of items in the cache. This time the cache contains four entries – two entries for the HistoryContext related queries, one entry for the data that was initially cached and one entry for the result of the query where we asked for the number of airlines in the database. Now we add the data to the database again and again print the number of items in the cache on line 13. This time the number of entries is two. This is because inserting the data to the database invalidated cached results for queries that used either Airlines or Aircraft entity sets and as a result only the results related to the HistoryContext remained in the cache. We again query the database for the number of airlines (line 14) and print all the data from the database. The results for both queries are added to the cache and we end up having four entries in the cache again (line 18).
That’s pretty much it. Try it out and let me what you think (or report bugs). You can also look at and play with the code – it is available on the project website on codeplex.
Note that this version is an alpha version and there is still some work remainig before it can be called done – most notably support for async methods, adding some missing overrides for the ADO.NET wrapping classes. I am also considering adding support for TransactionScope.
Code, the Universe and Everything... 
Nice work. I swapped out in memory cache with Azure Cache in my project and it works great.
LikeLike
Thanks for kind words! I am glad it works for you!
LikeLike
Hi, We are implementing EF 6.0 second level cache in our application. We decide to use worker role cache. Hence, we replaced Dictionary object that is taking CacheEntry as parameter with Azure DataCache object. But, we are not able to hit the database after insertion or deletion. Please share your ideas to make it workable. If you share the code if possible, its really appreciated.
LikeLike
Thanks very much for this! I’m looking forward to swapping out ef5/cache wrapper in the mvcforum.com project (built on .NET 4). Can you please update package with a .NET 4.0 build? Would be much appreciated!
LikeLike
Good feedback. I will add this to my list for beta release. If you would like to see this sooner you can contribute to the project – the code is on http://efcache.codeplex.com.
Thanks,
Pawel
LikeLike
moozzyk I just grabbed the source and made a .NET 4.0 build myself, it was a breeze. I haven’t made any NuGet packages before, I’ll just update when you do 🙂
LikeLike
Good to hear that it was easy to build for .NET Framework 4. So far I’ve been building the NuGet packages manually but I’m thinking about automating this for Beta – it will be even more important with .Net version specific assemblies. Another thing I realized after I saw your request was that I would need .NET version specific assemblies anyways after I add support for asnyc since async is not supported on .NET Framework 4.
LikeLike
Perfect solution!
is it bug?
CachingReader.cs
public override bool IsDBNull(int ordinal)
{
return GetValue(ordinal) == null;
}
public override object GetValue(int ordinal)
{
EnsureReading();
//Should add code like
object obj = _resultRowsEnumerator.Current[ordinal];
if (obj == DBNull.Value) return null;
return _resultRowsEnumerator.Current[ordinal];
}
LikeLike
Yes. I think it is the root cause of https://efcache.codeplex.com/workitem/1
LikeLike
The fix for the bug was checked in and I uploaded a new version of the package with the fix to NuGet today.
Thanks,
Pawel
LikeLike
I want to use your cache, but I need to design custom EntitySets invalidation rules. This is due to use views in my database. So modification done to some table may invalidate not only EntitySet representing this table data, but also other EntitySet representing a view, which is based on this table. As I see it now I would need to create my custom TransactionHandler, but this is internal object of your cache mechanism, which – as I understand – is also responsible for inserting records to cache storage, so this is rather not a solution. May it be achieved easy?
Thanks,
Michał
LikeLike
OK, sorry, I did not check TransactionHandler source code earlier – I see it may be achieved with easy with custom TH 🙂
LikeLike
OK, I still do have problem with that. No as easy as I thought. Any idea how to achieve that?
LikeLike
I think you could derive from the CacheTransactionHandler and override InvalidateSets method so that you invalidate related sets if there is no transaction or invoke AddAffectedEntitySets if a transaction is present (note: AddAffectedEntitySets is now private you would have to make it protected – I will file a bug to make it protected since this can be useful).
Another thing you could try is to derive from or wrap your cache and customize InvalidateSet method so that it invalidates not only the set that was passed but also related sets.
LikeLike
Very Informative. I have a question on Custom Policy class that can be inherited from CachingPolicy class – How can we include/exclude entities? I see three methods CanBeCached, GetCacheableRows and GetExpirationTimeout that overridden from base class but did not find way to configure what entities that I need to put in cache.
Any help on this would be greatly appropriated.
Thanks,
Paresh
LikeLike
Hi Paresh,
The cache works at the provider level where there is no notion of entities. It basically caches results as they come from the database (data from DbDataReader is stored in arrays that are later exposed as DbDataReader when requested to avoid the trip to the database) and uses the input SQL queries to generate keys for the cache entries. Currently the CachePolicy allows to control what is being cached based on the store EntitySet (think: table in the database) and the number of results returned by a query – i.e. you can look at the affectedEntitySets parameter to see what tables the result is coming from and decide whether you want to cache this or not (this is the CanBeCached method). Results for queries containing code that has side effects are not cached. If you think about entities they are actually cached by EF for the lifetime of the context (unless you delete/detach them) and at the level this cache works it would be quite hard to convert data that comes from the database to entities – this is actually EF’s job.
Hope this helps,
Pawel
LikeLike
Hi Pawel,
Thanks for prompt reply. A have a question: Where and how should I specify list of database tables should included or excluded for cache? as per your reply looks like i can use CanBeCached method, but not clear with how can i write code. A small sample code would help greatly.
Thanks,
Paresh
LikeLike
Let’s say you have a list of names of tables for which results should not be cached. In this case you can do something along these lines:
return !affectedEntitySets.Select(e => e.Table ?? e.Name).Any(tableName => nonCachableTables.Contains(tableName));
LikeLike
Hi Pawel, You can ignore my last reply. I found the way how it can be configured in CanBeCached Method.
Is it possible to use XML file to configure queries/table included or excluded for cache, if yes, can you point me out any article that help understand it?
Thanks,
Paresh
LikeLike
Currently it is up to you to code the logic to decide whether the results should be cached or not (the CanBeCached method implementation). If you wish you can use Xml or store the information in the app.config you can (if you do that you may want to make sure that for performance reasons this information is not loaded each time a query is sent – the easiest way to do this would be to cache the values inside your caching policy implementation and make sure that the caching policy is not recreated each time the CachingProviderServices are instantiated (i.e. create an instance of your caching policy beforehand outside the Load event registration and pass this instance to the CachingProviderServices ctor like this:
var cachingPolicy = new MyCachingPolicy()
Loaded +=
(sender, args) => args.ReplaceService(
(s, _) => new CachingProviderServices(s, transactionHandler, cachingPolicy));
)). I am not sure how much it is better than just hardcoding names but there might be scenarios where this might be desired.
Having said that, I have been working on a solution where you can exclude specific queries from being cached. This consists of two pieces:
The above should be included in Beta.
Hope this helps,
Pawel
LikeLike
Hi Pawel,
Provided information was really helpful. I could reach to some stage where I have used table to be cached configuration from Web.Config in Custom Caching Policy class/object.
Now I have further requirement to store subset of table results using where clause in query. Any suggestion on how can I do it in current prerelease version?
Thanks for your support,
Paresh
LikeLike
Hi Paresh,
Take a look at Beta version I just published. The
CanBeCachedmethod was modified and now also takes the Sql and parameters.Hope this helps,
Pawel
LikeLike
Hi Pawel,
Thanks for update. I have installed Beta Version and for .Net Framework v4.0, as soon as I installed, I am getting compile time error for CanBeCached Method in my Custom Cache Policy Class. It says “There is not suitable method to override”. Any clue on what am I doing wrong?
Thanks,
Paresh
LikeLike
Ooops. I found the problem. It is because you have added SQL parameter. I placed it and now its compiling. I will try further and let you know how it works.
Thanks a bunch,
Paresh
LikeLike
hi
I want use cache in ef but I have two context
I want use cache in one context also my context connect to more database and need caching by databasename
how can i do ?
LikeLike
Sorry for the delayed answer. I don’t think you currently can reliably cache items depending on the database if you use the same model but connect to different databases. However, it could be done relatively easily by changing the code so that the database name is part of the cache key. I will think about doing this.
Thanks,
Pawel
LikeLike
Today I commited a change where I made database name part of the key. If you need it now you can clone the repo and build the sources. Otherwise you would have to wait until I ship the beta-2 version on NuGet.
Thanks,
Pawel
LikeLike
very thanks for answer and update source
for different database i use different connection name
for customize your source code can i see beta-2 your source code ?
LikeLike
Code is on codeplex – http://efcache.codeplex.com/
LikeLike
if have more context how can choise eny context cahing and eny context no cashing?
LikeLike
I think there are currently two ways of doing this:
– create a custom caching policy which will return true from the
CanBeCachedmethod only for entity sets from the context you want the query results to be cached– use
Cached()/NotCachedmethods to manually select queries for which results should be cached/not cached.I can see that selecting this on the context level can be useful – I would accept a contribution that implements this.
Thanks,
Pawel
LikeLike
Is there support for large results where there are many nested relationships? I’m wondering if this will be automatically broken down into individual entities when persisted to cache. That would be a great help. My parent entity is too large to store in cache.
Thanks,
Steve
LikeLike
I would like to know if the query result stored in the memory cache is kept up to date or becomes stale over a period of time.
How does the memory cache know if someone has updated the data via another process or directly from the database?
My main point is the cache just a snapshot at a given time or does it keep a real-time update which would make it very powerful.
LikeLike
Database/EF is not pushing changes to clients so the results can get stale. This is why Caching Policy along with
Cached()/NotCached()extension methods are so important. Caching policy allows you to decide whether to cache results or not depending on the EntitySets involved or the number of returned rows, and how long to keep results in the cache.Cached()/NotCached()extension methods allow to hand pick queries whose results should be cached (or not cached).Even if the push was available I don’t know if it was possible to update cache with new data given how this cache (and EF) works. It would be possible to invalidate the cache though (e.g. using SignalR).
Thanks,
Pawel
LikeLike
Thanks, so it does not support a fully functional real-time cache system which I can trust the data within the cache will always be up to date.
LikeLike
Nope.
LikeLike
at first thanks for this, its great.
i have a problem in use this, i use from your exam,and so work,but from start project,any query that execute, go to cache, but i want determine that which method(query) ,when execute go to cash.
how i can do this?
LikeLike
You need to derive from the `CachingPolicy` class and override the `CanBeCached` method. If you want more granular control you can use `.Cached()`\.`.NotCached()` extension methods on the queries. Finally you can mix both approaches – for instance you can create a caching policy that always returns false (so by default nothing is cached) but then use the `.Cached()` extension methods on the particular queries if you want to cache results only for these queries.
LikeLike
Any benchmark comparison?
LikeLike
Not sure what to really benchmark here. Reading from memory will always be faster than making a trip to the database. How much faster it is going to be? It depends on where your database is. Accessing a local database will be probably faster than accessing a database on a separate box which will probably be faster than accessing a database in the cloud. Also I believe one should not cache results for all queries blindly – rather one should look at one’s application and identify queries whose results should be cached.
LikeLike
Very nice! I will try it out!
LikeLike
This is a pretty cool project. One thing I would suggest is to allow a Caching Policy to be scoped to a lifetime. For instance, I want to cache all queries for the http request cycle. So if I get an object by id in one place, and then get it again else where, I only make one request to the DB.
LikeLike
It’s an interesting idea but I am not sure if it could/would work in practice given how the cache is implemented. The issue is that the cache caches results for each query and it does it at the lowest possible level i.e. the DbReader level. At this level there is no notion of entities, objects etc. – only resultsets. Note that the way EF build queries is that regardless of how many effective joins you have in your query you will always get just one resultset containing all the data which is then processed to create corresponding entities. As a result if you send a query and cache results and send another query that from the end user perspective should/could use results from the previous query (e.g. the first query fetches all the Orders and the second query fatches all the Customers with Orders) the queries will be treated as totally different queries and as a result the results of these queries will be cached in separate cache entries. Caching results this way can work since typically apps use a limited number of queries and you want to cache results only for some of them.
I think the scenario you are mentioning is kind of supported today even without the EF Cache. If you used one
DbContextinstance when processing a request the context will automatically cache the entities you queried and you can retrieve them using theDbContext.Find()method. This method first checks if the entity you are asking for is available locally and returns it if it is, otherwise it fetches from the database (more details here: http://msdn.microsoft.com/en-us/library/gg696418(v=vs.113).aspx). Another doubt I have is whether having a very short lived cache entries is useful. In case of a web server you want to process web requests as fast as possible so you don’t want to send the same query to the database twice. Therefore you would never use the results you cached. Rather, I would think that in this scenario it makes sense to have a longer lived cache. This way other requests could use results from the cache. Imagine a scenario where text resources in different languages are stored in the database and are used to render the contents of web pages. In this case the resources need to be fetched from the database for each request. Since the resources don’t change frequently it makes sense to cache them – after the first request all other requests can use locally cached version instead of going to the database. It would save time (accessing memory is much faster than database) and would remove a lot of load from the database since the requests are not even being sent there.Hope this makes sense.
Thanks,
Pawel
LikeLike
I am using your caching provider in a plugin for a web service. Because I am using a plugin architecture there is no way that I can ensure that the Loaded event in the Configuration constructor is added before a DbSet is used. Do you have any advice for how I can ensure this, or if it’s possible to remove that event handler?
LikeLike
The event handler is essential to the EFCache because EFCache wraps the EF provider and you cannot change the provider once it has been set up.
Without knowing more about your architecture it is hard to tell what you can do – for instance I don’t understand “there is no way that I can ensure that the Loaded event in the Configuration constructor is added before a DbSet is used”. EF instantiates the DbConfiguration class only once per app domain and before the model is built and there is no way of reconfiguring it later. You just need to put this class next to your DbContext derived class and the caching will be enabled.
Thanks,
Pawel
LikeLike
The problem is that this configuration is in plugin A. But the service doesn’t necessarily go and look for plugin A as soon as it initializes. If the service loads plugin B, and plugin B uses EF, then when plugin A is finally loaded it will try to configure EF and throw an exception. Since both plugins are on the same app domain they can conflict. So basically, is there any way to dynamically enable caching after the database has been configured?
LikeLike
The only logical place to put the configuration class is the project where the EF model is defined. If I understand your architecture correctly the model cannot be defined in any of the plugins since this would require to always load the plugin that contains model before any other plugin that uses this model. EF always uses a configuration to build the runtime model/metadata – it has a built-in configuration you override by providing your own `DbConfiguration`. By putting it to the same project where your EF model is you ensure that EF will use yours instead of the built-in one. EF also looks at the web.config/app.config file but I am not sure if you can configure the Loaded even in the config file.
Thanks,
Pawel
LikeLike
There is bug in using efcache with http://www.hibernatingrhinos.com/products/efprof
LikeLike
Can you log an issue (https://efcache.codeplex.com/WorkItem/Create) and provide details what the bug is? Please show how you set up EF Cache and EFProf and post the exception with all the details (the stack trace, inner exceptions etc.) if there is any.
Thanks,
Pawel
LikeLike
Amazing piece of work, speeds my application by an order of magnitude, with no disadvantages. Many thanks.
LikeLike
Thanks. I am glad it works for you!
LikeLike
Looks fantastic, any chance of a vb.net example, as having trouble raising events after using online convertor,
LikeLike
Imports EFCache
Imports System.Data.Entity.Core.Common
Imports System.Data.Entity.Infrastructure.DependencyResolution
Public Class Configuration
Inherits System.Data.Entity.DbConfiguration
Public Sub New()
Dim transactionHandler = New CacheTransactionHandler(New InMemoryCache())
AddInterceptor(transactionHandler)
Dim CachingPolicy = New CachingPolicy
AddHandler Loaded, Sub(sender As Object, e As DbConfigurationLoadedEventArgs) e.ReplaceService(Of DbProviderServices)(Function(serviceInterceptor As DbProviderServices, o As Object) New CachingProviderServices(serviceInterceptor, transactionHandler, CachingPolicy))
End Sub
End Class
It caches everything! how do i stop it from caching certan tables, can i configure it using the above configuration class?
LikeLike
You would have to either override the `CachingPolicy.CanBeCached` to return false for entity sets for which you don’t want the results to be cached or use the `NotCached()` extension method on queries. See this post to learn more about the `.Cached()/.NotCached()` extension methods.
Pawel
LikeLike
Am i correct in thinking that the worker process shares the cache so anything updated with entity framework will show on all other users on that worker process? If that is the case then i am happy to cache everything as rarely make direct changes to database. Without restarting the worker process, is there an easier way for me to clear cache and maybe restrict how long it stores? Thanks
LikeLike
This is true assuming you have just one AppDomain in which EF runs and all clients are connecting to the EF instance/AppDomain. To control how long items are cached you need to override the CachingPolicy class and implement the GetExpirationTimeout method. If you want to force clear the cache you can invoke the InvalidateSets method for each entity set. Take a look at this discussion for more details https://efcache.codeplex.com/discussions/571007.
Pawel
LikeLike
Reblogged this on ITelite.
LikeLike
for some reason I don’t know why, after commiting entities the InvalidateSets method not calling
how can I call it manually?
LikeLike
Invoking it manually does not make much sense since it’s tied to transaction. Make sure that the transaction handler is registered properly.
Hope this helps,
Pawel
LikeLike
let me explain it more
I have 2 different project and they are committing data properly using same method
from one place exactly same one of them is my api and another one is nservice bus.
for some reason(api) called InvalidateSets but (nsb) skipping.
they are using same DbContext
and I config Dbcontext with this code
public class Configuration : DbConfiguration
{
public bool Iscacheable { get; set; }
public Configuration()
{
string configstr = System.Configuration.ConfigurationSettings.AppSettings.Get(“idealink.ef-cache”);
Iscacheable = false;
if (!configstr.IsNullOrEmpty())
{
ConfigStringBuilder config = new ConfigStringBuilder(configstr);
string provider = config.Get(“provider”, required: true);
object cacheProvider = null;
switch (provider.Trim().ToLower())
{
// default Azure
case “azure”:
cacheProvider = new AzureCache(config);
break;
// default Memcached
case “memcached”:
cacheProvider = new MemcachedCache(config);
break;
case “localmemory”:
cacheProvider = new MemoryCache();
break;
// custom type
default:
try
{
}
catch (Exception exception)
{
throw new ConfigurationErrorsException(
“Could not find an implementation of cache provider ‘{0}’.”.FormatWith(provider),
exception);
}
break;
}
if (!cacheProvider.IsNull())
{
var transactionHandler = new CacheTransactionHandler((ICache) cacheProvider);
AddInterceptor(transactionHandler);
Iscacheable = true;
Loaded +=
(sender, args) => args.ReplaceService(
(s, _) => new CachingProviderServices(s, transactionHandler,
new CachingPolicy()));
}
}
}
}
my question is how can I get my memory provider object that is on fly to call InvalidateSets manually ? or how can I checking transaction registered properly?
LikeLike
Assuming you will end up using the InMemoryCache the way you register cache seems correct. Not sure how you concluded that IvnalidateEntitySets is being skipped, but if the projects are running separately (i.e. in different processes (AppDomains)) they will have their own instances of cache and invalidating entity sets in one instance won’t invalidate entity sets in the other instance. If this is the behavior you are looking for then you need to make sure that both projects are using the same cache instance. In this scenario I don’t think the simple InMemoryCache can work.
Hope this helps.
Pawel
LikeLike
Hello ,
i have web forms application and I want to use the second level cache framework you have implemented. I use EF 6.0 Database first. i just have an Entity Called Persons. I use the AdventureWorks2014 database.
please have a look at the very simple code below. Ηow can I use the second level cache in this example? would it put in cache all my select queries? thank you
public partial class WebForm1 : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
SelectAll();
}
private static List SelectAll()
{
AdventureWorks2014Entities ctx = new AdventureWorks2014Entities();
var query = from p in ctx.People
orderby p.BusinessEntityID ascending
select p;
return query.ToList();
}
}
LikeLike
Did you add the a DbConfiguration derived class as shown in the post above? You need this to enable 2nd level cache.
If you are using the default caching policy results for all queries will be cached. However you can subclass the CachingPolicy class and override CanBeCached method to change this behaviour to cache results only for selected queries.
Thanks,
Pawel
LikeLike
Hello again,
it works now
thanks again.
may i ask this
if you have queries like this
string sqlComand = String.Format(@String.Concat(“select distinct mycolumn from mytable);
IEnumerable result = db.Database.SqlQuery(sqlComand);
will the efcache work?
LikeLike
SqlQuery and SqlCommand are really backdoors bypassing most of the EF logic. As a result the cache does not have enough information to be able to invalidate results returned by SqlQuery nor cached results when using SqlCommand. As a result results from SqlQuery are not cached and SqlCommand does not invalidate cached results even if the command affects the data that is cached.
Thanks,
Pawel
LikeLike
hello again,
most of my queries are like that. they use SQLQuery. so you are saying not to use your solution for caching for my particular project. is that correct?
LikeLike
No, I am saying that caching won’t work SqlQuery/SqlCommand. I think there are two important points I want to make. First you should be careful with what you cache. Caching blindly results for all queries is probably not a good idea and may lead to incorrect application behavior. (I think that if you are using caching you actually should consider using concurrency token to prevent situations where you overwrite current data with stale data that came from the cache). So, possibly after analysis you would identify entiy sets for each you would want to cache the results and for which you don’t use SqlQuery to get results. The second point is that SqlQuery is kind of a backdoor – the idea is that you use it in cases where for some reason Linq to Entity is not able to create a query you need. If most of your queries use SqlQuery then it feels like either EF is not the right tool for the job (you pay for the overhead but you don’t and can’t use some/many EF features) or you need to rethink how you use EF.
LikeLike
Im loving your work, it provides a very simple solution to a profound problem. Much obliged.
LikeLike
Thanks!
LikeLike
Hi
I wanted to include your cache in my application, but only have Entity Framework 6.0 (not 6.1), because the clients are ships and updates run over a costly satellite connection 🙂
Can you tell me the reason that this requires EF 6.1 instead of just 6.0?
Thanks,
Felix
LikeLike
I just tied to “force” it and now i know 🙂
System.TypeLoadException: Could not load type ‘System.Data.Entity.Infrastructure.TableExistenceChecker’ from assembly ‘EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089’.
even if I do not know why your cache would need something like TableExistenceChecker, but maybe it is just in the internals and you do not use it directly.
Long story short, I have to use something different 🙂
LikeLike
Ok, now I know 🙂
I tried to forced it with EF 6.0, but that Version does not have the required IDbTransactionInterceptor yet. OK, have to search for something else
LikeLike
If you are willing to use EF Cache it means that you control the application and you can make changes. In that case why not update EF to 6.1+?
Pawel
LikeLike
Sorry, that error had nothing to do with your cache. I did not copy all EF 6.03 libs (the current version uses EF 6.1.3 for current setups, but older setups still use EF 6.0.3. This works, because EF just says “Version 6.0.0.0” always)..
Will try again later if your cache runs with EF 6.0
LikeLike
Hi again,
i registed an account now, so that i can edit my posts hopefully 😀
feel free to remove all but my first comment (with my question why this requires EF 6.1) to kill all useless this spam 😀
LikeLike
I am trying to use the cache in my application. To test what I implemented I am trying to select a single record with the same info. The Cache count is not increasing as expected as the same query is being executed. I created a new tracer in SQL Profiler and I could see the database is being hit all the time. It appears that the Cached queries are not being used and the db is being hit all the time.
LikeLike
Make sure you registered the EF Cache provider correctly. If you did and it still does not work you need to post the repro somewhere. Also there are cases (like using the `SqlQuery` function) where EFCache won’t kick in since in those cases EF pipeline is bypassed entirely.
Thanks,
Pawel
LikeLike
Can we use EF with WebApi?
LikeLike
I don’t see any reason why you couldn’t…
LikeLike
I notice that EFCache does not handle situations where DbContextTransactions are used with rollback and commit. Looking through the source in theory it would not be much effort to modify the caching to handle transactions although there are some tricky details around cache invalidation and promotion. Has anyone done the work to make EFCache work with transactions?
LikeLike
EFCache should work with transactions – the transaction handler (http://efcache.codeplex.com/SourceControl/latest#EFCache/CacheTransactionHandler.cs) is responsible for either invalidating the cache if the transaction has been committed or removing sets to be invalidated in case of roll back so that they are not invalidated.
Pawel
LikeLike
Hi,
Nice job, it was very easy to provide 2nd-level caching in my app. However I ran into a problem and I don’t know how to exclude some entities from caching.
Could you please take a look at the following question?
http://stackoverflow.com/q/33885939/3967440
Thanks,
LikeLike
As you found out you can control what should be cached and what should not be cached.
Cheers,
Pawel
LikeLike
Hi!
Hope you still support!
When I use DbSet.SqlQuery, the data retrieved not cache.
exam:
– dbcontext.MyTable.Where(….).TolistAsync() ==> cached
– dbcontext.MyTable.SqlQuery(“my_storedprocedure”).ToListAsync() ==> not cached
How do I cached data from sql query?
Thanks
LikeLike
Results from SqlQuery can’t be cached. Commands executed with ExecuteSqlCommand will not invalidate cache. Both SqlQuery and ExecuteSqlCommand bypass EF pipeline and therefore bypass the cache as well.
Thanks,
Pawel
LikeLike
Ok, but, for example, since you have sliding and absolute as something you have to return which one takes precedence if I override GetExpirationTimeout? Should I call base to set both and then change the one I want?
These small things are missing and they force me to test it all to be sure how it behaves.
LikeLike
It’s up to the backing cache mechanism to handle expired entries. The sample MemoryCache uses the aggressive policy where cache entries will be considered expired if either of the values is exceeded (http://efcache.codeplex.com/SourceControl/latest#EFCache/InMemoryCache.cs). Obviously the entries will be invalidated automatically if there is a CUD operation on the table the entries where taken from. To configure these values you need to subclass the CachingPolicy class, implement the methods accordingly and pass your own policy when configuring the cache.
LikeLike
Hi. I’m using your package and have a problem That is conflict data in memory when i cache the data from methods.
i have multiple db context (the configuration just can be on one of the contexts, if you want i can send the error) and using structure map for injection . Also the platform of designing is Asp.net Webform. Can you say some things about this problem that conflict data when the cache is enabled? or is there any way to debug??
LikeLike
In short this scenario is currently not really supported. For more details you can start from here: https://efcache.codeplex.com/discussions/657132. In one of the posts I provided links to discussions on this topic.
In EF6 there is one configuration for the entire application and configuring EF Cache is per provider not per context. You can override caching policy if you want to decide what you want to cache (and in most cases you actually should do that because caching everything indiscriminately is in general not a good idea).
Thanks,
Pawel
LikeLike
Hi moozzyk,
Any chance you can limit the size of the cache. Ie. stop it from taking x amount of memory?
Thanks
LikeLike
Currently there is no means to calculate how much memory the data is using. If you think you cache too much the main thing to do is to review what you are caching – caching blindly all results is in general not a good idea. Cache only results that are not frequently changing. Check your queries and make sure that the amount of data they are returning is reasonable (e.g. if your query returns e.g. 1000000 rows it likely does not make much sense). You can configure expiration of cached items – note that in case of InMemoryCache you need to Purge items periodically since the expired items are not removed from the cache until they are replaced by fresh items.
Hope this helps,
Pawel
LikeLike
Ok, thanks. I just wanted to clarify this. We limit rows and have expires set. Thanks for the tip on the purge! That’s one thing we are not doing.
LikeLike
Is it applicable for EntityFramework 6.2
LikeLike
Yes, it will work with EF 6.2 too
LikeLike
Hi moozzyk,
First of all thanks for this great library. I just have problems about reading from cache old values. In my scenario in our project, I read a data from db (without transaction) and it’s started to be stored in cache and then later i update a field of the same record (with a transaction) and then later i started to read same record(without transaction) in the code and it reads the previously cached value. as i analyzed the source code it only invalidates the record with the given transaction so thats why probably it doesnt invalidate. i just tried to initialize the dbtransaction to null in my implemented InvalidateSets just to see what happens and it’s started to work correctly as for me. so do I have something wrong about my implementation? or what’s the purpose of this design maybe i understood something wrong about the concept.
Thanks for your support,
LikeLike
Hi Hakan,
Thanks for the kind words. I am glad to hear that you found EFCache useful.
How are you creating a new transaction – are you using TransactionScope? Because TransactionScope is currently not supported. I have an issue on that: https://github.com/moozzyk/EFCache/issues/2 but so far there is no progress on this. Some time ago there was a PR to fix that but it was never finished. So, if you are using TransactionScope you are unfortunately out of luck. DbContext.Database.BeginTransaction should work though and is much cheaper than TransactionScope.
Hope this helps,
Pawel
LikeLike
Thanks Pawel,
Actually we are not using TransactionScope. We are using IOC container (Microsoft Unity) and when we changed some information on some repositories and call the function ‘Commit’ of unit of work then we see a DbTransaction at the InvalidateSets method. So after calling this commit function IOC creates a transaction behind and after that EFCache doesnt invalidate the previously cached value.
I hope i could explain well,
Hope to find out the right solution,
Hakan
LikeLike