In an earlier blog post in December, I described one use case for the new Rollup Fields feature introduced in Microsoft Dynamics CRM 2015. This example involved rolling up data from email events tracked via ClickDimensions and summarizing this on the contact’s form, so you’ll want to check out the steps listed there if you don’t have any hands-on experience about this new feature yet. In this post I’m going to dig deeper into the details about how Rollup Fields actually work behind the scenes and what limitations you should be aware of when considering whether they are the right tool for the job in your own use cases.
The first thing you need to understand about Rollup Fields is that they are not updated in real time. If you’re familiar with the difference between the real time workflows introduced in CRM 2013 and the asynchronous versions that were available in earlier versions, then this is something a bit like that, but not quite. As you might know, the traditional background workflows were triggered by an event that took place on a CRM record and the resulting workflow instance was scheduled to be executed by the asynchronous process running on the CRM server at the earliest possible date (depending on the overall workload on the server). Whereas this usually meant a delay of perhaps a minute or two at most, the new Rollup Fields are even further from real time than this.
As we saw in my earlier post, when you create a new Rollup Field, a new mass calculation job will be created for the field in question. This will be scheduled 12 hours into the future, based on the assumption that this will most likely fall outside the office hours when actual CRM end users are working with the system. (Because us CRM customizers or system admins never work during the night, right? Yeah, what a funny assumption that is, but anyway…) The reason for such precaution is that the very first calculation job will have to populate each and every record that exists for that entity, which could be up into the millions, depending on what type of data you manage in your CRM.
So, does this mean the Rollup Fields only get updated once per day, during that nocturnal schedule? No, actually they get updated once every hour. If you go to the Settings – System Jobs menu you’ll see that there are jobs of type “calculate rollup field” type running for each of your entities that have one or more Rollup Fields defined for them. They are not scheduled to start at exactly the same time, but they all run at one hour intervals. Another thing worth noting in the Rollup Field implementation architecture is that these calculation jobs are only applied to records that were created, updated or deleted after the last job finished. No point in processing a million records if only a handful of them could possibly have new values to be calculated, right? This is why the initial rollup and the recurring rollup requests are handled by different system jobs in the CRM platform.
Rollups and Workflows
Now that we know the Rollup Fields may not show a current values in the UI for quite some time, the next logical question to ask is: anything we can do to speed the calculations up? As an end user, you could go and look at any Rollup Fields that have been added onto an entity form which you have the necessary rights to view, then hover over the field and click the “recycle” icon to force the recalculation of the Rollup Field value. As a developer, you also have the option to force a Rollup Field to be recalculated on demand via a plugin, by using the CalculateRollupField message. As a system customizer… Well, there’s not much you can do, at least in the CRM 2015 version. [Read more…]