CRM 2011 to CRM 2016 Migration

Question

I am planning to upgrade my CRM (On premises) 2011 to CRM 2016 (On premises). And now I am looking for the best way to migrate the data. By the way, there're a lot of custom data (entities, fields, WFs). Microsoft recommends a step-by-step upgrade 2011->2013->2015->2016 (because db strucutre changed significantly, as they say), but this is not the best way for me. I want to do clean installation first and the move the data from 2011 to 2016. The solution I've came to is to investigate the new structure then write custom SQL scripts, that will do the work. Is there any out-of-the-box way?

Another question is about CRM editions. Microsoft offers Dynamics CRM 365 (On premises) and Dynamics CRM 2016 (On premises). What is the difference? As I could found, 365 is something like a subscription license, when 2016 is one-time payment?

TL;DR:

1. Best way to upgrade from 2011 to 2016 CRM, best practices.
2. Best way to migrate data from 2011 to clean 2016 CRM.
3. Difference between CRM 2016 and CRM 365 (On premises, both)

Thank you so much for upcoming answers.

Alurd

What's wrong with my question, that it is being minused? How can I make it better? I have been searching for the information in the Internet pretty long but didn't find anything that corresponding my needs, if the apparent subject broadness is a reason. – Alurd

I dont know whats wrong with it, but we are in the same boat, and this is a good question for us also. Ill upvote it. – Formyl

Answer 1

I did a lot of migrations from CRM 2011 to CRM 201X including CRM 2016 and Dynamics365. Here are my gathered suggestions/thoughts on the problems

1) Basically the organization migration that is handled by CRM Deployment Manager does its job pretty well. I usually create a staging environments on separate servers (so CRM 2013 -> CRM 2015 -> CRM 2016) make a full copy of the database, restore it on the next server and import organization using Deployment Manager. As for the customizations - it depends how old is the CRM, how many customizations it has and if it was upgraded from CRM 4.0. If the last one is true and the amount of customizations is huge - in most cases i remove all Javascripts and plugins and write them all from scratch. Although after successfull migration to CRM 2011 after Rollup 12 may make all the scripts and plugins working properly, most of the logic from CRM 4.0 can usually be achieved using some new features of CRM 2016 (not only Business Rules, which I don't like personally, but mostly Calculated Fields or Rollup Fields) and it makes no sense to keep all that stuff in some JavaScript or plugins. If the system origin is CRM 2011 then I would only audit the functionalities that can be made simpler, but usually not rewrite the whole thing, just make some adjustments. If the scripts contain OrganizationData service calls, I usually rewrite them to use webAPI - OrganizationData is going to be removed from CRM soon, so this is a good thing to do.

Of course after importing the organization to the destination environment I would apply all the modified customizations (that were made on a separate DEV environment and tested thoroughly).

2) I always use Kingswaysoft SSIS Connector for such purposes. I tested all the other tools and this is simply the most flexible, because it embraces all the power of SSIS packages and simply provides you an easy to use connector. Of course this is my personal preference, all the tools should do their job in the end.

When I migrate the data between two on-premise environments (usually in the same domain) I don't bother with migrating special fields like createdby, modifiedby, statuscode, statusreason etc. I focus only on record Ids. When I have all records created I simply update all the fields that are problematic using T-SQL scripts, because that's much faster. Of course you can't do that when migrating to Online, so this migration will take much more time (and you will not be able to migrate for example modifiedon field)

The simplified flow I always use is like that

1. Create all the records without proper statuscodes (use default) and lookup values (because most likely you will not have the related record). If it's online then focus on special fields that cannot be changed later - createdon, createdby
2. Update all lookups for all records (because after 1. you have all records in CRM)
3. Update all statuses, if it's on-premise run all the custom scripts copying values
4. Apply the customizations

3) Quite good answers were already given, Dynamics365 is simply an update for CRM 2016 (that's why it's version 8.2 not 9.0), so from CRM perspective there are not many changes. The biggest change that can be taken into consideration during upgrade is editable grid - it's very easy to prepare a view that can be editable (although it has it's drawback like no in-line record creation) which can simplify some scenarios (or will allow you to drop some custom solutions). Business Process flows are better handled, because they have separate database table, where all the important information about the process state is kept (that also introduces new SDK access to this processes) Other things are simply cosmetics, mostly for the business guys, not for developers.

4) As for the bounty question - all the apps can still be using plain old Organization.svc (by getting IOrganizationService object), which is a SOAP endpoint to CRM. There are no plans to remove this endpoint for now, so I cannot see any advantage in rewriting this application to be using webAPI (of course that's possible). XrmServiceContext is simply a wrapper over the IOrganizationService, which allows you to use it in more "Unit-of-work" way - certainly useful for retrieving data, but I don't like it as it comes to all other CRUD operations. But anyway - it is still only a wrapper, so the only thing that matters is how you obtain IOrganizationService. Back in CRM 2011 most likely you did that using OrganizatoinServiceProxy class which you instantiated using proper credentials data (like I explained here: https://mcmap.net/q/1013630/-connecting-to-crm-online-through-crm-365-plugin). Currently the suggested approach is to use Microsoft.Xrm.Tooling.Connector assembly (simply obtain it from nuget - Microsoft.CrmSdk.XrmTooling.CoreAssembly) and using CrmServiceClient together with connection string (see: https://msdn.microsoft.com/en-us/library/jj602970.aspx). This will allow you to get your IOrganizationService which you can wrap in xrmservicecontext or whatever you want. This approach is suggested, because most likely in the future, this package will start using webAPI instead of Organization.svc endpoint, so if Microsoft decides to remove or deprecate this service, your application will be still working without a problem.

Rocaille

Answer 2

As the question is very broad, the answers are precise to the point and do no go into depth of the subject.

Upgrade from CRM 20XX to 20YY

---

Although you could do an in-place upgrade (existing CRM Server + existing SQL database), which in essence is almost as if you were applying a cumulative update, or provision a new server and use the existing SQL server, the best and the Microsoft's recommended way to upgrade is to go about doing a Migration Upgrade (new CRM server + new SQL server).

Steps to migrate (short story):

1. Provision a new CRM instance with a new SQL Server instance and an SSRS instance (if applicable)
2. Apply any product updates/roll ups/cumulative updates. Backup the existing CRM database.
3. Restore the database to the new SQL Server instance provisioned.
4. Use the deployment manager and start the Import Organization process pointing to the restored database, which will start the upgrade process.

Long story would involve upgrading plugins with the latest version of the SDK (which would involve un-registering them, upgrading them and re-registering all the plugins and steps), setting up authentication, SPNs etc. I would recommend giving the article linked above a good read.

Note that the upgrade has to be incremental (e.g. 2011 - 2013 - 2015 - 2016, applicable CUs in between if any).

Best way to migrate data from 20XX to 20YY CRM

---

You do not need to migrate data from 20XX to 20YY if you go by the migration upgrade route or for that matter any supported upgrade path. There is a thought process that an upgrade would inadvertently need a data migration, but in reality it does not. Unless you are moving data from another system or changing/cleaning up your existing CRM data structure (consolidating entities, moving notes around etc.) you most probably do not need any migration.

Assuming that you need to do one of the above, some of the most used integration tools are Scribe for Microsoft Dynamics CRM or KingswaySoft for Dynamics CRM. My favourite being KingswaySoft for being easily extendable and also the pricing model (you could essentially buy a 3 month license and get your migration done as migrations are a one time operation).

Difference between CRM On-Prem and CRM Online

---

Apart from the whole cloud, licensing model differences between the two, there are still some features which are online exclusive (at least for the moment or till the next on-prem update).

Choosing between the two, in my experience working with clients who were both online and on-prem essentially boils down to:

1. Upfront cost, on-going maintenance.
2. Existing infrastructure (if a company is already on cloud with office 365, they would most probably end up going with CRM online).
3. Control over upgrades, databases. On-Prem customers are usually the ones which like more control over the databases, servers, when to update/upgrade.
4. Features that are online only (although Microsoft does roll most of them out to on-premise installations too, but they tend come much slower than they do for online instances, usually 3-6 months). Some features such as inside view and social listening are online exclusive for now.

Dynamics 365:

---

Dynamics 365 is a combination of ERP (GP, NAV, AX), Dynamics CRM and some integration extension tools like Parature. From a CRM standpoint it is going to be no different than CRM 2016 online. Just the back end data structure which probably will be more one size fits all model. Although more details are yet to come out, from a functionality point of view it won't be a whole new product. They might come up with a couple of new functionalities like they do with every major release, but CRM as we know is still going to be the same predominantly.

Decontrol

I thank you for such an answer. Other than that, which I will consider a bit later, I want to clarify my third question, maybe I did not precisely expressed what I want to know: the question was about differences not between Online and OnPremises, but between CRM v2016 and CRM v365 (sometimes I see that Microsoft provides info about 365 and sometimes it is written like 2016, but because all this 2 features can be installed on premises, I don't think that they are the same which just differs in licensing method). – Alurd

@R.Matveev, I have updated the answer, Dynamics 365 is not going to change CRM as we know it. Just MS bringing different products under one umbrella (more unified services). – Decontrol

Answer 3

Best way to migrate data from 2011 to clean 2016 CRM.

I was in the same position as you a couple of Months ago for a project I'm managing (delivery this weekend).

After looking a bit into this solution, I ended up with the following conclusion: The way Microsoft recommends is hard to implement, time consuming and doesn't diminish the risk what so ever. Note that in my case we migrated from On Premise to Online which is harder to achieve than On Premise to On Premise.

In our case, we went for Mapping of the Old vs New database and migrated everything to the CRM 2016 Online via ETL jobs configured by our integrator. I believe this is the best way since we have total control on what is going on and if there is a field missing (example the Fax for Leads was not Migrated) you can easiliy migrate just that field.

If you go for the 2011==> 2013 ==> 2015 ==> 2016 road, you'll have triple work since you'll have to cover each gap between versions (example there are 4 fields for Phone in a Lead in 2011 and only 3 in 2016) and will have to come up with solutions three times instead of just once.

Difference between CRM 2016 and CRM 365 (On premises, both)

None for the CRM. Unless I'm mistaken, the 365 version with Server Side Synchronization will allow you to use the 365 plugin for Outlook which has many more functionalities than the other version (which is pretty much similar to the 2011 one). Edit: I was mistaken. Dynamics 365 is the new name of CRM 2016 (following Microsoft's new 365 branding which was also applied to Dynamics AX). Your system should already be updated and besides the name change (and the brand new Outlook add-on I haven't tested yet), you have Voice of the Customer integrated into the system instead of coming as a Solution. I think there are some other changes as well. Note that you'll need to upgrade your solutions such as "Field Service" manually from the Administration interface.

Hydroscope

In my case, as we did an upgrade already, we first set up CRM 2013 on CRM 2011 updating it. Almost everything was perfect with data. Then migration from 2013 to 2015 and to 2016 had some pitfalls, so we did as you did: we created mappings for tables in db we interested in then just installed 2016 and run our script to move data. Migrating from 2011 to 2013 and then creating custom script was much more simple than creating a script to migrate data from 2011 to 2016 directly. – Alurd

May I ask how did you manage to set up a CRM 2013 environment? From what I had seen, licenses are no longer being distributed? – Hydroscope

You can use trial license key to setup a new environment or upgrade existing to migrate your data. Trial key gives you 90 days (as I remember). – Alurd

Answer 4

Migration plan:

As the question was nominated for bounty I will share the steps I had taken during the upgrade from 2011 to 2016 (although I am not the official source, it may be helpful).

1. I had unregistered and deleted following customizations: javascript libraries, html and silverlight (xap) web-resources, plugins, solutions, workflows.
2. Then, I had made step-by-step upgrade of my database. I had 3 VMs with different CRM versions installed: 2013, 2015 and 2016. When I finally imported my organisation into CRM 2016 and done the checks — all data was accurately updated.
3. Then I refactored and updated my customizations (what were deleted in step 1). After all I tested them on the staging environment.

---

What about customizations upgrade:

1. 80% of js code was replaced by the new feature introduced in 2013 — Business Rules. They help with field-level security: if we need we can hide/disable a field basing on specific condition or the state of another field.
2. .net customizations: workflow activities, plugins, custom applications stay unchanged. All the code from 2011 version is supported in 2016 (but CRM 4 code is not supported).
3. 50% of workflows were implemented as plugins, as more rapid and robust solution.

---

Now to the question of the bounty:

Working with xrmservicecontext and organisationservice didn't change significantly.

Make sure that you have no references to CRM 4 dlls, then just replace all your CRM SDK references with their 2016 version (I recommend using nuget packages instead of direct assembly-referencing). Your code should compile without errors.

Alurd