Provisioning Managed Metadata fields in Office 365 - Part 1: dealing with multiple environments

Managed Metadata fields have always been slightly painful for SharePoint developers, but if you did any kind of site templating or Feature development in SharePoint 2010, chances are that you did some research, read some blog articles and came to understand the solution. Here I’d like to talk about it for Office 365, and show a little customization I built to help with the process we currently use. This article became long, so I split it over two articles:

1. Provisioning Managed Metadata fields in Office 365 – dealing with multiple environments [this article]
2. Provisioning Managed Metadata fields in Office 365 – building WSP packages for a specific environment/tenancy

So, back on the SharePoint 2010 situation - the deal is that some Managed Metadata details change between SharePoint environments – so unlike other fields, the same provisioning XML could not be used across dev/test/UAT/production. To be specific, the IDs of the Term Store, Group and Term Set all change between environments. As a reminder, broadly the solution was to:

• Use XML to define the “static” details of the field
• Use server-side code to “finish the job” – i.e. use the API to ask SharePoint what the IDs are (for the current environment), and then update the field config with that value

Without the 2nd step, the field will be created but will be broken – it will be grayed out and users cannot use it (SP2010 example shown here):

Posts by Ari Bakker (whose image I’m using above, thanks Ari), Wictor Wilen and Andrew Connell were popular in discussing the steps to solve this.

It’s the same deal in SharePoint 2013/Office 365 – but it turns out we need different techniques.

Why the approach doesn’t work for Office 365/SharePoint Online

Well firstly, code in sandboxed solutions is deprecated. Full-stop. [RELATED - did you hear? Microsoft are starting to clarify that sandboxed solutions without code aren’t really deprecated after all (and hopefully MSDN will reflect this soon), but CODE in sandboxed solutions is deprecated and could be phased out in future versions. Clearly this is a very important distinction.]

But even if we were happy to use sandboxed code - in Office 365/SharePoint Online, we cannot use the Microsoft.SharePoint.Taxonomy namespace in server-side code anyway – the net result is that we are unable to “finish the job” in this way to ensure the field is correctly bound to the Term Store. This is a problem! Even worse, whilst it is possible in the CSOM API to bind the field, having this execute in the provisioning process (e.g. as a site is being created from the template) is challenging, maybe impossible. Maybe you could come up with some imaginative hack, but that’s probably what it would be. And what happens if this remote code (e.g. a Remote Event Receiver) fails?

Possible solutions

A colleague of mine, Luis Mañez, did some great research – I’ll give you a quick summary here, but I strongly recommend reading his article - Deploying Managed Metadata Fields declaratively in SharePoint 2013 Online (Office 365). Here’s a summary:

In fact, it IS possible to provision Managed Metadata fields without any code, if you are willing to accept a big trade-off – you can declaratively specify the key details (such as the Term Store ID (also known as the SspId), the Group ID, the Term Set ID etc.) into your XML field definitions. Wictor alluded to this possibility in his post. But remember, these details change between environments!

So in other words, the trade-off is that you would need to rebuild your WSPs for each environment.

This is tricky for us, because on this project we choose to run multiple Office 365 tenancies, for development/test/production (something I’ll talk about in future posts) – just like a traditional mature process. So at first we said “No way! That’s against all of our ALM principles!  The exact same packages MUST move between the environments!”. But then we rationally looked at the alternatives we could see:

• Option 1 - Some elaborate “remote code” solution, perhaps involving code running separately AFTER the site has been provisioned. Until this code executed, it would not be possible to upload documents to libraries with MM fields within the sites (and similarly if this remote call actually fails for any reason,  these libraries would not function correctly until an administrator intervenes).
• Option 2 - The client needs to fix-up any Managed Metadata fields manually – across all 5000 sites we were expecting. In every list and library. Knowing that some lists/libraries have up to 5 such fields. Yeah….

Since neither of these was attractive, we continued looking at this idea of a 100% declarative definition of Managed Metadata fields. And then we realized that..

..if you do things in a certain way, you can get to the point where ONLY THE TERM STORE ID (SspId) CHANGES BETWEEN ENVIRONMENTS. That’s kinda interesting. It means that just one find/replace operation is all that’s needed – assuming you’re happy to accept the overall trade-off. Of course, having to replace the SspId is still sub-optimal, error-prone and less than awesome. But maybe we could work on that too – and that’s what these posts are really about - to show a Visual Studio customization we made to simplify this process, and make it less prone to human-error. If you want to skip ahead to this, see Provisioning Managed Metadata fields in Office 365 – Part 2: building WSP packages for a specific environment/tenancy. 

But first, let’s talk about that “if you do things in a certain way” thing (which means that the only the SspId changes between environments)..

The full recipe for Managed Metadata fields

Taking a step back for a second, if you are in “development mode” (e.g. creating a template for SharePoint sites), then successful provisioning actually involves more than just provisioning the field itself in a certain way. Effectively you should seek to provision both the Term Sets AND the fields. Do not allow administrators to create new Term Sets in the admin interface. This is because:

• This way, you can control the ID of all your Term Sets – rather than let SharePoint generate that GUID
• Because this is a static “known” ID, we can reference it elsewhere

Here’s what needs to happen:

• Term Sets are provisioned into the Term Store with “known” IDs
  • We create Term Sets (often with some default Terms) in Office 365 using PowerShell and CSOM, instead of through the admin interface. In the article I referenced earlier, Luis provides you with the core PowerShell to do “taxonomy provisioning” in this way. In our project, we basically hook this up to an XML file which defines the desired Term Store structure (e.g. Groups, Term Sets etc.) – and this XML file defines the GUIDs of such objects; WE generated them – not SharePoint/Office 365
• The “known IDs” are then used in the XML definition of the fields
  • The code sample below is an example of a Managed Metadata field being provisioned the 100% declarative way. Notice all the properties being specified in the ‘Customization’ section (something a field using the combined declarative + code approach does not have):
    
    ** N.B. My newer code samples do not show in RSS Readers - click here for full article **

If you do this, your Managed Metadata fields will work just fine:

Great. It’s certainly very valuable to know this is possible for Office 365. So now we have things so that only the SspId value needs to change between environments. But that’s still a nasty find/replace operation – how could we make this situation better?

I describe the mechanism we use in the next post - Provisioning Managed Metadata fields in Office 365 – Part 2: building WSP packages for a specific environment/tenancy.