Editing .cmp files to fix lookup field issues

I ran into an interesting (well, it’s all relative) content deployment issue the other day, which I’m pretty sure will apply to both SharePoint 2007 and SharePoint 2010. In preparation for some SharePoint training I was delivering at my current client, I wanted to move some real data from production into the training environment to make the training more realistic. To do this, I used my Content Deployment Wizard tool, which uses SharePoint’s Content Deployment API to export content into a .cmp file. (Quick background – the tool does exactly the same thing as ‘STSADM –o export’ and out-of-the-box content deployment, but allows more control. A .cmp file is actually just a renamed cab file i.e. a compressed collection of files, similar to a .wsp). However, when importing the .cmp file containing my sites/documents etc., the operation failed with the following error:

The element 'FieldTemplate' in namespace 'urn:deployment-manifest-schema' has invalid child element 'Field' in namespace 'http://schemas.microsoft.com/sharepoint/'. List of possible elements expected: 'Field' in namespace 'urn:deployment-manifest-schema'.

So clearly we have a problem with a field somewhere, and it’s an issue I was vaguely aware of – cross-web lookup fields deployed with a Feature break content deployment. Michael Nemtsev discusses the issue here, saying “There are several samples how to deploy lookup fields via feature (http://www.sharepointnutsandbolts.com/2007/04/feature-to-create-lookup-fields-on.html) but all of them are not suitable for the Content Deployment Jobs. Because you will get the exception...”

Oops that’s a link to an old article of mine. So effectively *my* code to create lookup fields doesn’t work with *my* content deployment tool – don’t you just love it when that happens?! However, I actually have a clear conscience because I know both utilities are doing valid things using only supported SharePoint APIs – this is simply one of those unfortunate SharePoint things. As Michael says, all of the cross-web lookup field samples would have this issue. So what can we do about it?

For fields yet to be created

In this scenario my recommendation would be to use the technique Michael suggests in his post, which is to strip out the extraneous namespace at the end of our code which creates the lookup.

For fields which are already in use (i.e. the problem I ran into)

If your lookup fields have already been deployed, then you have 2 options:

• develop and test a script to retrospectively find and fix the issue across your web/site collection/farm/whatever scope you need
• fix the issue in the .cmp file you were trying to import in the first place, so this particular import will succeed

Clearly your decision might depend on how much content deployment you want to do to the site/web/document library or list which has the problem. If you’re anticipating doing it all the time, you should fix the underlying issue. If, as in my scenario, you just need to get an ad-hoc import to succeed, here’s how..

Hacking the .cmp file

The process is effectively to fix-up the .cmp file, then rebuild it with the updated files. I noticed an unanswered question on Stack Overflow about this process, so clearly it’s something that can occasionally arise. Of course even in these WSPBuilder/SP2010 tools days, all SharePoint devs should know you can use makecab.exe with a .ddf file to build a cab file – but what happens when you have hundreds of files? That’s hundreds of lines you’ll need in your .ddf file? Certainly you could write some code to generate it for you, but chances are you’re looking for a quick solution. 

The first process I came up with was:

1. Rename .cmp file to .cab, extract files to other directory.
2. Fix-up files (more detail shortly).
3. Use WSPBuilder to generate the .ddf file from files on filesystem – edit this to ensure paths are correct.
4. Use makecab.exe + the generated .ddf file to build the .cab file.
5. Rename extension to .cmp.

However, a slightly better way is to use Cab File Maker, like this:

1. Rename .cmp file to .cab, extract files to other directory.
2. Fix-up files – to do this, edit manifest.xml, remove all instances of following string - “xmlns=http://schemas.microsoft.com/sharepoint/”
3. Open Cab File Maker 2.0, drag files in and set options for where to generate ddf file and cmp file, like so:
   
   
   
4. Voila – your cmp file is now ready to go and should import successfully.