Provisioning the Content Search web part with a custom Display Template

I hit an interesting issue recently, around provisioning the Content Search web part onto a page with the AllUsersWebPart element. I should say upfront that I’m not (currently) providing an awesome “here’s some code” solution to the problem, but will suggest approaches that should work fine. Mainly I want to raise awareness of the problem at this stage – if you hit it, you’re not going crazy.

Articles in this series:

• Using the Content Search web part (and understanding SP2013 search)
• Displaying the right data in the Content Search web part in SharePoint 2013
• Provisioning the Content Search web part with a custom Display Template [this article]

If you’ve worked with the Content Search web part in SharePoint 2013, you’ll know that you can provide custom JavaScript templates (known as Display Templates) to control how items are displayed. Effectively you’re specifying the HTML which is output. Once the file is in the right place, you can select it in the ‘Display Templates’ section of the Content Search web part properties, like this:

These Display Template files are expected to live in the Master Page Gallery, and if you browse there you can see all the out-of-the-box Display Templates for various bits of SharePoint (Content Search web part, search hover panel, refinement panels etc.). There’s actually a significant difference between sites that have the publishing feature activated, and sites which do not. Specifically, if you browse in a publishing site, you’ll see two sets of files – HTML and JavaScript:

..but if you look at a site without publishing, then you only see JavaScript files:

This is one aspect of the “Design Manager” support provided in publishing sites – you get a HTML and a JS file for each template, and you can simply edit HTML file. When you do this, the associated JavaScript display template (which is going to output data in the HTML format you provided) is *generated* from your HTML file. This happens through an event receiver on the Master Page Gallery, and happens fast enough for iterative development to work out fine – as a reasonably hardcore dev, doing this bit in SharePoint Designer works fine for me. However, another option (provided by this support for HTML editing) is using a HTML editor with a design surface (Dreamweaver/Expression Web/Allaire Homesite (!)/whatever) – you can map a drive in Windows Explorer to the Master Page Gallery, to facilitate saving directly to your SharePoint dev environment from your editor.

Provisioning in a Feature - deploy the JavaScript and/or the HTML file?

So, before we get to my actual issue, we have an interesting decision point if we are working against a site with publishing in the "developer/Visual Studio” way. Do we:

• Provision the HTML file in a Feature, and then “touch” the file somehow (e.g. custom event receiver) to ensure the JavaScript file gets generated?
• Provision the JavaScript file only

After playing around with both options, my strong view on this is that you should only provision the JavaScript file. By all means, use the HTML support in the core dev phase to obtain the JavaScript file – but when you’re ready to take this to other environments, package it up in Visual Studio, and include it in your farm/sandbox WSP (N.B. none of this really applies to apps). I say this because:

• You are NOT going to need those HTML files in production – they are not used at runtime. The whole thing is purely to support the design phase!
• If you need to go back and make changes, then you can either edit the JavaScript file directly (frankly no sweat for most devs) or fire up a publishing site somewhere, use the HTML support, and then drop it back into Visual Studio when done

Touching a file –> to trigger an event receiver –> to generate a file –> which you already have in the development environment == no sense to me :) However, I would say that if you *are* working in a publishing site (e.g. in SPD), always edit the HTML file if it exists – otherwise of course, your changes to the .js file will be overwritten whenever that event receiver executes.

I use XML like the following for this – I found that the key property, ‘HtmlDesignAssociated’ defaults to 0 (zero) but when deploying the .js file only, I like to set it to zero explicitly:

With that out of the way, let’s discuss the problem I see when extending the above - to have a Content Search web part automatically added to a page with AllUsersWebPart element.

Problem - custom Display Template not linked when provisioning Content Search web part using AllUsersWebPart

[Update April 30, 2013 – Ivan Neganov has found a solution to what follows, see the comments at the bottom of the article!]

Hopefully that title was descriptive enough. So the goal is to provision the Display Template in a Feature, but also make use of it – i.e. provision some kind of page which has a pre-configured Content Search web part on it, where the configuration ‘points to’ a custom Display Template. I think this scenario will be pretty common – you’d use it in many places, such as rolling up some news articles with a particular look and feel. The process I used is what any experienced SharePoint developer would probably expect:

1. Configure the CSWP with the right settings on a page
2. Export the web part to get the XML
3. Drop into an AllUsersWebPart element in a Feature
4. Check the settings all look good
5. Test

The key bit of configuration we’re interested in is this line, but it all looks good at this stage:

<property name="ItemTemplateId" type="string">~sitecollection/_catalogs/masterpage/Display Templates/Content Web Parts/Item_Picture3Lines_COB_blue.js</property>

The overall web part XML is as follows (the “ItemTemplateId” is the last property to be listed):

The result is that seemingly all aspects of CSWP configuration take effect, except the custom Display TemplateSo. all other properties are applied, so the search query works fine and shows the correct items - but the look and feel is the out-of-the-box branding supplied by the ‘Item_Picture3Lines.js’ template (i.e. not my ‘Item_Picture3Lines_COB_blue.js’ file). What’s weird is that if you then edit the page, you find that your custom template is indeed available for selection (see first image in this article), and selecting it works just fine and the branding gets applied. At first I assumed this was some sort of issue with the format, or a ‘provisioning sequence’ issue, and played around with some ideas (breaking things into multiple Features with dependencies etc.) – none worked, so it was time to break out Reflector Pro.

After a pretty forensic ‘debugging SP2013 code’ investigation (I had a colleague trace my steps independently, thanks Hrayr!), this looks like a possible bug to me. We also tried environments before and after the March 2013 Public Update, but saw the same results. Essentially the provisioning process eventually calls into some code in the ContentBySearchWebPart class itself, and in particular a method called IsTemplateLocationValid. We see that one check appears to expect the path to start with “~sitecollection/_catalogs/masterpage” (this has been stripped off by earlier SharePoint code, despite the value being specified that way in the XML), so that one fails (click to see larger image – and sorry if you’re not into the Visual Studio ‘Dark’ theme!):

..and the next check expects SPContext to be present. However, it isn’t! I’m not sure why, given the Feature is being activated through the browser in the SharePoint ‘Activate Site Features’ page in my case, but there is definitely no SPContext. So between these two checks, I’d expect one to succeed. I can’t help but think that there is currently a minor bug in SharePoint 2013 - where SPContext is expected to be present in this provisioning process, but for whatever reason it isn’t. I say this partly because I notice that even for out-of-the-box templates, the value comes through stripped of the “~sitecollection” token:

I also tried using *just* the filename:

<property name="ItemTemplateId" type="string">Item_Picture3Lines_COB_blue.js</property>

..but then the page does not load (blank screen, HTML not output properly) and ULS shows the following runtime error:

Application error when access /SitePages/CSWP_Provisioned.aspx, Error=Cannot make a cache safe URL for "item_picture3lines_cob_provisioned.js", file not found. Please verify that the file exists under the layouts directory. 
at Microsoft.SharePoint.Utilities.SPUtility.MakeBrowserCacheSafeLayoutsUrl(String name, Boolean localizable, Int32 desiredVersion)    
…

Possible solutions

Broadly I think these are the options:

1. Manually fix-up any pages where you are adding the Content Search web part using AllUsersWebPart – i.e. edit the page by hand, select the Display Template in the web part properties, then save the page.
2. On-premises only - run some server-side code (e.g. in a Feature receiver) to fix things up. Use SPLimitedWebPartManager to update the web part properties.
3. For Office 365 – run some client-side code (JSOM or CSOM) to fix up the web part.

I don’t think any of the code would be hugely challenging to write – I might do it soon if no-one else does :) But for now, I just wanted to highlight the issue. Do you see a different behavior? Please leave a comment and let me know if so!

Slide decks for “SharePoint-hosted apps deep-dive” and “Customising SP2013 with JavaScript” published

A brief post to mention that I’ve now published my slide decks for the two talks I delivered at the SharePoint Evolutions Conference in April 2013. I had a great time at the conference (as always!), and it was awesome to have so many people come to the sessions – thanks to all who came, I appreciate it. Just for a bit of atmosphere, here’s the attendee perspective in one of these conference talks (not mine – I was at the back, taking the photo):

As always, technical talks are made richer by the demos and the decks alone do not convey this – I’ve added some screenshots for one talk (the JavaScript session), but I haven’t for the apps talk as I’m not sure they would add as much. Anyway, here’s the summary and links:

Customizing the SharePoint 2013 user interface with JavaScript

Covers several approaches for user interface customization in SP2013 - using JSLink to customize a list and/or view, creating custom Display Templates for the Content Search web part, and different approaches for customizing the search hover panel.

Slideshare link - http://www.slideshare.net/chrisobrien/customizing-the-sharepoint-2013-user-interface-with-javascript-chris-obrien

Deep dive into SharePoint 2013 hosted apps

Covers key aspects of SharePoint 2013 apps, with a focus on SharePoint-hosted apps. Includes detail on app parts, using web parts within an app, configuring SSL, troubleshooting apps and possible reasons to move away from a SharePoint-hosted app to a cloud app. Also covers "high-privilege" apps which provision to the host web.

Slideshare link - http://www.slideshare.net/chrisobrien/deep-dive-into-sharepoint-2013-hosted-apps-chris-obrien

 

Other notes

In upcoming posts I’ll be publishing more detail on this content, including the source code I used for my demos. Stay tuned.

Rolling out SharePoint 2013 apps to the enterprise - tenant scope and PowerShell installs

Let’s say you’ve decided to make some custom functionality available to your users as a SharePoint 2013 app. You’ve decided that this will be published to the appropriate internal SharePoint App Catalog (as opposed to being available in the public SharePoint Store). In a large company, it’s probably not realistic to expect all users to install such internal apps, even if we want to make the functionality available to everyone.

The answer *might* be to use one of the options for deploying the app at “tenant scope”. This works for both on-premises deployments and Office 365/SharePoint Online. This article looks at these options, but just as a reminder of where we are in the overall series, here's the table of contents:

1. SharePoint 2013 apps – architecture, capability and UX considerations
2. Getting started – creating lists, content types, fields etc. within a SharePoint app (provisioning)
3. Working with data in the app web, and why you should
4. Access end-user data (in the host web) from a SharePoint 2013 app
5. Rolling out SharePoint 2013 apps to the enterprise - tenant scope and PowerShell installs [this article]
6. Azure is the new SharePoint ‘_layouts’ directory
7. “Host web apps” – provisioning files (e.g. master pages) to the host web
8. “Host web apps” – provisioning fields and content types
9. Deploying SP2013 provider-hosted apps/Remote Event Receivers to Azure Websites (for Office 365 apps)
10. Working with web parts within a SharePoint app

Microsoft even provide a nice admin interface for tenant-scope installs, which offers the following deployment options:

• Deployment to named site collections
• Deployment to all sites under a particular managed path
• Deployment to all sites created from a certain site template

I said *might*. When deploying apps in this way, there is a crucial detail to be aware of – something which could either be highly-desirable or a deal-breaker. Essentially, all installed instances of the app *share* a single app web. This means any lists or libraries you provision as part of your app, are shared amongst all the webs where the app is installed. As I say, this could be exactly how you want your app to operate, or completely against it’s design.

The process for using this model is as follows:

1. Upload app to App Catalog.
2. Install the app to the App Catalog site collection – yes, for this scenario we actually have to install the app there (for reasons which will become clear later).
3. Go to the Site Contents page and find the app. On the callout menu here (and only here), you’ll have an action labelled ‘DEPLOYMENT’:
   
    
4. Click the ‘DEPLOYMENT’ button to go to the admin page.

On this page, I see the 3 options for deployment – individual site collections:

..or deploying to all sites under a Managed Path, or all sites created from a certain template:

Once you’ve selected appropriate options and hit OK, after a few minutes time you’ll find your app has landed in the target locations:

As you can probably guess, it’s a timer job which performs the deployment. This is the App Installation Service timer job, set to run every 5 mins by default.

Important aspects of tenant-scoped app installations

As I highlighted before, all app instances share a single app web. The way this works is that the instance which is installed to the App Catalog site is used for all instances – effectively the user is always redirected to this URL on the app domain, regardless of which site they entered the app from:

There are also important by-products of this:

• Users in sites where the app is installed cannot remove the app – regardless of their permissions there
  • This makes sense because they would effectively be uninstalling the single instance used by everyone
• Individual “usages” of the app do not get reported by Get-SPAppInstance – only the instance in the App Catalog site gets returned (N.B. applicable to on-premises SP2013 only)

Any changes to the app (e.g. app upgrades, app removal) need to be managed at the instance which is installed to the App Catalog site.

But I want something else - to roll out my app in bulk, but it NOT to be “single instance”

Then in this case, PowerShell is your friend – this is a two step process to install a new app:

• Use Import-SPAppPackage to initially install the app to the site collection, then..
• Use Install-SPApp to install to individual webs

For a full list of cmdlets around SharePoint 2013 apps, see App Management Service cmdlets in SharePoint 2013 on TechNet. Corey Roth also has a post on some of these cmdlets.

If you’re on Office 365 (with it’s limited set of PowerShell cmdlets, as of April 2013), you’ll need to accomplish the same thing using one of the client APIs. I’m starting to form the view that .NET CSOM code “wrapped” with PowerShell is a good way to do this – since it can then be integrated with TFS automated builds etc.

Happy app installing!