Extending the web part framework - part 1

Today I want to show some of the interesting things we've been doing with web parts for one of our clients. There's quite a lot to talk about so it will be over two articles:

• Part 1 - background and implementation
• Part 2 - issues and resolutions

There are a couple of things in particular which I think are quite cool, as we've effectively combined classic WCM (publishing) site functionality with a customized implementation of the web part framework. The context is a fairly large roll-out to an enterprise client, but what we're rolling out is a centralized platform for 80-100 internet sites. The idea is that content authoring teams in 80-100 countries will take what we've delivered on MOSS to create their own sites - replacing the existing mish-mash of sites on different technologies with inconsistent branding/look and feel.

Clearly a key challenge here is satisfying the diverse needs of so many stakeholders. So a cornerstone of the platform is that sites can be tailored somewhat, so each country has some flexibility to communicate with their audience in the way they think is best. We effectively give the authors a set of page templates and building blocks, and a system which governs how the blocks can fit together so that the user experience will still 'make sense'. Needless to say, a lot of analysis and consideration has gone into this - both in terms of what functionality was needed but also user journeys and navigation through the site, and the experience architects on our side (LBi) played a vital role here. There are many aspects to the project I could zone in on, but since I want to focus on the implementation details here, I'll briefly list some of these building block requirements before showing how we did it.

Key requirements/challenges:

In order to create the different page types, we needed around 15 page layouts, including these:

• Home page
• Channel hub/Alternative hub/Sub home - these are different template options for '2nd and 3rd level' pages 
• Content page
• Product page
• Media release
• List - provides links to a series of related pages
• Etc.

And whilst some aspects of page functionality was 'fixed' on the template, there were many other items which were optional - these were to be added to pages by the authors, either in a 'web part' kind of way or perhaps something else. Some examples of these optional 'page modules' were:

• 'Hero' feature - used to highlight something on prominent pages with an image/flash/text
• Right-hand promo
• Content editor module - allows an author to enter arbitrary content, but for reasons which will become clearer we developed an interesting custom control which is kind of a cross between a publishing HtmlField and a Content Editor web part (covered later)
• Generic content module - rolls-up formatted content/links to a selected page
• List/tabbed list - provides links to a series of related pages
• Dynamic share price - displays latest stock price based on web service call
• Product selector - using AJAX cascading dropdowns to filter products
• Etc.

Although there were lots of other challenges (such as multi-lingual content, packaging/documenting every deployment aspect so the hosting company could deploy etc.!), I felt that building the 'framework' could be more challenging than individual functionality bits. To help frame what you'll read next, some initial questions we had for the implementation were:

• How do these optional bits of functionality get added to the page? As web parts, or something else?
• How do we get accessibility-compliance if web parts?
• How do we provide configuration if not web parts?
• How do we restrict which modules can be used where (as per the specification)?
• Since we're in a 'flexible' publishing site, how do we determine which fields are needed on the content types? Does each content type need to have all the possible fields the author might choose to add?
• If we are working with publishing controls, how would we bind the dynamically added control to the 'back-end' publishing field on the content type?

The implementation

As well as the optional page modules, most of the templates had a classic set of publishing fields. After looking at custom approaches, we concluded the web part framework had a lot going for it for the optional stuff - clearly we could avoid building a user interface to pick the module from a list/add to page/allow configuration of properties specific to the module, and also get drag and drop (amongst other things) as an added bonus. The concept of web part zones - as a container where one or more modules could be placed - was also important to our page structure.

Another challenge for the optional modules was where to store the data. If they were publishing fields, we would need every possible module to have a corresponding field on every possible content type, and this was pretty impractical when looking at the spec. Web parts, of course, use a different model and the framework takes care of data storage regardless of how many controls are on the page.

On the downside, a key thing to remember with web parts in publishing pages is that web part data is (by definition) not stored in publishing fields, and therefore isn't versioned in the same way. After discussing with the client, in our case this proved to not have as big an impact as we initially thought, due to the split and nature of what content would be stored in publishing fields vs. what would be web parts. So, having the client's acceptance of this trade-off, we went with web parts and came up with these solution elements:

1. Module matrix
   
   This comprised two SharePoint lists which contained the 'rules matrix', to enforce the design team's specification of what functionality could be used on which page type. Effectively the data provides the mapping of modules and page layouts. Being list data, it meant that it could be easily updated by the central team if a policy change was required. This data was consumed by our base web part (point 4).
   
   
2. SmartPart-like approach, but with web part properties
   
   We wanted the actual functionality of our web parts to be implemented in user controls, for the typical reason of avoiding building HTML in C# code (wrong on so many levels!). This is obviously what the SmartPart does using LoadControl(), but we had the additional requirement of needing to pass web part property values to our user controls - this meant we could use the familiar 'tool part' interface (i.e. setting web part properties in the right-hand pane) for control configuration. 
   
   In our model, each user control has a corresponding 'wrapper' web part/tool part which understands which properties are required and how to build the properties UI. In the web part's OnInit() method, values are passed from the web part properties to the user control so that the latter is initialized ready to do it's processing.
   
   
3. Base web part/base tool part
   
   All our web parts/tool parts were derived from our custom classes which abstracted some responsibilities. Since we couldn't easily change the web part picker screen to only display appropriate web parts for the zone the author had selected, we built the check into the base web part - if an 'invalid' web part was added, the web part renders nothing in presentation mode but in edit mode we display a message to the author like this:
   
    
   
   Adding too many web parts to a zone (count determined in the module matrix data) would have a similar effect.
   
   
4. Combine interface of publishing field controls with web part storage
   
   Having decided to use web parts for our control architecture, we had one requirement for something similar to the standard Content Editor web part (CEWP). However, this control is pretty lame compared to the MOSS publishing HtmlField, and we quickly established our client needed more than the basic CEWP. So we combined the bits we wanted from both - the front end control used by the publishing field type (the RichHtmlField control), but the backing store of web part storage rather than a publishing field. This meant authors could add multiple instances of this optional module to their page (and get the nice editing experience), but because it's a web part we didn't need to worry about having a corresponding set of fields on each possible content type. In code/integration terms it's the same approach, but in the end we actually swapped the standard MOSS control for the control which fronts Telerik's RADEditor field since the client wanted to move to this: 
   
     
   
   Also note use of another control typically used with publishing fields here, the AssetUrlSelector - this provides the 'Browse...' button shown above, and can be used to provide a friendly way for an author to browse to a file.
   
   
5. Control adapter for WebPartZone for accessibility compliance
   
   Since web parts normally render with a stack of nested HTML tables which won't validate against AA, action needs to be taken to remedy this if accessibility is a design goal. However this isn't necessarily a big deal - the approach is that you 'correct' the HTML for the WebPartZone control in presentation mode only, thus leaving the tables intact in edit mode for all the web part editing framework stuff which needs to happen. You do lose the client-side Web Part Services Component (WPSC) API doing this, but we had no requirement for it anyway (I rarely see it used). I initially assumed I'd have to write a control adapter to do this, but I found that David Schneider has already done the job - this works fine. It's also possible the latest version of the AKS has one, can't remember if I checked.
   
   
6. Present only our web parts in the web part picker
   
   Since this is a highly bespoke WCM platform rather than a standard collaboration environment, we don't want to see any of the standard web parts in the picker for these sites. Two steps to this one:
   
   - delete all the .webpart files from the web part galleries in the sites (N.B. we used Kivati for rolling out such changes across all the site collections - more on this in the future). However, doing this will still leave you with ListView web parts for all the lists/libraries in your site, so you also need to..
   - ensure all your WebPartZone declarations have the little documented 'QuickAdd-ShowListsAndLibraries' property set to false:

   <WebPartPages:WebPartZone id="g_AB07678E486C46bc962DFC8446A6CD13" runat="server" title="Zone 1" QuickAdd-ShowListsAndLibraries="false" />

   
   
   
   Authors are then not confused by any standard web parts which aren't appropriate for our scenario:
   
   



7. Remove unnecessary options when editing web part properties (tool parts)
   
   Finally, we do a bit of work with the accompanying tool parts (for properties editing) for our web parts to avoid confusing our authors with options which won't take effect. As an example, for a web part which looks like this in presentation mode:
   
    
   

The tool part looks like this:



In case you're wondering what to look at, it's that we've removed the standard options SharePoint would normally provide for every web part (such as chrome style etc.), since we want to control these to ensure proper formatting. Normally we'd have these sections at the bottom of the tool part:

Summary

There are many ways SharePoint's web part framework can be extended, and here I'm only showing the path we followed. For a requirement such as our client's, web parts provided a great starting point, perhaps showing there can sometimes be a place for web parts in an accessible publishing site so long as the trade-offs are understood and accepted.

In part 2 of this series we'll look at issues encountered and their resolutions.

A better Config Store for SharePoint sites

I've recently made some enhancements to my Config Store framework on Codeplex which I'm now ready to share. Many of these enhancements are a result of adapting the solution to a large project where we built a platform for 80-100 internet sites - hence it's now become a bit more 'enterprise'. With such things I generally make a deal with my employer where I do the work (or most of it) in my spare time and then get to share the code publicly, so here we go. Before I delve into the details, let's have a reminder of what the Config Store is all about:

Recap - the SharePoint Config Store in a nutshell

Regular readers may remember this is a solution which allows use of a SharePoint list to store configuration values used by your SharePoint application - the idea is that your webparts/server controls/page layouts etc. store any strings/data they need for configuration in here as a more flexible alternative to web.config or similar. Since our values are now in a SharePoint list, configuration can be updated across the farm through the browser to administrators who have the appropriate permissions. We can also optionally take advantage of all the other things lists give us such as auditing, item-level security, alerts and version history etc. Finally, a caching layer is used to avoid round trips when your code fetches configuration values.

On the last couple of projects where my team has used it, we've finished up with 100+ config items in the list - storing all sorts of things from URLs, strings and 'application behaviour' switches:

If you need a more complete overview, see:

• http://www.sharepointnutsandbolts.com/2008/05/introducing-sharepoint-config-store-for.html
• http://www.sharepointnutsandbolts.com/2008/05/config-store-faqs.html

Enhancements in the new release

1. Optional "hierarchical" configuration model similar to web.config
   
   In the first release, since all the config values are stored in one list this means your configuration is stored in one 'nominated' site collection. This is fine for WCM sites which may only use one site collection, but may not easily map to certain enterprise requirements. What's new in this release is that the framework can now use a 'hierarchical' model, where a Config Store list exists in whatever site collections should have one, but a 'master' site collection is nominated which contains the 'master' config values. What happens is that if the config item you request is in the 'local' Config Store you'll get that value, and if not you'll get the value from the master list. This allows local overriding of the parent values if required - in practice we found 95% of the config items would be stored in the master list only, but having the facility to override the other 5% was critical to supporting some of the functionality we developed.
   
   Needless to say, this is the implementation which best suited our requirements. It could be it doesn't really suit yours, but developers could consider starting with my source code and modifying, since other aspects such as the caching layer, Feature files, event handlers etc. might not need to be changed much.
   
   If you want to continue to just use one Config Store list (even if you consume the config values in multiple site collections), the new code will continue to work just fine for this model too.
   
   
2. Easier to use in ASPX markup
   
   Similar to my recent Language Store framework, the Config Store now supports easily dropping values into ASPX markup by implementing an expression builder. So if all you want to do is set the Text property of a control, you can now do with this without cluttering up the code-behind. Or, as an alternative example, here we're retrieving an URL from the Config Store (stored in Category 'PageUrls' and key 'MyAccountPage') to assign to a hyperlink:

   <asp:HyperLink id="hyperlink1" NavigateUrl="<%$ SPConfigStore:PageUrls|MyAccountPage %>"
     Text="My account" ImageUrl="images/pict.jpg" runat="server"/>

   
   
   
   
3. Fixed caching bug for farm environments
   
   Yes, something I have to hold my hands up to here - the caching layer in the initial release didn't adequately deal with multiple servers. The effect was that it required an app pool recycle to pick up changes to config values, rather than them taking effect immediately. Not the end of the world, but certainly inconvenient and taking away some of the benefits of using a SharePoint list for configuration. So in this release, the implementation correctly relies on a back-end resource (using a CacheDependency) to invalidate the cache across all servers in the farm. The implementation I chose was to add the items to the cache with a CacheDependency on a text file - this needs to be located on a path which all servers can access - and the event handler now updates a timestamp in the file to invalidate the item in the cache across all servers.
   
   I went with using a file as the dependency item as I thought it was more lightweight than using a SqlCacheDependency - I didn't really want to impose the SQL configuration etc. to be able to use the Config Store. However, with the file cache dependency I've chosen, be aware that some production configurations may have internal firewalls which prevent all WFEs from accessing a shared file in this way - check before you deploy. Of course the code is there for you to modify should you wish to make changes in this area.
   


4. Amendment to Feature to prevent web.config modifications being made on Feature activation
   
   Another thing that got in the way when I tried to implement the Config Store on our enterprise project was the Feature event receiver which adds the required appSettings keys to web.config. The idea here is that, to help simplify installation and initial setup, the required web.config entries are added with empty values so all the developer has to do is plug in the appropriate values for his/her site. Sounds great - and it is for single site collection sites. But for multiple site collections, when we come to activate the Feature in the 2nd, 3rd, nth site collection - of course the receiver runs and adds the empty entries to web.config again, despite the fact that we'd already inserted the real values on the first activation. And since the new values come later in the file, guess which ones are used?
   
   So in this release there's a Feature property which determines if web.config modifications are made - it's set to 'False' by default but it's there if you prefer to change it.
   
   
5. 'Config value' column is now bigger
   
   Previously this column was a 'single line of text' but it's now a 'note'. This means you can happily store HTML/XML fragments or other larger values, which is very useful in some scenarios. However, note that if you're already using the Config Store on your site and want to 'upgrade', this schema change is significant - I discuss it in the readme.txt file.

So that's it. You can download the updates from www.codeplex.com/SPConfigStore - hope you find it as useful as we have.

Speaking at the SharePoint Best Practices conference

By now you've probably seen a few posts on the Best Practices™ SharePoint Conferences which are coming up - I'll be speaking at the European 'edition' which is being held 6th - 8th April in London, UK. I submitted a couple of sessions, but here's the one which got confirmed:

Customizing SharePoint the supported way: from end-user to admin interfaces

One of the key concepts SharePoint developers must remember is that modifications to core shipped files are typically unsupported. This session looks at ways in which you can implement the kind of customizations your users may ask for without doing anything naughty. Aspects covered include custom site definitions, modifying the site administration/central administration areas, and changing the user controls SharePoint uses with the DelegateControl architecture. The session is packed with demos, and also contains tips on the best way to make other common customizations not shown in the detailed examples.

I'll probably be off his Christmas card list for saying this, but when Steve Smith (fellow UK MVP and conference organiser) mentioned a while back that he wanted to put something like this on and I should think of sessions, I kind of assumed he meant something with the usual UK suspects, a bit like a user group meeting on steroids. It was only when the site went up I saw how wrong I was! Many of the world's biggest SharePoint names are on the speaker list - the developer track has Andrew Connell, Todd Bleeker, Eric Shupps, Maurice Prather (ex SharePoint product group) and fellow UK SharePoint MVPs Andrew Woodward and Ben Robb. And that's not all - there are 4 tracks in total and I've already noticed some non-dev sessions I'd like to get to from experts such as Spence, Mike Watson, Ben Curry and others. The keynote speaker is Joel Oleson. Regardless of your level with SharePoint, for my money there is huge potential for pushing your SharePoint capability forward at this conference.

As an aside, the other session I submitted (which I really wanted to do!) was this:

Approaches and best practices for deploying SharePoint sites through multiple environments (dev, QA, UAT, production)

Solutions and Features, STSADM export, the Content Deployment API – all can be used to move SharePoint site artifacts from one environment to another, and all have different capabilities and limitations. Failing to plan your deployment processes properly can add significant risk to a SharePoint development project, and in some scenarios the problems may only become apparent when initial development is complete and updates need to be released. In this session we’ll explore the nuts and bolts of each development/deployment approach, and you’ll emerge on a firmer footing in this complex area of SharePoint. We’ll also touch on 3^rd-party tools which can help with the deployment puzzle, from Codeplex utilities such as the Content Deployment Wizard to emerging tools such as Kivati Studio.

If you're thinking of attending and would be interested in this topic, leave me a comment and if there's a good response I'll try and persuade the organisers to shoehorn me in somewhere (perhaps in one of the lunchtime sessions which haven't yet been announced).

European Best Practices™ SharePoint Conference

Look forward to seeing you there!