Tips for building SharePoint 2010 base VM images

Now that SharePoint 2010 has RTM’d, lots of new virtual machines are emerging chrysalis-like from their proud host machines (hmm, that analogy might need some work) - whilst it’s mainly the early adopters at the moment, this will be a common process for most folks involved with SharePoint 2010 during it’s lifespan. Of course if you build many VMs, it’s worth spending time on some base images which you will use as templates for others. In this post, I’ll run through what I’ve been using as my VM “tree” which saves me time when creating new machines – if you don’t have images like this at the moment, you might be wasting time when it comes to spinning up a new environment. Even worse is being reluctant to create a new VM because it entails lots of hassle – having a barrier like this is not conducive to accurate development/testing/research because frankly, developer VMs tend to grow “baggage” which isn’t always cleaned up. My setup is developer-focused and is actually what I’m running at home on my laptop (8GB RAM) rather than at work, but you should be able to easily identify if you are working with different constraints/requirements.

The goals of creating decent base images are:

• Reduce the effort across different configurations (e.g. Foundation/Server, standalone/farm etc.) by installing/configuring common elements once
• Reduce the time it takes to create a new VM
• Reduce human error in install process (since we do it fewer times)

Overall configuration

When creating SharePoint 2010 VMs, one of the early decisions is how to deal with platform services such as Active Directory, DNS and SQL. Briefly the choices are (focusing on AD here, but many also apply to SQL):

• Have each VM host AD Directory Services, so each VM is in it’s own domain
• Have a separate VM to host AD, other VMs are members of the same domain
• Don’t run under AD, instead use the “create config DB using PowerShell” trick to run under local accounts. Without AD you won’t be able to do certain things (such as user profile sync, obviously) so you need to be happy with these trade-offs.

I’ve opted for:

• One VM running Windows 2008 Server Core for AD DS, DNS etc. (I allocate just 256MB RAM and less than 5GB of disk space to this!)
• One VM running ‘full’ SQL 2008 + SP2010 (I peg SQL to use a maximum of 1GB RAM – my feeling is that SharePoint needs it more in a developer VM)

However I wouldn’t necessarily recommend this factoring for everyone. I do it this way because I think if I have a hardware failure shortly before say, a big talk (as you can probably tell, it’s happened), I could get up and running on someone else’s borrowed laptop with a backup of the SP2010 VM pretty quickly. Sure I’d have some domain/account issues to deal with (if I didn’t have a backup of the AD VM), but it feels more self-contained and easier to restore if I had to. I might be slightly crazy on some of the logic here, and at some point I might cut over to the approach I’d generally recommend:

• One VM running AD, DNS and SQL
• One VM running SP2010

If you ever have other VMs running at the same time (e.g. an SP2007 VM or another SP2010 VM if you have the RAM), you will definitely see more performance with this option. You really don’t want more instances of SQL (in particular) running on the same physical hardware than you need.

My base image “tree”

In terms of a set of base images for creating new VMs from, this is what I currently have – the blue boxes represent base images, the gold ones are VMs which I actually run and use:

Hopefully the names are descriptive enough to get a quick idea of each machine’s purpose. As you can see my “instance” VMs aren’t imaginatively named, and the “-1” suffix is simply because I’ll accumulate ‘n’ instance VMs over the next few years. In the unlikely event it’s not clear, SP is SharePoint Server, whilst SPF is SharePoint Foundation!

2010-BASE

• Win 2008 R2
• Roles
  • Application Server
  • File Services
    • Web Server (IIS)
• Features
  • Remote admin tools e.g. AD tools
• SQL 2008 SP1 CU2 (at some point I’ll upgrade this to R2) – note there is an element of risk in having SQL in a base image, and it’s important to know about this. For example I’ve not verified *everything* SharePoint needs works – I discuss the issue issue and how to deal with it further on.
• WCF hotfix for R2 (KB976462)
• Conveniences
  • Disable shutdown event tracker (this drives me crazy on developer VMs, I can’t understand why it doesn’t for everyone!)
  • BGInfo (in startup folder)
  • Disable loopback check

Then sysprep’d.

2010-DEVBASE

As above with:

• SharePoint 2010 pre-requisites installed
• Visual Studio 2010 (RC)
• SharePoint Designer 2010
• Dev tools such as Reflector Pro (with the SharePoint assemblies decompiled and ready for debugging), I’ll probably add ReSharper 5.0 here soon etc.
• CKS:DEV

Then sysprep’d.

SP2010-BASE

As 2010-DEVBASE but with SharePoint 2010 installed – * Config Wizard/psconfig not run.

SPF2010-BASE

As 2010-DEVBASE but with SharePoint Foundation installed – * Config Wizard/psconfig not run.

Benefit

This allows me quickly create new VMs as most of the install pain went into the base images. Even things like having Reflector Pro decompile the SharePoint assemblies (shh!) in the base image means I don’t have to repeat that step on each VM I end up using – if I’m knee-deep debugging SharePoint code I can guarantee I won’t be happy waiting 15 mins for that step to complete. It would be great if we could do more configuration than the above in base images, but I don’t think you reliably can. Also, I’d love to hear if you do things differently with your base images.

General notes

The following are some notes around the images and the process I use to clone:

• The base image VMs are not joined to any domain
• The images are all sysprep’d – if you’re not doing this with your base images, you will run into issues if you ever run 2 machines at the same time on the same domain
  • Note that sysprep resets lots of things, including Windows activation and the Windows Update status (Windows will report that updates have never been installed, though whatever updates the base image had will actually be there).
• Hopefully everyone knows this, but in SharePoint 2010 (as with 2007), it’s not possible to have base images which have SharePoint installed and configured. Or rather, it’s possible but if VMs are created from such images they cannot reliably exist on the same network/domain, let alone the same SharePoint farm. This is because SharePoint stores the machine name in the config database, and if you ever want two instances running on the same network/domain you’ll need to rename one instance, which will then not match up with it’s database entries. The latest point you can get to is with the SP installation done but the Config Wizard not run – kind of inconvenient, but…
• …of course PowerShell can help with making the rest of the configuration (e.g. provisioning/configuring service applications) as repeatable and quick as possible. Gary Lapointe (a man that should know if ever there was one) suggests that a 100% PowerShell scripted configuration may be out of reach in the current release due to some config steps which can’t really be done with PowerShell, but at some point I’ll likely try and stitch together some scripts which suit most of my needs.

Process to create a new VM

To create a new VM, I follow this process – note there are some hooky SQL steps here due to it being included in a sysprep’d base image, keep reading for why and how this can now be avoided:

• Do a VMware (or Hyper-V) “clone” from a sysprep’d image (this will usually be my SP2010-DEVBASE image)
• Start VM and then:
  • Change machine name
  • Join to COB domain
  • Do Windows updates, reboot
• Grant the account which will be used to run the SharePoint config wizard rights in SQL. Since sysprep has changed the SIDs of the accounts, no user has permissions to SQL Server at this point. To fix, we need to start SQL in single-user mode and add logins/permissions:
  • Any any accounts which need SQL permissions outside of what the SharePoint config wizard will grant when it runs. The main example here is the domain account used to install/configure SharePoint, in my case “COB\sp_setup” (which should also be a local admin):
    1. Open SQL Server Configuration Manager
    2. Stop the main SQL service.
    3. Go into the properties of this service, and on the Advanced tab find the Startup Parameters property. Add “;-m” to the end (without the quotes, ensure you have no spaces) and click ‘OK’.
    4. Restart the service (you’re now in single-user mode).
    5. Create a SQL login for your install account, and add to the following server roles:
    1. securityadmin
    2. dbcreator
    6. Note that if you had SQL service accounts running as built-in system accounts, there could be further work to do there. I have a domain account for this purpose and didn’t run into issues with SQL service accounts.
    7. I’ve also seen references that the server name in SQL needs to be updated, but that’s not my experience.
• Run SharePoint configuration wizard/psconfig
• Do other configuration e.g. service apps

Sysprep and SQL – a better way

So that’s a bit of a smell – what’s the deal? Briefly, it is not supported to run sysprep on a machine running SQL Server, something I’m not sure lots of SharePoint folks know. There is no way in hell I’d think about doing things this way with images used for a client’s environment – some research suggests things like Reporting Services and DTC transactions are examples of SQL areas which do not play well with sysprep. However, like me, you might find sysprep’d images with SQL work just fine for SharePoint developer VMs, depending on what you’re doing. The alternative is somewhat more complex, but I’ll likely cut over to it fairly soon even for my home stuff – SQL Server 2008 R2 (released a couple of weeks ago) adds support for sysprep! Basically it’s a two-step installation process, the first half of which is done in the base image (“prepare”), the second in each instance (“complete”).

Josh Meyer has a great walkthrough of the overall process for creating SharePoint 2010 VMs.

Some background reading on sysprep and SQL:

• Why Not to Sysprep SQL Server
• Changing Machine SID With NewSID Breaks SQL Server (And How To Fix It)
• How does changing Windows SID affect SQL Server?