Implementing AD integration with Office 365 using a sub-domain (for dev/test)

As discussed in Challenges in Office 365 development – and how to address them, it’s fairly common to create multiple Office 365 tenancies to get around the fact that there’s currently no such thing as a “test environment” in Office 365. However, as the previous post discusses in detail, it’s certainly true to say that some trade-offs come with this method, generally related to environmental differences, code issues, testing, Application Lifecycle Management and so on. This article series attempts to provide some options and techniques which might be useful:

• Challenges in Office 365 development – and how to address them
• Implementing AD integration with Office 365 using a sub-domain for dev/test (this article)
• Enabling Yammer Enterprise with SSO in dev/test environments
• Using Azure Deployment Slots to implement dev/test/production ALM for Office 365 apps and SharePoint Add-ins

Summarizing the problem

Non-production tenants used for dev and test typically lack some of the aspects of the production environment – things like:

• Directory integration (i.e. users sign-on with their “real” Windows account – e.g. “chris@mycompany.com” rather than “onmicrosoft.com” cloud accounts)
• Synchronization of user profile data between AD and Office 365
• Yammer SSO (or Yammer in general!)

And this leads to a long list of compromises – from not being able to test the real user experience properly, to sometimes having to write code to use a different “mode” in dev/test compared to production (perhaps because user account names work in different ways, or profile data isn’t populated).

Background – using one AD and multiple subdomains for Office 365 integration

So, we’ll look at one option for mitigating this issue – a technique which allows configuring non-production Office 365 tenancies with full directory integration and synchronization of user account data, but WITHOUT the need for a separate custom domain and Active Directory for each (and all that entails). This post is effectively a big HOW-TO article which describes the configuration process. In a team working on perhaps 15+ Office 365 implementations at any one time, you can probably see the attraction of this technique for us! Frankly, it can be challenging in the enterprise to get a test AD and domain set up which can be used with test Office 365 environments – with servers, networks, AD, security and operations all involved, quite a few people across I.T. might be needed there. So, the idea of having ONE Active Directory which could be used for directory integration with MULTIPLE Office 365 environments can be useful for lots of different teams in many different contexts.

What we're doing here

Ultimately the way we achieve this is to perform the regular “custom domain integration with Office 365” config steps, but with a couple of tweaks. Office 365 administrators and platform people will be very familiar with the standard process, and it’s a key element of hybrid SharePoint/Office 365 configuration. However, most developers will be less familiar with this stuff – but I personally think it’s hugely beneficial for any technical person working with Office 365 to have been through this process at least once.

At this point, I need to make clear that a lot of the smart thinking behind this approach actually comes from one of my colleagues at Content and Code – I’m really just being the mouth-piece here ;) Tristan Watkins is Head of Research and Innovation at Content and Code, and Tristan did the initial work of proving the approach – awesome work as usual.

What we actually do is use a sub-domain for each different Office 365 tenancy we want to integrate with the Active Directory/domain. So this could be:

• Client1.MyCompany.com
• Client2.MyCompany.com

Of course, you might substitute “Client” with “Project”, or anything else that you’re using different tenancies for.

Or perhaps:

• Dev.MyCompany.com
• Test.MyCompany.com
• UAT.MyCompany.com

..and so on.

Ultimately where we’re going with this is that we’ll have one AD which can provide accounts to multiple tenancies. Whilst it may not be appropriate for production, this can really help us with our mission to make dev/test environments more like production. User accounts in Office 365 don’t *have* to come from AD when cloud identities are used (more on this later), but it can be useful to stay closely with the model that you’ll use in production and implement sync between AD and O365.

Things you’ll need

We need to be ready with each item in the list below – if not, you’ll need to purchase/obtain the following:

• An Office 365 tenant ready and provisioned
• The top-level domain (e.g. MyCompany.com) – this needs to be something you/your organization already owns and has registered
• Ability to manage DNS for the domain (usually at the ISP hosting the domain)
• Ability to create user accounts in the AD you’ll use

At a high level, we go through the usual steps to add a custom domain to an Office 365 tenancy – I’m using GoDaddy to host my domain/DNS, and Office 365 offers some useful integration/simplified admin with this and some other hosters. However, with the sub-domain approach, even if we ARE using GoDaddy to host our domains, we’ll need to manually configure our DNS records rather than can click any of the “magic buttons” to buy or configure the domain (like this one):

So, just be aware that although Office 365 has numerous “simplified admin” options if you’re using GoDaddy, for the most part we’ll be ignoring these and doing some extra configuration ourselves.

HOW-TO: configure Office 365 domain integration with a sub-domain

For reference, in this walkthrough I’m using the following:

Office 365 domain	Custom domain (top-level)	Subdomain used for this tenancy
cobsp ..so this gives: - URL = cobsp.sharepoint.com - Usernames = user1@cobsp.onmicrosoft.com	chrisobrien.com	cob.chrisobrien.com

So here’s the process. Firstly we go to the Office 365 admin center, and then into the “Domains” area. Once there, we click the “Add domain” button on the “Manage domains” screen:

We then start to step through the configuration of adding a custom domain to Office 365:

Hit the “Let’s get started” link on this screen. On the next screen (below), we should enter the subdomain/domain we are intending to use for this Office 365 tenant – as mentioned previously, we’re going to need to add DNS records there so we should be ready for that.

Once the domain is entered, hit “Next” and see the screen below. Here, you need to NOT sign-in to GoDaddy and have Office 365 attempt the config for you. Instead, the DNS records need to be created manually, so click the “use a TXT record” link as shown below:

This next screen gives details of the TXT record you’ll need to add at your hoster (GoDaddy in my case):

Doing this helps authenticate the fact you do indeed own and control the domain, and aren’t trying to pull some sneaky internet trick. So, we now need to go to the control panel of the ISP hosting our domain, and specifically the DNS area. At GoDaddy, it looks like this:

I use the “Add Record” link to add a TXT record with the details Office 365 gave me. Once done, I wait a while and then go back to Office 365 and confirm I’ve now done this step. If I added the TXT record as specified and DNS has propagated, I should see something like the following:

I click the “Next” button here. This will update any user’s e-mail addresses from user1@cobsp.onmicrosoft.com to user1@chrisobrien.com in my case:

You can then optionally add some additional users (manually). If you plan to sync users from an on-premises AD using AAD Sync you probably won’t use this option, but for dev/test environments you might want to add a couple more users at this stage (assuming you have licenses for them):

Now we get to the important part – adding the main set of DNS records to support the integration with your custom domain:

Office 365 asks you which services you plan to use/integrate, so that you only need to worry about the relevant DNS records:

On the screen which follows, there’s another one of those super-handy shortcut links that unfortunately we can’t use (because we’re doing the non-standard thing of using a sub-domain) J Skip the “Add records” easy button, and instead click the “add these records yourself” link:

The screen which follows lists all the DNS records you need to add at your hoster. You now need to cross-reference this screen with the DNS panel there – for each item listed, configure the DNS record in the admin panel. The key difference compared to the standard process is that the sub-domain is specified in each DNS record, rather than just the top-level domain:

Here’s an example of adding a record in the GoDaddy DNS admin panel:

Once you’ve added all the records, go back to Office 365 and click that “Okay, I’ve added the records” link. Office 365 will now check the records you added, and if successful you should see a confirmation message as shown below:

Your custom domain is now integrated, and the status should be reflected on the “Manage domains” admin page:

Review - so what do we have now?

At this point, we now have our on-premises AD integrated with our Office 365 tenancy, using a sub-domain. However, we don’t have user accounts actually existing in Office 365 yet – accounts can be created there in different ways, and we need to use one of them to actually get identities up into Office 365 for testing. This applies regardless of whether we’ll actually log-in with these accounts, or simply use them as “passive” users. Ultimate these are cloud identities, so if these users will log-in our model is to use cloud authentication rather than the other option (federated authentication e.g. with ADFS or similar).

By the way, if some of this is confusing and you need a good background on the different options for identities and authentication in Office 365, I’d suggest starting with User Account Management on TechNet.

Actually creating users in Office 365 – AAD Sync, manual, or bulk update

With or without a custom domain (or subdomain), users can be created in the following ways:

• Manually in the Office 365 administration UI
• Using PowerShell
• Using bulk upload from a CSV file
• Using AAD Sync
• Using an Exchange mailbox migration

In other words, it’s worth remembering that even when you’ve integrated a custom domain with an Office 365 tenancy, users do not *have* to come from AD. However, this is definitely possible if you want to stay close to production and perhaps have synchronization occurring from AD to Office 365.

Using AAD Sync to sync users from AD to Office 365

Implementing sync from AD with this sub-domain approach is made possible by adding an alternative UPN suffix to AD domain (as detailed in the “Assign a UPN domain suffix” section of Configure Office 365 for SharePoint hybrid on TechNet) – this effectively allows us to have different users in the directory be matched to different Office 365 tenancies:

Once the UPN suffix has been added in AD, it’s then possible to assign this suffix to individual user accounts – thus “matching” them to a particular Office 365 tenancy and URL domain:

I also tend to put each set of accounts in a different OU so I can easily see accounts for each tenancy.

At this point, you could now implement AAD Sync to synchronize accounts – you would download and install/configure the AAD Sync tool, which is a step that Office 365 administrators will be familiar with:

This will provision FIM and the scheduled task for ongoing sync from AD. In FIM, you could then restrict the sync to a particular tenancy to users you earlier put in a specific OU:

If you want to do this against multiple Office 365 tenancies, one thing to be aware of here is that you’ll need multiple instances of FIM and the sync tool/task (as far as I’m aware). Since none of this sync stuff to dev/test is mission-critical for you, this can be managed pretty easily by just deploying AAD Sync in a couple of different small VMs (i.e. one per tenancy).

At this point, you’d now have users being sync’d from your AD instance to your dev/test Office 365 tenancy. Cool!

And finally, don’t forget that if you want these test users from AD to be able to login and consume Office 365 services, you’ll need to assign them a license in the tenancy.

Summary

Implementing directory integration is required in some Office 365 contexts (e.g. hybrid, and where Yammer Enterprise is used), but is useful in many others too. If you’re using multiple Office 365 tenancies to represent non-production environments, it’s a pain when dev/test don’t match the production set-up, especially if you find yourself implementing any custom functionality around user profiles. The approach detailed here provides one option for facilitating directory integration for dev/test Office 365 environments, without some of the infrastructure requirements and headaches that would usually come with this.

This series will continue to discuss options and techniques for improving Office 365 development. Other posts:

• Challenges in Office 365 development – and how to address them
• Implementing AD integration with Office 365 using a sub-domain for dev/test [this article]
• Enabling Yammer Enterprise with SSO in dev/test environments
• Using Azure Deployment Slots to implement dev/test/production ALM for Office 365 apps and SharePoint Add-ins

Challenges in Office 365 development - and ways to address them

Over the last 2 years, I've spent quite a lot of time thinking about "cloud-friendly" SharePoint development - approaches and techniques which will work for Office 365, but also on-premises SharePoint deployments which need to be designed with the same principles. With my team now having now done at least 20 or so Office 365/SharePoint Online implementations (and counting), in this time we’ve established certain ways of working with Office 365. A good example is creating multiple tenancies to represent dev/test/production to help with our ALM processes, since Office 365 doesn't have the concept of a test environment. Sure, on the SharePoint side (my focus here) you could just create a special site collection - but that doesn’t give you the isolation needed across any global elements. Examples include things like user profiles, taxonomy and search. And since our clients often ask for minor customizations which interact with these areas, the “test site collection” idea just doesn’t cut it for us. Instead, we go with the multiple tenancy approach, and I’ve advocated this for a while for anyone with similar needs.
As I’ve previously discussed at conferences, our choice for most clients/projects is to create separate dev and test tenancies, and it generally looks something like this:

It’s not the most important point here, but we tend to use different Office 365 plan levels for these environments - most of our clients use the “Office 365 E3” plan in production, and we’ll ensure TEST is on the same plan, but reduce costs by having DEV use “SharePoint P2” (so no Exchange, Lync or Yammer Enterprise). This generally works fine, because most of our development work centers on the SharePoint side. But regardless of what you do with plan levels, it’s also true that some trade-offs come with using multiple Office 365 tenancies (of any kind) – recently I’ve been thinking about options to mitigate this, and these broadly can be categorised as:

• Making dev/test Office 365 tenancies more like production
• Finding ways to test safely against the production Office 365 environment

The next few blog posts will detail some techniques which can help here. In this first post I’ll discuss the problem space - some problems I see with current Office 365 development which might lead you to consider these approaches. But in the article series I’ll be discussing things like:

• Implementing AD integration with #Office365 using a sub-domain (for dev/test)
  
  • N.B. Directory integration is mandatory for true hybrid deployments or where Yammer Enterprise is used, but frankly is also useful to avoid other issues in dev/test
• Enabling Yammer Enterprise with SSO in dev/test environments
• Using Azure Deployment Slots to implement dev/test/production ALM for Office 365 apps and SharePoint Add-ins

For now, let’s talk about some of the issues you might hit in Office 365 development.

Challenges which come with multiple Office 365 tenancies

When we talk about dev and test environments, implementation teams always have a need to make these as similar to production as possible. The more differences, the more likely you’re going to have a problem - usually related to invalid testing or defects which only become apparent in production. Unfortunately, I notice our Office 365 projects do have certain trade-offs here. We really do want the multiple Office 365 environments for dev/test/prod (with the way Office 365 dev currently works at least), but it can be hard to make those other environments closely reflect production. Here’s a list of things which might be different:

• Typically, directory integration is configured in production, but NOT for other environments
  • In other words, users sign-in with “chris@mycompany.com” in production, but “.onmicrosoft.com” accounts are used in dev/test
  • [N.B. You might know this step as “implementing Azure AD Sync”, or “implementing DirSync” to use its previous name]
• Lack of SSO to Office 365 for users logged on to the company network (which relates to the point above)
• Lack of a full directory of users
• User profiles are not synchronized from AD in dev/test environments
• Lack of Yammer Enterprise
• Lack of Yammer SSO
• Different license types (e.g. E3/E4 in production, but something else in dev/test)

OK, but why should I care about these things?

Depending on what you’re developing, some of these things can definitely cause problems! Some tangible examples from our experience are:

• It’s not possible to do end-to-end testing – we can’t see the “real” user experience, especially across connected services e.g. Office 365 and Yammer
• The experience on mobile devices is different
• Code sometimes has to be written a different way, or use a “mapping” in dev/test - especially anything around user profiles. For example, any kind of user name lookup/e-mail lookup/manager lookups and so on
• Any integrations with 3^rd party/external systems might not work properly if they use the current user’s details in some way (because a different identity is used)
• Yammer – the lack of SSO means a couple of things:
  • Any standard usage e.g. Yammer web parts or Yammer Embed won’t “just work” - a login button is displayed, and the user has to supply secondary credentials to authenticate/get the cookie
  • Any Yammer API code might need a special “mode” – because you probably have Yammer SSO in production, but not elsewhere

What can we do about these challenges?

So, it would be nice if we could make this situation better. Many of the issues stem from the fact that dev/test environments don’t have identity integration and AAD Sync configured, and what’s generally getting in the way there is that standing up a dedicated URL domain and Active Directory often isn’t trivial. The good news is that the sub-domain/UPN suffix approach I’ll talk about in the next post allows you to get past this – regardless of how many Office 365 environments you have, all you need is one URL domain and one Active Directory. In dev/test for us, this means ONE URL that we registered at GoDaddy for all our clients/projects. We run the on-premises AD simply in a small VM, which runs on developer machines.
Once the domain integration is implemented, the next step is to implement AAD Sync to actually get some users from AD to Office 365.  It will run off a set of users you choose (perhaps you would group some users for each dev/test tenancy into different OUs), and will perform the step of actually creating the users in Office 365. You could then optionally assign them a license if you want this test user to be able to login and use Office 365 functionality, and actual authentication will happen in the cloud if/when the user logs-in. If you want to implement Yammer Enterprise and Yammer SSO, you can now do that too. Directory integration is a pre-requisite for both of these things, but having solved that problem without major platform/infrastructure headaches, these possibilities open up for our dev/test environments.

Summary

So that’s a little on the problem space. Ultimately, developing for Office 365 at enterprise level does have some challenges, but many can be overcome and we can still strive for robust engineering practices. The next few blog posts will cover some of this ground – I’ll add links to the list below as the articles get published:

• Implementing AD integration with #Office365 using a sub-domain (for dev/test)
• Enabling Yammer Enterprise with SSO in dev/test environments
• Using Azure Deployment Slots to implement dev/test/production ALM for Office 365 apps and SharePoint Add-ins

Thanks for reading!