I spend a chunk of my time talking to CIOs and other leaders deciding the AI strategy for their organisation, and a common conversation at the moment is how to frame Copilot for Microsoft 365 against other options. It's the new "what to use when" conversation in Microsoft-land. If the challenge being addressed is 'providing the right set of AI tools to a business', there are multiple options from Microsoft and beyond, and choosing the right approach comes down to properly considering what you're trying to solve for and understanding not just the orientation, but also the capabilities and limitations of the various options.
Common questions I'm hearing include:
- We're not sure about licensing Copilot for M365 across the entire business, what other options do I have?
- I see ChatGPT has an Enterprise version now - it seems to solve the data privacy risks of the consumer version, and all of our employees are familiar with ChatGPT. Should I use that?
- Why could I not just use the free version of Microsoft Copilot (i.e. what was "Bing Chat Enterprise")?
- I understand Copilot for Microsoft 365, but where does Copilot Studio fit in?
Understanding the characteristics and pricing of the major options becomes vital to make the right decisions. Sometimes it will be an economic decision, sometimes it will be capability led - we've seen it all. I'm a strong advocate of Copilot for Microsoft 365 (as is my employer, Advania UK - we licensed 100% of employees), but it's not necessarily the answer to every AI question. Indeed, my team has solved AI challenges with "custom Copilot" approaches using Azure OpenAI that simply couldn't be done with Copilot for M365, including integrating large volumes of data from SaaS apps and custom databases, and steering the AI past the one-size-fits-all behaviour of Copilot. Despite it's strengths, Copilot doesn't quite offer the flexibility or cost effectiveness needed for certain scenarios.
Major AI options on a slide
To help with AI strategy conversations, I produced the slide below which I'm sharing here in case it's useful for others (you can download it in PowerPoint form). It's an attempt to summarise some of the key factors and differences, though I'd be the first to say it doesn't cover every consideration and has some subjectivity to it. When I'm asked a question like "where does X fit?" or "how should I think of Y?" question, I often put the slide up to call out some of the major factors and differences as we walk through some options.
When considering AI options, I see some of the major considerations as:
- Overall positioning and value prop ("Headline" in my slide)
- The cost model
- Costs
- Data sources - which company sources of data can the AI talk to?
- Automation - whether you can fully automate a process with the tool, or whether it's purely end-user driven
- The surface - where the AI shows up
- Key limitations
I often feel this kind of thinking and comparison is what's missing from Microsoft and other vendors. The strengths of a technology are extolled in the documentation and content, but rarely the limitations and "but bear in mind...." considerations. As an example, the Copilot Studio page is unlikely to ever say that it's a great technology for some use cases, "but good luck forecasting your run costs, because the pricing model makes it really hard!" It's factors like this which are hugely relevant if you're deciding (or paying for) the AI strategy for your organisation however.
There's quite a lot of info - and some opinion - in the slide, so I unfold some of my thinking in the notes below. Here's the slide itself (download link at the end):
Headline/value proposition
Most will be familiar with the value proposition of Copilot for Microsoft 365 - your data in M365 (documents, mail, Teams chats and more) integrated with LLM capability and surfaced in the flow of work via Teams and Office apps. If you're an existing user, you'll probably cite the very strong capabilities around Teams calls and meetings as a highlight - especially intelligent recap, with auto-generated meeting summaries, action items, speaker attribution, being able to ask questions of the transcript and more. Alongside, there are lots of other capabilities which boost productivity across many organisational use cases and roles/functions, particularly when working with documents. Sometimes accuracy and results can be mixed, and that sometimes that's due to certain Copilot limitations today - more on this later. Sometimes I'm asked how Copilot Studio relates to Copilot for Microsoft 365 - but this has a different value prop entirely. Copilot Studio is suited to 'focused' Copilots for specific use cases - perhaps to provide answers on HR policies, employee onboarding, product documents, or an FAQ, and they can be surfaced internally or externally on a .com website. Copilot Studio solutions are often modern day chatbots, much more intelligent than those of the past because they combine LLM power, a focused knowledge base or set of data, and particular instructions (grounding) to the AI on how to provide the best possible answers. However, Copilot Studio isn't suited to solving broad AI needs (e.g. an internal Private ChatGPT) because of the limitations on data integration and difficulties predicting costs. For smaller needs it's perfect though - our team has built some amazing solutions already, and I predict many apps and in particular Power Apps, will shift to be Copilot Studio solutions in the next few years. We're just all comfortable with using chat for different things compared to 5 years ago now that it can actually work - CIOs and app makers should be cognisant of this.
Microsoft also offer Copilot (free) and Copilot Pro (£19 per month) as the evolution of what was Bing Chat Enterprise, but these should be seen as competitors to public/consumer ChatGPT rather than a true organisational solution. The main callout is that you have no roadmap or possiblity of integrating company data with these tools - they are LLM only. In Copilot Pro you can use the AI with a document you have open in Word (e.g. for summarisation, generation, rewording, analysis etc) and there is enterprise data protection so any sensitive data can't leak out, but Microsoft Copilot/Copilot Pro don't provide any way of answering questions from company data at scale. A CIO might find Microsoft Copilot appealing as a free option with no barrier to entry, and while it's arguably safer in the workplace than public ChatGPT, my view is you're likely to confuse employees if it's made available as a first step in AI, but then subsequently replaced by other AI tools (e.g. Copilot for M365 or an internal Private ChatGPT) as you unfold your AI strategy.
A Private ChatGPT solution built on Azure OpenAI can be attractive because the costs aren't per-user. Because the costs scale in a different way (i.e. they are "platform + AI consumption" rather than per-user), we've seen a lot of interest in this from mid-sized and larger organisations who aren't sure about a large scale Copilot investment but do want to provide generative AI tools integrated with organisational data. In quite a few cases, the organisation is choosing to deploy a platform like this in addition to Copilot rather than instead of, and at Advania we've had quite a lot of success with our clients in this space. We commonly integrate Microsoft 365/SharePoint/Teams data so the AI is able to answer questions related to the company's clients, projects, people, policies, sales and product information etc. It becomes a powerful tool that changes the employee experience significantly because it provides a new way to find organisational knowledge and get straight to the answer. As alluded to earlier, it can also be the foundation for a tailored AI platform designed to support specific use cases and integrated with data from different platforms. We've integrated Azure OpenAI with incident management systems, HR data, employee skills/certification data, access card systems and much more. As such, a Private ChatGPT solution built on standard gen AI approaches can go beyond simply being a digital assistant for common productivity and creativity tasks, and be the AI platform where high value processes and use cases are enabled. We're seeing many organisations think in terms of an AI platform for the business that can scale and be extended over the next few years as AI opportunities and use cases emerge.
Further thoughts
As you can probably tell, there are many considerations across multiple dimensions trying to come out in the slide above - and even then it's only a biased, partial view. I'm not unfolding every point referenced on the slide, but just a couple of others to call out:
- Is the tool automatable?
- Whether the AI tool supports automation is a key factor that isn't always considered. For some use cases you want to throw 100 or 1000 items to the AI in bulk, but ready-to-go solutions like Copilot for Microsoft 365 don't lend themselves to this because they have no API. We're doing a lot of work in this space with Private ChatGPT/Azure OpenAI solutions for Due Diligence Questionnaires, Cyber Security/InfoSec questionnaires, RFPs etc. - in many of these cases, organisations are seeing the potential for better outcomes than with specialist tools they already have, but which don't use AI with their data effectively and/or haven't really landed with the business, which is interesting. I posted about this observation here.
- Limitations
- Every AI tool has boundaries and someone supporting your decision-making should understand them. Some examples are called out on the slide, but of note is that Private ChatGPT solutions typically can't index Teams chats and e-mails in the way that Copilot can (nor do they provide Teams meeting support) - they are limited to understanding knowledge found in documents primarily. Conversely, Copilot for M365 has limitations in understanding long documents, and this can be very relevant in some scenarios. Microsoft recently announced the limit is now around 80k words when using Copilot in Word, but it's worth also understanding that - as far as I know - it's still only around 20k words in terms of what Copilot Chat will understand (i.e. when asking Copilot questions of your data generally), because that's how much gets indexed into the Semantic Index.
At the same time, these limits and the general value of the Copilot proposition should only improve from here.
Conclusions
If you're interested in these perspectives and can make it to this year's ESPC24 Conference in Stockholm this December, I'll be speaking about this in my session AI on your data Deep-Dive – Comparing Copilot Studio, Copilot GPTs, and Azure OpenAI on your Data. Hopefully the slide is useful to someone either way.
