As detailed in the last article, “autofill” columns in SharePoint are a very interesting built-in AI capability in Microsoft 365 where LLM/GPT capabilities can be used very easily with your documents. Unlike Copilot, which can’t really be used in bulk across your documents because it’s driven by the end-user, autofill columns could be used to summarise, categorise, or extract info from hundreds or thousands of documents very easily indeed. Pseudo-automation approaches with Copilot do exist such as creating an agent, but not all employees will do that and autofill columns are perhaps the simplest way of tapping into AI to get some of the benefits. Today, autofill columns use GPT-4 Turbo as the LLM and we can expect Microsoft to continue upgrading as newer models become available.
As a recap, I gave these examples in the last article:
- Document
summaries – the full document summarised automatically in your
preferred format (e.g. 3 bullet points, a few sentences etc.)
- Key
takeaways – a summary focused not on the full document, but the
conclusions only
- Key
info extraction – identify the client/project/business unit this
file relates to
- Key
info extraction – list the
people/organisations/concepts/[something else] mentioned in this document
- Classification –
categorise the document, either generically or according to a list you
provide (e.g. into “RFP”, “proposal”, “statement of work”)
- Automated
assessment - based on the contents of the document, should this
be approved or sent for human review?
- New
info generation – based on the contents of the document, how does
it correspond to X?
If more inspiration is useful, my last article shows the results of implementing a couple of these scenarios, showing the prompt used and what the AI did.
Costs - does this cost more, less, or the same as other AI tools I could use?
Experimenting with AI is generally cheap, but to use in real-world scenarios someone somewhere needs to understand the costs and pricing model closely. However, you’ll rarely see comparison tables on different AI approaches and respective costs published by vendors like Microsoft – you’ll see them for AI models, but rarely for overall approaches. It’s not that AI vendors are necessarily being disingenuous, more that there are a few variables involved such as whether your developer/team/partner has the skills to use different approaches – perhaps they can only use low-code or easy to consume AI for example and code development is out of the question.
But things do open up when you can plug into Azure OpenAI directly for LLM models for instance, because you avoid paying the “abstraction premium” for simpler approaches where Microsoft have done more of the work for you. It was ever thus as we say in England, and this is certainly the case for autofill columns.
Hit me with some numbers
Let’s make two comparisons with other ways you could consume AI, focusing on use cases like those above which align well with autofill columns:
- Syntex document understanding (for extraction or classification scenarios especially)
- A Copilot agent or Power Automate solution where the document contents are passed to an AI model (most likely by calling into Azure OpenAI)
Comparing with Syntex costs
SharePoint autofill columns are priced exactly the same as Syntex unstructured document processing at $0.05 per transaction (which means a page in both cases) – this is intentional by Microsoft no doubt, since some large-scale use cases could use either technology and I can see why Microsoft wouldn’t want a big disparity. Interestingly if there’s a pre-built Syntex model that suits your needs (e.g. for simple work with contracts, invoices, receipts, forms, PII detection or language detection), costs here are much lower at $0.01 per transaction (page).
Result – a draw (assuming you need 'full' Syntex, with the unstructured document model)
Syntex specialises in extracting key info from documents – but if you want to use AI in some other way and/or you need generative AI and LLMs you’ll need to look elsewhere. So, what about the idea of a Copilot agent or Power Automate solution?
Comparing with Azure OpenAI costs (via a Copilot agent or Flow)
Here’s where things get interesting. If you have the technical skills to create an agent or automation which calls into AI and passes documents to it, you can dramatically reduce AI costs – partly because going directly to AI is always cheaper than going via an “end-user” approach in Microsoft 365, and partly because you can control the AI model used. With autofill columns the model used is down to Microsoft (GPT-4 Turbo is referenced in the documentation), but with a built solution, choosing one of the latest “mini” models rather than full blown GPT-4 Turbo or similar has a dramatic effect.
A cost comparison calls for a quick spreadsheet. Let’s compare autofill columns with using two of the current AI models but going direct:
- GPT-4o
- GPT-4o mini
Here are the details of the scenario I’m modelling (see footnote at the end if you want more on this):
- 1000 documents per month processed by AI (e.g. summarised, analysed etc.)
- Each doc has 100 pages
- Assume average word density
- Azure OpenAI pricing details – USD (to help compare to autofill pricing which is also in $), East US Azure region, non-provisioned capacity
Here’s the calculation and findings (based on pricing in January 2025) – the costs for the scenario per month are highlighted in blue,
with the differential to autofill pricing in red:
To summarise:
- You’ll pay 33 times as much to use autofill columns compared to going direct with a similar model (GPT-4o)
- You’ll pay 551 times as much to use autofill columns compared to going direct with a cheaper “mini” model (GPT-4o mini)
The costs for the scenario using each approach are:
- GPT-4o mini - $9.07 per month
- GPT-4o - $151.20 per month
- Autofill columns - $5,000 per month
Those are quite striking differences – and over a year, the difference is magnified of course and we’d be talking $109 for the cheapest approach with GPT-40 mini vs. $60,000 for autofill columns. I had to check the calculations several times and ask a colleague to double check my working – if anyone can see any flaws in the method I’d love to hear via LinkedIn comments, but we’ve checked and double checked and this differential absolutely does seem to be the case.
Result – going direct to Azure AI is significantly cheaper
What do we take from this? Should autofill columns be avoided?
Even with these cost differences, autofill columns could absolutely be the most cost effective and ‘valid’ choice for your AI use case. The key considerations which come into view are:
- The scale you’re working at/how much AI you need to fulfil it (e.g. how many documents and pages)
- Possible avoidance of development costs
Autofill columns are ready-to-go AI that the business can use in a self-serve way. Certainly, cost governance measures need to be put in place to avoid bill shock, but there are options for this – at least in terms of someone being notified that spend might go beyond a defined threshold so they can intervene (by way of implementing an Azure budget on the Syntex meters). With other approaches, AI costs using Azure OpenAI are much lower but may require development or solution maker time – and that implementation cost could outweigh the cheaper AI costs, depending on effort required, day rates etc.
Overall, any options which put AI in the hands of the business should be welcomed since that’s where processes are understood and value can be easily created. Just take care with those cost governance measures if you enable any form of Pay As You Go AI.
Summary (and a view as a Microsoft partner/solution provider)
As someone working for a partner delivering services at the forefront of Microsoft AI, it’s interesting to consider the impact of “ready to go AI” in Microsoft 365 on companies like ours. If our clients can use AI without our help (such as autofill columns), will we deliver less AI advisory and implementation work? Will all our efforts to deeply understand the technologies, the factors in using each one, and how to create business value with them be to waste? Well, frankly it’s hard to imagine that based on the realities of getting the value from AI in production scenarios – if anything I think the reverse might be true. What we’ve seen in this analysis (and any other time we look at the “what to use when in AI”) is:
- You always pay a significant premium for simplified/abstracted AI – whether it’s Copilot, autofill columns, or something else. As a result, a ‘developed’ solution can save £1000s per year for many use cases due to the sheer difference in AI costs
- It’s a complex landscape where partners can add a lot of value - understanding the factors takes a lot of context and closeness to Microsoft developments
- Deriving likely AI costs for a use case is not simple – some modelling is required using a deep understanding of Microsoft technology. And cost comparisons differences across approaches are rarely found in the documentation
Nevertheless, the value is out there. While it's certainly true that the right decisions need to be made within a well-run "POC to production" motion, we're seeing more and more AI use cases related to high-value processes where there's a significant cost saving to the organisation. Often there are some accompanying benefits to employees too, and both will only accrue further as time goes on.
Understanding the right AI technology and approach for the use case is the key of course - no more, no less.
Footnote - more detail on the cost comparison
For simplicity, I’m considering the AI consumption costs only. Depending on circumstances you may also have some Copilot agent costs or Power Automate costs to consider too.
To compare the AI costs for autofill columns vs. Azure OpenAI, we need to calculate a "per page" cost for both approaches. Autofill columns and Syntex models are priced by page, but consuming GPT models directly is always priced by token usage. To compare apples to apples, I modelled an AI usage similar to the “create summaries for all my documents” autofill column scenario, using:
- 600 input tokens (average word density of a Word document is 500-700 tokens from my testing)
- 100 pages per document
- 120 output tokens for the AI response (which corresponds to around 200 words of output, the same as the prompt I used in my autofill column examples above)
No comments:
Post a Comment