Thursday, 23 January 2025

Autofill columns in Microsoft 365/SharePoint – comparing AI costs to alternatives

As detailed in the last article, “autofill” columns in SharePoint are a very interesting built-in AI capability in Microsoft 365 where LLM/GPT capabilities can be used very easily with your documents. Unlike Copilot, which can’t really be used in bulk across your documents because it’s driven by the end-user, autofill columns could be used to summarise, categorise, or extract info from hundreds or thousands of documents very easily indeed. Pseudo-automation approaches with Copilot do exist such as creating an agent, but not all employees will do that and autofill columns are perhaps the simplest way of tapping into AI to get some of the benefits. Today, autofill columns use GPT-4 Turbo as the LLM and we can expect Microsoft to continue upgrading as newer models become available. 

As a recap, I gave these examples in the last article:

  • Document summaries – the full document summarised automatically in your preferred format (e.g. 3 bullet points, a few sentences etc.)
  • Key takeaways – a summary focused not on the full document, but the conclusions only
  • Key info extraction – identify the client/project/business unit this file relates to
  • Key info extraction – list the people/organisations/concepts/[something else] mentioned in this document
  • Classification – categorise the document, either generically or according to a list you provide (e.g. into “RFP”, “proposal”, “statement of work”)
  • Automated assessment - based on the contents of the document, should this be approved or sent for human review?
  • New info generation – based on the contents of the document, how does it correspond to X?

If more inspiration is useful, my last article shows the results of implementing a couple of these scenarios, showing the prompt used and what the AI did.

Costs - does this cost more, less, or the same as other AI tools I could use?

Experimenting with AI is generally cheap, but to use in real-world scenarios someone somewhere needs to understand the costs and pricing model closely. However, you’ll rarely see comparison tables on different AI approaches and respective costs published by vendors like Microsoft – you’ll see them for AI models, but rarely for overall approaches. It’s not that AI vendors are necessarily being disingenuous, more that there are a few variables involved such as whether your developer/team/partner has the skills to use different approaches – perhaps they can only use low-code or easy to consume AI for example and code development is out of the question.

But things do open up when you can plug into Azure OpenAI directly for LLM models for instance, because you avoid paying the “abstraction premium” for simpler approaches where Microsoft have done more of the work for you. It was ever thus as we say in England, and this is certainly the case for autofill columns.

Hit me with some numbers

Let’s make two comparisons with other ways you could consume AI, focusing on use cases like those above which align well with autofill columns:

  • Syntex document understanding (for extraction or classification scenarios especially)
  • A Copilot agent or Power Automate solution where the document contents are passed to an AI model (most likely by calling into Azure OpenAI)

Comparing with Syntex costs

[Updated February 2025, see the "newsflash" section later in this article] From March 2025, SharePoint autofill columns are priced at $0.005 per transaction (which means a page) - significantly cheaper than Syntex unstructured document processing at $0.05 per transaction, or the pre-built Syntex models at $0.01 per transaction. However, we may see similar cost reductions in Syntex which evens this out following changes to autofill pricing. On the two Syntex approaches, it comes down to whether there’s a pre-built Syntex model that suits your needs (e.g. for simple work with contracts, invoices, receipts, forms, PII detection or language detection), costs here are much lower at $0.01 per transaction (page).

Result – cheaper with autofill (unless Syntex pricing gets adjusted too)

Syntex specialises in extracting key info from documents – but if you want to use AI in some other way and/or you need generative AI and LLMs you’ll need to look elsewhere. So, what about the idea of a Copilot agent or Power Automate solution?

Comparing with Azure OpenAI costs (via a Copilot agent or Flow)

Here’s where things get interesting. If you have the technical skills to create an agent or automation which calls into AI and passes documents to it, you can reduce AI costs *fairly* dramatically - but note the differential has (happily) come down since writing this article - full details of the update are below. This is partly because going directly to AI is always cheaper than going via an “end-user” approach in Microsoft 365, and partly because you can control the AI model used. With autofill columns the model used is down to Microsoft (GPT-4 Turbo is referenced in the documentation), but with a built solution, choosing one of the latest “mini” models rather than full blown GPT-4 Turbo or similar has a dramatic effect.

A cost comparison calls for a quick spreadsheet. Let’s compare autofill columns with using two of the current AI models but going direct:

  • GPT-4o
  • GPT-4o mini

Here are the details of the scenario I’m modelling (see footnote at the end if you want more on this):

  • 1000 documents per month processed by AI (e.g. summarised, analysed etc.)
  • Each doc has 100 pages
  • Assume average word density
  • Azure OpenAI pricing details – USD (to help compare to autofill pricing which is also in $), East US Azure region, non-provisioned capacity

Here’s the calculation and findings (based on pricing in January 2025) – the costs for the scenario per month are highlighted in blue, with the differential to autofill pricing in red:

Newsflash! Significantly reduced autofill costs
Clearly this entire topic was in focus for Microsoft, because soon after publish of this article a 90% reduction to autofill costs was announced - from $0.05 per page to $0.005 per page, starting March 2025. The change would have already been in motion rather than triggered by anything I highlighted, but demonstrates that Microsoft *are* being proactive in getting newer and cheaper AI models behind built-in gen AI services like autofill and Copilot and passing on cost reductions to clients.

*With that said*, the updated calculations below show there can still be a significant differential between autofill and developer-led AI usage. The cost decreases certainly diminish the impact unless you're working at scale, but in some cases that's exactly where the AI value is. See the updated numbers below - what do you think?

 
To summarise:
  • You’ll pay 3.31 times as much to use autofill columns compared to going direct with a similar model (GPT-4o)
  • You’ll pay 55.11 times as much to use autofill columns compared to going direct with a cheaper “mini” model (GPT-4o mini)

The costs for the scenario using each approach are:

  • GPT-4o mini - $9.07 per month
  • GPT-4o - $151.20 per month
  • Autofill columns - $500 per month

Before the update, those numbers were even more striking (at 33x and 551x difference respectively, and a total bill of $5,000 per month using autofill for the scenario) - but even with the February 2025 price reductions, it does still leave something to consider. Over a year, the difference is magnified of course and we’d be talking $109 for the cheapest approach with GPT-40 mini vs. $6,000 for autofill columns.

Result – going direct to Azure AI is (still) significantly cheaper

What do we take from this? Should autofill columns be avoided?

Even with these cost differences, autofill columns could absolutely be the most cost effective and ‘valid’ choice for your AI use case. The key considerations which come into view are:

  • The scale you’re working at/how much AI you need to fulfil it (e.g. how many documents and pages)
  • Possible avoidance of development costs

Autofill columns are ready-to-go AI that the business can use in a self-serve way. Certainly, cost governance measures need to be put in place to avoid bill shock, but there are options for this – at least in terms of someone being notified that spend might go beyond a defined threshold so they can intervene (by way of implementing an Azure budget on the Syntex meters). With other approaches, AI costs using Azure OpenAI are much lower but may require development or solution maker time – and that implementation cost could outweigh the cheaper AI costs, depending on effort required, day rates etc.

Overall, any options which put AI in the hands of the business should be welcomed since that’s where processes are understood and value can be easily created. Just take care with those cost governance measures if you enable any form of Pay As You Go AI.

Summary (and a view as a Microsoft partner/solution provider)

As someone working for a partner delivering services at the forefront of Microsoft AI, it’s interesting to consider the impact of “ready to go AI” in Microsoft 365 on companies like ours. If our clients can use AI without our help (such as autofill columns), will we deliver less AI advisory and implementation work? Will all our efforts to deeply understand the technologies, the factors in using each one, and how to create business value with them be to waste? Well, frankly it’s hard to imagine that based on the realities of getting the value from AI in production scenarios – if anything I think the reverse might be true. What we’ve seen in this analysis (and any other time we look at the “what to use when in AI”) is:

  • You always pay a significant premium for simplified/abstracted AI – whether it’s Copilot, autofill columns, or something else. As a result, a ‘developed’ solution can save £1000s per year for many use cases due to the sheer difference in AI costs
  • It’s a complex landscape where partners can add a lot of value - understanding the factors takes a lot of context and closeness to Microsoft developments
  • Deriving likely AI costs for a use case is not simple – some modelling is required using a deep understanding of Microsoft technology. And cost comparisons differences across approaches are rarely found in the documentation

Nevertheless, the value is out there. While it's certainly true that the right decisions need to be made within a well-run "POC to production" motion, we're seeing more and more AI use cases related to high-value processes where there's a significant cost saving to the organisation. Often there are some accompanying benefits to employees too, and both will only accrue further as time goes on. 

Understanding the right AI technology and approach for the use case is the key of course - no more, no less. 



Footnote - more detail on the cost comparison

For simplicity, I’m considering the AI consumption costs only. Depending on circumstances you may also have some Copilot agent costs or Power Automate costs to consider too.

To compare the AI costs for autofill columns vs. Azure OpenAI, we need to calculate a "per page" cost for both approaches. Autofill columns and 
Syntex models are priced by page, but consuming GPT models directly is always priced by token usage. To compare apples to apples, I modelled an AI usage similar to the “create summaries for all my documents” autofill column scenario, using:

  • 600 input tokens (average word density of a Word document is 500-700 tokens from my testing)
  • 100 pages per document
  • 120 output tokens for the AI response (which corresponds to around 200 words of output, the same as the prompt I used in my autofill column examples above)
These details are shown in the yellow box at the bottom of the Excel sheet and are used in the formulas.

Monday, 6 January 2025

Automate AI prompts across your files - autofill columns in SharePoint Premium/Syntex

As we go through the AI era, Microsoft continue to embed generative AI into the Microsoft 365 user experience in interesting ways. If you’re a Copilot user like me, you’ll most likely appreciate the capabilities in chat and meetings of course, and Copilot in Word and PowerPoint can be powerful for document work - drafting new content, rewording and enhancement, summarisation, identifying key points, and performing analysis on the text are all tasks Copilot can do well. But these are all single document scenarios – one consideration with Copilot is that there’s no way to process lots of documents in bulk, but for some use cases that’s exactly what you want to do. If you could effectively “run the AI” over many documents without having to open each one and type a prompt, this unlocks AI being applied to many processes and workflows. I list some examples below, but to call out one – the idea of having a 3-sentence summary displayed next to each document could be powerful. How many documents would that save you opening per week as you go looking for something? Or what about asking AI for the broad type of document (perhaps from some pre-defined categories), so you had more context than just the filename and location? There are lots of ways AI can be useful with documents in a ‘pre-processing’ way, where the AI has done it’s work before you come to the document.

The realisation is that Copilot itself cannot easily be automated. While the new Copilot Actions capability brings a certain level of this, the concept there is more about a scheduled prompt rather than something to run across your files. In terms of other options, a more technical maker could create a Copilot agent and find a way of looping through lots of files to do something with AI – but that’s not something most employees will do.

Microsoft have thought about this and have introduced a ‘built-in AI’ capability for files in Microsoft 365 – it’s part of the SharePoint Premium/Microsoft Syntex capability and is called autofill columns.

The idea is that you can run an AI prompt across every file in a SharePoint library and have the result stored in a column next to the document. You’re essentially passing each file automatically to a GPT model with the prompt of your choice. This offers a powerful new AI approach in how we manage information – I can foresee that every content management app and platform will have this in the future. As far as I can tell, Microsoft are the first to weave the capability into a core platform in this way.

Here are my top uses for SharePoint autofill columns – many of which you'll notice are forms of metadata generation:

  • Document summaries – the full document summarised automatically in your preferred format (e.g. 3 bullet points, a few sentences etc.)
  • Key takeaways – a summary focused not on the full document, but the conclusions only
  • Key info extraction – identify the client/project/business unit this file relates to
  • Key info extraction – list the people/organisations/concepts/[something else] mentioned in this document
  • Classification – categorise the document, either generically or according to a list you provide (e.g. into “RFP”, “proposal”, “statement of work”)
  • Automated assessment - based on the contents of the document, should this be approved or sent for human review?
  • New info generation – based on the contents of the document, how does it correspond to X?

Of course, all those are quite generic approaches – even more powerful examples can come when dealing with specific document types. You might consider asking AI to extract contract values and start dates from contracts, to analyse the same contracts for specific risks, or to synthesise the key themes from research papers for example. There are lots of interesting possibilities arising here - we’ve had ways of automatically extracting information from or classifying documents before (in the Microsoft world, that’s where Syntex started), but nothing with the capabilities of today’s GPT-based AI models. If you're familiar with Microsoft Syntex, it's fair to see autofill columns as the new way of implementing many AI document understanding solutions in Microsoft 365 - a shift away from the 'machine teaching' Syntex approach of training a model to understand your document and then supplying some positive and negative examples - to something which uses GPT powers to understand the document via your prompt without the need for training. That said, the original approach will still suit some scenarios better and in some cases combining the two might be the best thing - nothing is going away at this point from Syntex.  

Try autofill columns free until June 2025
Like most forms of AI, autofill columns have a cost based on consumption - more on this later. However, until June 2025 Microsoft are giving some free usage to support exploration of the capability - this covers processing of 100 pages per month (e.g. 10 documents of 10 pages each), so doesn't get you too far but will support some free testing. This is baked into the wider promo covering lots of SharePoint Premium/Syntex capabilities - more details on the documentation page at https://learn.microsoft.com/en-us/microsoft-365/syntex/promo-syntex#monthly-included-capacity

Examples of autofill columns in action

As one example, let’s see what happens when we ask SharePoint to automatically summarise a bunch of project statement of work documents. These are real documents from my work at Advania but with client names redacted. In each case, the column on the right is what AI has generated.

Prompt:

“Summarise the project that this document relates to in around 200 words or less. If possible, provide any key dates or expected durations.”

Result:

From this generated summary, I instantly understand the document without having to open it - and at a time when we're creating more documents than ever, that's powerful. Note that while it may seem the summaries are truncated, that’s just SharePoint formatting things to keep the full list in view. Expanding the column shows the full value:

So auto-generated AI summaries work great and can be very powerful, but as per my earlier list you can imagine lots of other prompts too - document classification, extraction, and analysis all have interesting possibilities.

Because autofill columns can work on images too, the capability can be used to solve one of the big challenges with images – search engines don’t work well with them because by default there’s no description or tagging.

Prompt:

“Describe the image, giving as many details of the context as possible.”

Result:

That’s pretty incredible in terms of image recognition (and all down to GPT-4 Turbo’s multi-modal capabilities), with the model even recognising that it can’t fully read the text on the road signs in the first image. Imagine if every image stored in SharePoint got an automatic text description and set of image tags – the impact on searchability alone would be huge.

Some other examples might be:

  • What is the nature of the complaint in this document?
  • Identify the main theme of this piece of event feedback into “optimism”, “constructive”, “appreciation” or “collaboration”
  • Score the agent’s performance in terms of helpfulness and ability to resolve the caller’s issue between 1-5 (with 5 being the highest), and give your reasoning
  • Propose a decision on this customer loan application based on the credit history and other background information in the file

The possibilities are as wide as for generative AI prompts in general. With that in mind, clearly autofill columns can be used for a range of purposes including AI decision-making – so Responsible AI considerations may come to the fore in some usages. It’s another AI tool in the toolbox for the business to use, and leaders may want oversight of the use cases to validate against RAI principles.

I have ideas already! Should I apply autofill columns everywhere in Microsoft 365?

I have some great ideas too, but we all know that AI carries a cost and generally comes with some practicalities to consider. Should you leap forward and run autofill columns across tens of millions of documents in your environment? What are the considerations?

To my mind, a few interesting questions emerge - such as:
  • Does this cost more, less, or the same as other AI tools I could use?
  • How do I control costs?
  • Documents are changing all the time in Microsoft 365 – does the AI run each time a document somewhere is edited and saved? That could cost a fortune!
  • What types of files can the AI run across?
  • Which languages are understood?
  • If I have a have a document library with a million documents in and I create a “summarise this” autofill column, will all the existing files be processed?
I'm going to save the cost analysis and my Excel calculations for a part 2 article on this subject. The headline is that "simplified AI" like Copilot or autofill columns always comes at a cost compared to a built solution which integrates more directly to AI services like Azure OpenAI - whether this abstraction premium makes sense comes down to the use case, and factors like whether you can access development skills, potential user experience differences, and so on. In the AI age, we'll all spend more time considering different options simply because the cost factor can rule some out immediately, unless you're working with tiny amounts of data.

Let's deal with some other practicalities here though.

Practical considerations for autofill columns 


Behaviour (e.g. does the AI run each time a document is edited? What about existing files?)

No, autofill columns will only run the prompt if a user asks for this 'manually'. The option is a bit hidden away in the SharePoint interface, accessed via the ellipsis menu (...) and not on the 'Automate' menu as you might expect. If you have multiple documents selected, it will show as:

When an autofill column is created for the first time, it does NOT run over your existing files automatically. That means creating a column is a safe operation in the sense that nothing is going to happen without someone opting in and asking for autofill to do it's thing - though it certainly is a consideration that, once enabled, if you have a document library with 100,000 files stored it's just a few clicks to run AI each one, and that could be expensive.

But what happens if you want to run the AI over existing files at scale? Microsoft say "bulk processing options for existing library files will be added in a future release".

How does billing work?

You need Syntex pay-as-you-go billing to be set up for your tenant, where Syntex charges appear on the Azure bill for the linked Azure subscription. Costs can then be managed using usual Azure cost management approaches (e.g. implementing an Azure Budget) - the process is found at Configure Microsoft Syntex for pay-as-you-go billing - Microsoft Syntex | Microsoft Learn

What types of files are supported?

Essentially all Office files, images, PDF, and markdown - the full list is:

.csv, .doc, .docx, .eml, .heic, .heif, .htm, .html, .jpeg, .jpg, .md, .msg, .pdf, .png, .ppt, .pptx, .rtf, .tif, .tiff, .txt, .xls, and .xlsx.

Which languages are understood?

Only English for now unfortunately - though Microsoft say other languages will come soon.

Can autofill columns be more than text? Which column types can be used?

Yes, and actually quite a few are supported - text, number, yes/no, choice, and date/time columns are all supported. Unfortunately other types such as managed metadata columns and lookup columns are not supported. See the full list at Overview of autofill columns Microsoft Syntex - Microsoft Syntex | Microsoft Learn

Summary

Autofill columns are something of a sleeper in Microsoft 365 at the moment, lost in the other AI noise around Copilot and agents. They offer a huge step forward though, allowing business users to easily tap into generative AI for their documents and processes by virtue of being woven into the SharePoint interface. This democratised AI will also help organisations sidestep development costs or build effort in many cases too - Microsoft have done the work and woven it into Microsoft 365 and SharePoint in a nice way, so there's essentially no work to start getting the benefits (other than establishing the AI prompt(s) you'll use). However, autofill columns do come with a "simplified AI cost premium" compared to other ways of plugging in GPT capabilities - so while you may save on some dev/build costs, could this be far outweighed by the additional AI costs you'll pay?

In the next post, I'll provide some comparisons to current Azure OpenAI costs for different models (e.g. GPT-4 and GPT-4o) and some Excel modelling. While Microsoft product documentation pages rarely cover angles like this, I'd suggest it's a vital consideration for anyone needing to understand how much the AI bill might be for a certain scenario and how different uses of technology can affect this.   

Further reading: