In previous articles on SharePoint Syntex I've talked mainly about the document processing approach - in this post I'll discuss it's counterpart, form processing. For those following along, my overall set of articles on this theme so far are:
Syntex - document processing
- Project Cortex - training SharePoint Syntex to read your documents like a human - part 1 (classifiers)
- Project Cortex - training SharePoint Syntex to read your documents like a human - part 2 (entity extractors)
Syntex - general
That last article in particular is designed to help you understand the difference between the two models and when to use each one. As you read about Syntex you might form the view that "form processing is for things like invoices and order forms and document understanding is for everything else" - certainly some of the guidance infers this. However, that position is far too simplistic - there are differences in licensing, capabilities, supported file types and more - and you'll want to get this decision right to avoid having to rework AI models. My "tips for choosing" article might be helpful since it has a table of differences and details of licensing aspects to look out for.
But today, we focus on form processing!
Syntex form processing - integrated AI Builder
As the briefest of recaps, Syntex form processing is typically more suited to highly-structured and consistent document formats compared to document processing, that much is true. Since the AI Builder technology within Microsoft's Power Platform is used, there are a few implications to consider:
- To use, AI Builder credits are needed in addition to Syntex licenses (see AI Builder calculator). However, if your org has 300+ Syntex licenses you receive a generous allowance of 1m credits - this more than gets you started
- Supported file types include JPG, PNG or PDF - but not Office files
- Entire tables can be extracted from the document (in contrast to document processing)
- The model is applied via a Power Automate Flow to the SharePoint document library where your documents reside (i.e. where you create the model from) - but there is no easy way to use this in other locations
In short, it's AI Builder conveniently built into SharePoint document libraries - so you don't have to do the integration or somehow pass each document to the model, it's taken care of for you.
There are some other considerations too - see SharePoint Syntex AI - tips for choosing between document understanding and form processing models for those.
Our invoice format
Before we get started on the process, it's worth seeing the format of documents used in this process. Like many classic examples of this type, they are invoices:
Implementing Syntex form processing
The approach followed here can be summarised as:
- Define the information to extract (i.e. teach Syntex what the fields are e.g. "Invoice Reference", "Invoice Date" etc.)
- Add documents for analysis
- Tag documents (i.e. teach Syntex where to find the relevant content in the document)
- Train the model
- Test
- Use in your document library
In your SharePoint document library, find "Automate" > "AI Builder" > "Create a model to process forms":
You'll see this message alerting you that AI Builder credits are needed:
Give your model a name - I'm using "COB invoice" for now. I want a new SharePoint content type to be created with this name so these documents are easily identified and classified amongst any others:
Syntex then begins to create your AI model:
Once the model has been created we define which information within the document we want to extract:
As the image shows, I start specifying some things I want to extract such as:
- The invoice date
- The invoice reference
- The VAT number
I create a new collection of documents for my invoice scenario:
I have some invoices ready to go, so I select those to upload:
Once uploaded I'm ready to analyze!
Once the analysis is complete we move into the tagging phase.
The tagging phase
As you move your mouse, Syntex allows you to highlight portions of the document by drawing boxes around identified pieces of text. By doing this, you map them to the fields you defined at the beginning - these appear in a picker for selection, with a checkbox indicating whether you've already mapped this item. So I move through the document teaching Syntex what is the invoice reference, what is the date, the supplier name and so on.
As you can see, Syntex allows me to pick something as granular as an individual word or even character, or expand to pick a phrase or string of characters. Items with a green border are already tagged:
Tables can also be tagged in this way:
Once I'm done tagging I'm presented with a summary of the model, with a list of the fields I've defined:
We're now ready to move into the training phase.
The training phase
We start by hitting the Train button:
Once the model has been training you can either run a quick test against a new document (not one used for training) or go ahead and publish it to your SharePoint document library:
Let's go ahead and publish the model. Once I have a published version, any subsequent changes will create a draft - this allows me to test things out (and get them wrong) whilst not disrupting the extraction that's already in place.
Once a model is published, we can go ahead and use it:
This makes the model available for use in a Power Automate Flow, and the person using will need to consent to the connections being used:
The resulting Flow looks like this:
If you're interested in the mechanics, the piece that does the extraction is this - the "Predict" action for AI Builder which links to the model we just created:
The results
So let's go back to the invoice format we are using:
When this file is uploaded to SharePoint, initially it's just any old document:
..but then after a couple of minutes the document is correctly identified and classified as a "COB Invoice" and the values I trained the model for are extracted:
Excellent. Now I can drag in many old invoices and have them properly classified and summarised:
..and after a couple of minutes:
Conclusion
Syntex is hugely powerful in automatically unlocking critical data from documents - it doesn't need to be buried inside any more. At the beginning of this series, we discussed how the best research suggests knowledge workers spend 20-30% of their time just searching for information or expertise, and many of us would recognise that having to open many documents to check their contents can contribute to this. As above, I can build SharePoint document library views so that information is readily-accessible or the view is sorted, filtered or grouped according to extracted information.
These benefits go far beyond search and views though. Having my documents correctly identified means that I can apply security and compliance policies to them, for example a conditional access policy which means employees can't print or download sensitive contracts from an unmanaged device, or a retention policy that means a Master Services Agreement is retained for 6 years. Syntex can drive these approaches so that policies are applied by the AI recognising the document, and this can work across documents of wildly varying formats so long as there's some consistency that a document understanding rule can be applied to.
Being able to automatically extract information also means I can build process automation around my documents, for example if something comes in for a certain region or above a certain value, I can route approval processes or notifications accordingly. There are many possibilities here alone.
Ultimately it comes down to classification and extraction, and there are so many possible use cases around CVs, proposals, statements of work, RFPs, employee contracts, invoices, sales/purchase orders, service agreements, HR policies and just about any other document type you can think of. This is democratised AI in action, and it's great to have it so accessible in SharePoint.
No comments:
Post a Comment