Infuse AI into your Office 365 apps – three approaches and pricing: Part 1

We’ve all heard how Microsoft are on a mission to democratize AI and bring it to the masses. The last Ignite conference (fall 2019) continued to bang this drum heavily, with several keynote demos featuring AI in interesting scenarios. In fact, Microsoft’s AI democratization journey started back in early 2015 with Project Oxford, a set of APIs which developers could use to recognize faces, perform speech-to-text processing, categorize images and more. Looking back, I remember presenting at Microsoft's Tech Ed 2014 conference showing an extension I created for Word which would use non-Microsoft AI to find similar documents based on the content - so AI has been around for some time. Also making it's debut around this time was LUIS (Language Understanding Intelligent Service), the Microsoft service that a bot developer would typically use to infer intent from some words. All this was great for developers, but true democratization would mean lowering the barrier to entry much further than that.

Fast forward to early 2020, and I think we’re in a different situation altogether. Microsoft’s AI capabilities have moved into well-defined Azure offerings as part of Azure Cognitive Services, and there will have been significant investment in performance, scaling, reliability and accuracy as individual APIs transition into service across the Azure datacenter infrastructure. Along with AWS, Google, IBM and other major cloud providers, Microsoft are out to land-grab as much of the global workload as possible, in order to recoup their infrastructure investments and make their margins work. Competition is fierce, and even if your organization isn't using AI significantly at the moment, it's you they are targeting. 

Alongside this, the Power Platform has emerged as Microsoft’s model for business applications which can be built without professional developer skills. So how easy is it now for an organization using Microsoft cloud tech to use AI, and what profile of person will be able to build such an app? This series of articles looks at several approaches, and also analyzes the pricing to consume AI in each case. Does easier to use AI come with an added cost? Which option is best if the organization *does* have developers or a partner?

This article is part one of a series:

1. AI in Office 365 apps - a scenario, some AI examples and a sample Power App (this article)
2. AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate
3. AI in Office 365 apps - pricing and conclusions

As we think about different approaches, a scenario is useful to base our analysis on - so let's think about that first.

The scenario

Some form of incident or situation reporting is a requirement across many sectors, ranging from monitoring hazards on a construction site, health and safety monitoring in a hospital, or even a store manager submitting evidence of merchandising to head office.So let’s say we’re building an incident reporting app which mobile workers will use in the field on their phones - I use this example frequently with clients, as it combines several ingredients for intelligent data capture apps. We’ve decided that the app itself will be a Power App to avoid costly native app development and for easy distribution to mobile devices, and that the photo and details of the image will be stored in SharePoint Online. AI will be used to detect what’s happening in the image – specifically, we’ll add some metadata to the image in the form of a description and keywords. The keywords will describe objects detected in the image and the overall setting. This is useful, because it could be used to automatically categorize the incident and/or alert different teams – using the health and safety example, if the keywords contained “casualty”, “injury” or “blood”, an alert could be raised immediately to a certain team. Other processing of the incident could also be built-in, depending on what other rules or workflows might be appropriate.

There are a number of ways we could build this app:

• Use of Power Apps AI Builder
• A Power App which talks directly to Azure Cognitive Services (via a Custom Connector)
• A Power App which uses a Power Automate Flow to consume AI services

For each approach, we’ll look at how the app would be built, pricing, and any constraints and considerations which come with this option.

AI image processing - looking at examples

Before we go on to look at implementation approaches and costs, exactly how can AI help in an application like this? Whilst the range of AI tools include text-to-speech (and vice-versa), language translation, pattern matching, data extraction, text processing and various data science related tools, in our scenario it's a form of image processing that is most relevant. Potential benefits we can unlock here include:

• Images (and the associated incidents in our case) become searchable when we have some textual data for them. Unless some interpretation has been performed, any search capability (including Office 365) is unable to determine what the various pixels and colours represent
• Images/incidents can be categorised once the application knows what they relate to
• Some automated 'triage' is possible once we've turned the image into information. Using the example described earlier, if the AI does identify concepts such as "casualty" or "injury" our system would take specific action - even if the process was simply to route these incidents for urgent human processing and/or we accepted that some would be false positives, there could be huge benefits across the a busy system
• AI can easily process large amounts of historic data. So if I already have a repository of existing files where I want to perform some automated image processing (or in the case of other files, pattern identification/translation/text to speech generation etc.), I can do that easily even if I didn't have the capabilities back when the app was first introduced

Overall, there are an almost unlimited array of possibilities here.

Anyway, back to the image processing. Here are some examples - using a couple of images captured with a test Power App I created to perform the above functions and one from the internet. For each one, consider the image and what that the AI detected:

Image	Result
Image	Result
Image	Result

I think there are some pretty amazing results there. Consider some of the objects and concepts being recognized - public transport, subway, conference room, emergency services, police - that's quite a lot of intelligence to be able to successfully detect those contexts, and to have this power available to you within easy reach opens the door to lots of innovative solutions. How could you use it in your organization?

A Power App for incident reporting

So that's what the back-end can do. But on the front-end, our scenario would need a means of capturing the image and reporting the incident. Here's a quick Power App I created to do this - it uses the camera control with Power Apps and allows me to plug in any of the three architectures we're looking at in this series:

In the maker view, it looks like this (featuring me hard at work, because the front-facing camera on my Surface Pro is activated):

Summary

So that lays the groundwork for this series. AI is one of those topics that gets a lot of coverage, but I see lots of organizations struggling to make practical use of it. In these articles my aim is to show some approachable methods which can add real value to a common business scenario, and between the options there are a few variables to consider - capabilities, cost profiles, skillset required to implement to name a few.

Having a good understanding of the options provided to you in the Microsoft stack can help you bring real innovation to your organization or clients. In the next article we'll go through our 3 implementation options in close detail, so you can see what's needed to tap into AI in these ways.

Next article - AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate