Microsoft is racing to add value to these AI tools before customers start asking whether they’re getting a proper return from the extra money they’re spending on this much-type technology.

Copilot versus ChatGPT

Copilot for Microsoft 365 has reached the most customers so far, after the company made it generally available in November. This is a version of the AI assistant that works alongside the company’s suite of business applications such as Word, Outlook, and Teams.

Feedback for the tool has been mixed to leaning positive so far, according to the Microsoft employees who spoke with BI.

There’s are the usual Microsoft problems: Some customers are using older versions of products like the Outlook email service and they expect whiz-bang AI Copilot capabilities to just work with this somewhat aging software.

But the comparisons to ChatGPT keep coming up, Microsoft employees told BI. These sources asked not to be identified discussing private matters.

The “work” version of this Copilot tool uses internal customer data to help provide automated support to employees working on tasks like summarizing meetings.

It queries this sensitive information from sources such as Microsoft’s SharePoint collaboration software. This sometimes means responses are not as quick or thorough as a free and open web-based chatbot like ChatGPT that’s been trained on information from the entire internet.

Customers who say this Copilot doesn’t compare favorably to ChatGPT just don’t understand how the different products work, Microsoft employees told BI.

This was confirmed by Jared Spataro, Microsoft’s corporate vice president of AI for Work, in an interview arranged by the company.

Microsoft’s Copilot tools are built on what the company calls the Azure OpenAI model. This takes OpenAI’s top GPT models and adds additional capabilities.

Related stories

These GPT models provide a broad foundation of knowledge. Copilots essentially sit on top of this and tap private customer data to provide bespoke support in specific work situations. The system naturally has more restrictions, according to Microsoft employees. For instance, it only temporarily accesses internal data, and then deletes it after each query.

Teaching customers how to prompt

Another problem for Microsoft is that users are typically bad at writing prompts, the employees said. These are special instructions that get the most out of AI models and chatbots. There’s even a new job for this emerging skill, known as prompt engineering.

“It’s a copilot, not an autopilot, you have to work with it,” one Microsoft staffer told BI.

The answer an AI tool gives can only be as good as the question asked. “If you don’t ask the right question, it will still do its best to give you the right answer and it can assume things,” this person added.

Microsoft has hired its partner BrainStorm to create training videos to help customers create better prompts for Copilot, one of the employees said. Microsoft and BrainStorm declined to comment on this.

Some Microsoft customers have even created their own internal teams to help train staff on how to make the best use of these AI tools.

Spataro said Microsoft has put a lot of work into providing support for prompt engineering within the Copilot for Microsoft 365 product itself, like FAQs and prompt examples.

“Work” Copilot versus “Web” Copilot

A source of customer confusion is that there’s also a “work” version of Copilot for Microsoft 365 and a “web” version of the tool.

The web-based version generates similar outputs to ChatGPT and runs in a similar way. Meanwhile, the “work” version is the one that uses internal private customer data to provide more bespoke and specific responses.

For instance, a Microsoft customer might use the web version of Copilot to search publicly available information about a client. Then they could switch to the work version to find out what extra information is available about this client from internal corporate data.

Still, customers get confused about why the “work” version doesn’t give responses as quickly or as thoroughly as the web-based Copilot, employees told BI.

Microsoft has been taking measures to help customers understand how these different Copilot offerings operate, Spataro said. For instance, the company is introducing a toggle switch so customers can swap between the “web” and the “work” version of Copilot to help them understand which dataset — the web or their internal files stored within SharePoint — is being queried.

“Copilot for Microsoft 365 is unlike any other AI experience at work, with a deep understanding of your job and organization that combines top of the line AI models, the web, and your business data to enable new scenarios that directly impact the bottom line in a way that wasn’t possible before generative AI,” Spataro said in a statement.

Are you a Microsoft employee or someone else with insight to share?

Contact Ashley Stewart via email ([email protected]), or send a secure message from a non-work device via Signal (+1-425-344-8242).