Work Trend Index Special Report

What Can Copilot’s Earliest Users Teach Us About Generative AI at Work?

A first look at the impact on productivity, creativity, and time

Eight months ago, Microsoft introduced Copilot for Microsoft 365 to reduce digital debt and increase productivity so people can focus on the work that is uniquely human. What everyone wants to know now is: will generative AI really transform work, and how?

New data shows that the productivity gains are real. Microsoft conducted research using a combination of surveys and experiments to deeply understand how Copilot is transforming work—learning with and alongside the customers as Microsoft scale. Already, Copilot makes people more productive and creative, and saves time. Early users don’t want to go back to working without it: 77% said they don’t want to give it up.

And this is just the beginning. As use of generative AI at work spreads, the real opportunity is to not only transform personal productivity but lift the capability of the entire organization. Copilot sets a new baseline—one where every employee gains the skills to write, design, code, analyze data, and more. And it amplifies expertise, taking work from good to exceptional.

Read on to learn what the first in-depth studies of Copilot at work reveal.

First Look
Across Micorosft research, here’s what stood out:

·      70% of Copilot users said they were more productive, and 68% said it improved the quality of their work.
·      Overall, users were 29% faster in a series of tasks (searching, writing, and summarizing).
·      Users were able to get caught up on a missed meeting nearly 4x faster.
·      64% of users said Copilot helps them spend less time processing email.
·      85% of users said Copilot helps them get to a good first draft faster.
·      75% of users said Copilot “saves me time by finding whatever I need in my files.”
·      77% of users said once they used Copilot, they didn’t want to give it up.

1.    Perceived productivity gains of Copilot

Microsoft surveyed 297 users in their Copilot for Microsoft 365 Early Access Program to learn how they are using Copilot—from meetings to drafting documents. Overall, they reported productivity gains and time savings consistent with the gains developers experienced using GitHub Copilot.

Copilot users are more productive—and complete work faster:

·      70% of Copilot users said they were more productive.
·      73% said they could complete tasks faster.
·      64% said Copilot helps them spend less time processing email.
·      85% said Copilot helps them get to a good first draft faster.
·      87% said Copilot makes it easier to get started on a first draft.
·      86% said Copilot makes it easier to catch up on what they missed.
·      84% said Copilot makes it easier to take action after a meeting.
·      75% said Copilot saves time by finding whatever they need in their files.
·      71% said they saved time on mundane tasks.

Copilot improves quality and creativity:

·      68% said it improved the quality of their work.
·      57% said it made them more creative.
·      68% said Copilot helped them jump-start the creative process.
·      72% said it helped them generate ideas while writing.

“It helps [people] get to their ideas faster…It’s not there to replace them, it’s just something they can use to supercharge their ideas.”
—   James Thomas, Global Head of Technology, Dentsu Creative

Copilot is shifting how users spend their time:

·      67% said Copilot saved them time so they could focus on more important work.
·      On average, users reported daily time savings of 14 minutes, or 1.2 hours a week.
·      22% of people said they save more than 30 minutes a day.
·      If users reported saving more than 30 minutes a day, we asked them what they spent that saved time on.
·      The top answer was focus work (53%), and hardly anyone was spending it on administrative tasks such as data entry (7%).
·      However, the second most common answer was using that time for additional meetings (16%)—which could be counterproductive on top of already high meeting loads.
·      These time savings are extraordinary, but it will be key for everyone to reinvest their refunded time wisely.

Copilot makes people more productive and creative, and saves time
Here’s how it breaks down, according to early users

Microsoft also wanted to understand how valuable Copilot is to employees. Early users told that once they’ve worked with Copilot, they don’t want to go back to working without it.

·      77% would not want to give it up.
·      Most people also said they would rather have access to Copilot than a free lunch at work—on a monthly (88%), bi-weekly (79%), or weekly (77%) basis.
·      30% even said access to Copilot would influence their choice of employer.

Is Copilot worth more than a free lunch?
If offered the choice, users overwhelmingly said they would pick Copilot

2.    Copilot’s impact on meetings, email, information search, and writing 

In addition to qualitative research, set out to quantify the productivity gains of Copilot through observational studies across four building blocks of knowledge work: meetings, email, searching for information, and writing. Microsoft knows the digital debt that people feel is real. Our heaviest Microsoft 365 users receive more than 250 emails a day as well as send and receive 150 chats1, and Teams meetings per person are up 3x globally since 20222.

With Copilot, people save time on key tasks
Quantitative findings show Copilot increasing speed on tasks like writing, summarizing a meeting, and searching for information

Study #1: A day in the life

The first study simulated a day in the life of a knowledge worker across a series of tasks. It divided 147 people into two groups, one working with Copilot and one without. Microsoft asked them to do three tasks: search for information across multiple sources, summarize a meeting recording, and write a blog post.

Copilot users completed tasks faster—while preserving quality and accuracy:

·      Users were 29% faster in all three tasks, taking 29 minutes and 42 seconds, compared to 42 minutes and 6 seconds.
·      For the writing task alone, a first draft with Copilot took 8 minutes and 12 seconds versus 13 minutes and 48 seconds.
·      Across all three tasks, there was no statistically significant difference in accuracy.
·      A Large Language Model (LLM) panel also scored the blogs written with and without Copilot. There was no statistically significant difference in quality.

Copilot actually lifts the weight of work:
·      Copilot users don’t just expend less effort—they feel like they do too.
·      85% of users said Copilot reduced effort to complete the tasks.

Study #2: The missed meeting

Here let’s look specifically at Copilot in Teams, where it has seen the heaviest Copilot usage to date. Microsoft split 60 Microsoft employees into two groups, one with Copilot and one without, and asked them to summarize a meeting they missed. The 35-minute recorded meeting included crosstalk, debate, and off-topic discussions to make the task harder and give it the feel of a real meeting.

·      Copilot users summarized the meeting nearly 4x faster (3.8), in 11 minutes and 13 seconds, compared to 42 minutes and 34 seconds.
·      Copilot users felt 2x more productive.
·      Copilot users found the task to be 58% less draining.
·      As expected, we saw a slight decrease in comprehensiveness. Out of 15 meeting details for a perfect summarization score, Copilot users included 11, whereas control users included 12.

“Our people are seeing immediate productivity improvements with Copilot.”
— Kate Johnson, CEO, Lumen Technologies

Study #3: Email effectiveness

Microsoft also asked a 62-person blind panel to rate the clarity and conciseness of multiple email messages, some written with the help of Copilot, some without. Study participants did not know which messages were written with Copilot.

·      Emails written with Copilot were rated 18% more clear.
·      Emails written with Copilot were also rated 19% more concise.
·      Participants scored emails written with Copilot 25% higher on, “Couldn’t have said it better myself.”

“We are seeing meaningful impact in our daily lives; it makes a difference when we are able to find information faster or we are shown relevant content. We are more agile, Microsoft Copilot is helping us to have the right information at our fingertips.”
— Sarah Lewandowski, Global Technology & Innovation Lead, Bayer
Study #4: The strain of searching

Microsoft asked 163 participants, 80 with Copilot and 83 without, to replicate the task of pulling together information from multiple sources—searching for information across files, emails, and calendars. Microsoft asked the subjects to imagine they were joining the procurement department of a fictional company. They were tasked with retrieving information about company procurement policy (requiring searching through company files), about what kind of help their fictional manager needed (requiring searching email), and about the timing of two upcoming meetings (requiring searching a calendar).

·      Copilot users were 27% faster, finishing in 17 minutes and 54 seconds, compared to 24 minutes and 18 seconds.
·      There was no statistically significant change in accuracy between the two groups.

3.    Role-specific pain points and opportunities 

Microsoft sees the broader opportunity for Copilot to drive organizational impact at scale across every function. It’s taking a learn-it-all approach, starting with our own salespeople and customer service agents.

Sales
Microsoft surveyed 133 Microsoft salespeople who use Copilot for Sales at least weekly. Here’s what they found:

On average, users reported saving 90 minutes a week.

·      83% said Copilot for Sales makes them more productive.
·      79% said it reduces the amount of administrative work they have to do.
·      78% said it helps them stay in the flow of work.
·      73% said they can complete tasks more easily.
·      68% said they can keep their Customer Relationship Management (CRM) systems up to date with less effort.
·      67% said they are able to spend more time with their customers.
·      64% said it allows them to better personalize customer engagements.
·      58% said they find CRM information they need faster.
Copilot for Sales gives sellers a better way to work
In a survey, Microsoft’s own sales team reported on key benefits. Here’s what they said:

Customer Service

In another study, Microsoft evaluated how Copilot in Dynamics 365 Customer Service has impacted agent productivity between April 2023 and July 2023 in Microsoft’s own Customer Service and Support (CSS) team, one of the largest customer service organizations in the world. They studied two groups: 6,500 agents who used Copilot and a control group of 5,000 agents who did not use Copilot.

·      Agents with Copilot had a 12% reduction in time spent resolving a case.
·      10% of cases that normally require collaboration with peers were resolved independently when agents had access to Copilot—meaning fewer customers had to be put on hold.

Security
There’s also an urgent need for Copilot to help bridge the skills gap for the cybersecurity workforce, with 3 million unfilled positions worldwide as cyber threats increase in frequency and severity. In one study, they looked at how Copilot for Security can help defenders, regardless of their expertise level. Microsoft compared the performance of 149 new-in-career security analysts divided into two groups, one with Copilot and one without. Both groups were asked to identify scripts used by cyber attackers, create incident reports, and identify appropriate remediation steps.

Results showed:

·      Copilot users were 44% more accurate and 26% faster across all tasks.
·      86% said Copilot for Security helped them improve the quality of their work.
·      83% said Copilot reduced the effort needed to complete the task.
·      86% said Copilot made them more productive.
·      90% said they want Copilot next time they do the same task.

Transforming business process with AI

Microsoft surveyed 18,100 people in 12 countries across six key functions to better understand the broader opportunity for generative AI to fundamentally transform business process. Overall, people told us they are drowning in digital debt. They estimate spending more time searching for information (27% of their day) than creating (24%), communicating (24%), or consuming it (25%). And they say only half (50%) of the information they consume each day is necessary for their job.

Microsoft asked people where generative AI could have a positive impact on job performance across all six functions. Topping the list was “finding the information I need.” From there, they saw unique needs emerge by function.

·      Salespeople cited identifying sales opportunities (75%) and unifying marketing and sales data (74%).
·      Customer service agents cited intelligently routing issues to appropriate agents (70%) and detecting trends across agent-customer interactions (68%).
·      In finance, people cited simplifying financial reporting (73%) and validating data quality (72%).

Global Survey: How AI will impact specific roles
Here’s how sales, customer service, and finance professionals say AI would improve their performance the most

While these are early findings, over time Microsoft expects Copilot to result in more deals closed, more customer service cases resolved, and happier customers—the hard measures that not only make work more satisfying and fulfilling but that ultimately drive the bottom line.

The Path Forward
The data shows that the personal productivity gains from Copilot are real and significant. But moving beyond individual experimentation to build an AI-powered organization will require committing to work in a new way. Those who treat Copilot not as a search engine but as a highly capable, infinitely patient, always available assistant will gain the most.

Build a daily habit with Copilot. Like exercising or mastering a new language, realizing Copilot’s productivity gains will take intentional everyday practice. Those who start building the habit early will pull ahead.

Think like a manager. Tapping into Copilot’s full potential requires the ability to delegate, exercise good judgment, and apply subject-matter expertise—the essential human skills for a machine age. Every employee will need to manage their new assistant and know when to delegate a task to AI and when to apply their own unique human intelligence.

Don’t squander reclaimed time. One of the new leadership imperatives is to help every employee be intentional about the time they save with Copilot. Managers can spend more time coaching and caring for their teams. Salespeople can devote more of their workday to building relationships with customers and closing deals, and marketers can carve out time to dream up new products. Going forward, everyone will have more energy for the rewarding work that has the most impact and value for the organization.

Source: Work Trend Index Special Report | Microsoft