Intel has disclosed the compute performance that you would need to run Microsoft’s Copilot locally on Windows-based AI PCs.
Running Microsoft Copilot Locally Would Require At least 40 AI TOPs, Making NPUs A Crucial Component For Intel, AMD & Future Windows-Based PCs
The possibility of running AI engines into local systems might finally come true, courtesy of Intel’s dedicated Neural Processing Units integrated into next-gen CPUs. In a QnA session by Tom’s Hardware at Intel’s AI Summit in Taipei, it was disclosed to them that Microsoft’s Copilot will finally run naively on Intel AI PCs, mentioning a 40 TOPS NPU performance requirement, which is the first time we have seen such a threshold. This marks the era of the adoption of AI PCs by consumers, and it looks like Intel’s NPU tiles will further catalyze the popularity of the new PC standard.
But to your point, there’s going to be a continuum or an evolution, where then we’re going to go to the next-gen AI PC with a 40 TOPS requirement in the NPU. We have our next-gen product that’s coming that will be in that category.
And as we go to that next gen, it’s just going to enable us to run more things locally, just like they will run Copilot with more elements of Copilot running locally on the client. That may not mean that everything in Copilot is running local, but you’ll get a lot of key capabilities that that will show up running on the NPU.
– Todd Lewellen, Intel’s VP of Client Computing Group via Tom’s Hardware
Before you get your hopes up too high, it is essential to note that no current CPU in the market can match the NPU performance requirement, and the closest you can get is with AMD’s Hawk Point APUs, which feature around 16 TOPS with their NPU onboard. Similarly, Intel’s recently-released Meteor Lake SKUs are also way behind the requirement, which means that running Microsoft’s Copilot locally would be an issue for consumers right now. However, Qualcomm’s Snapdragon X Elite SoC pledges to bring 45 TOPS power through its “Hexagon” NPU, which could potentially meet Microsoft’s threshold.
While with future lineups from AMD and Intel, we can anticipate the NPU performance to cross the barrier, but for now, it seems like Qualcomm is indeed in the lead, boasting almost 5 times more NPU performance than its competitors, and this is just their first entry into the CPU markets.
The era of AI PCs is indeed upon us, and with manufacturers racing against each other to integrate the technology’s capability into their product lineup, we will only see the NPU performance needs increase in parallel. Intel has already announced that its Lunar Lake CPUs will offer 3x the NPU AI performance uplift versus Meteor Lake CPUs while Panther Lake will further double that compute performance. AMD is also planning to offer a 3x AI NPU uplift with its upcoming Strix Point APUs.
2024 AI PC Platforms
| Brand Name | Apple | Qualcomm | AMD | Intel |
|---|---|---|---|---|
| CPU Name | M3 | Snapdragon X Elite | Ryzen 8040 “Hawk Point” | Meteor Lake “Core Ultra” |
| CPU Architecture | ARM | ARM | x86 | x86 |
| CPU Process | 3nm | 4nm | 4nm | 7nm (Intel 4) |
| Max CPU Cores | 16 Cores (MAX) | 12 Cores | 8 Cores | 16 Cores |
| NPU Architecture | In-House | Hexagon NPU | XDNA 1 NPU | Movidius NPU |
| Total AI TOPS | 18 TOPS | 75 TOPS (Peak) | 38 TOPS (16 TOPS NPU) | 34 TOPS (11 TOPS NPU) |
| GPU Architecture | In-House | Adreno GPU | RDNA 3 | Alchemist Arc Xe-LPG |
| Max GPU Cores | 40 Cores | TBD | 12 Compute Units | 8 Xe-Cores |
| GPU TFLOPs | TBD | 4.6 TFLOPS | 8.9 TFLOPS | ~4.5 TFLOPS |
| Memory Support (Max) | LPDDR5-6400 | LPDDR5X-8533 | LPDDR5X-7500 | LPDDR5X-7467 |
| Availability | Q4 2024 | Mid-2024 | Q1 2024 | Q4 2023 |
