Amazon's AI Chip Ambitions Hit a Snag: Startups Prefer Nvidia's Power
Exclusive Report by Eugene Kim
In the high-stakes world of artificial intelligence, Amazon is betting big on its in-house AI chips to drive the next wave of growth. But here's where it gets controversial: internal documents reveal that startups are finding Amazon's AI chips less competitive than Nvidia's GPUs, raising questions about the tech giant's strategy. And this is the part most people miss: despite Amazon's efforts, Nvidia continues to dominate the market with over 78% share, leaving Amazon's chips struggling to catch up.
Amazon's AWS Trainium chips, designed to be a cost-effective alternative to Nvidia's GPUs, are facing significant performance challenges. According to a confidential Amazon document obtained by Business Insider, AI startup Cohere reported that Trainium 1 and 2 chips underperformed Nvidia's H100 GPUs. Adding to the woes, Cohere noted that access to Trainium 2 was extremely limited and plagued by service disruptions. These issues are still under investigation by Amazon and its chip group, Annapurna Labs, with progress described as 'limited.'
But it doesn’t stop there. Stability AI, a prominent startup in AI image generation, echoed similar concerns. They found that Trainium 2 chips lagged behind Nvidia's H100 GPUs in terms of latency, making them less competitive in both speed and cost. These findings highlight the steep challenge Amazon faces in matching Nvidia's performance and profitability in the AI cloud race.
Amazon's homegrown Trainium chips are a cornerstone of its strategy to compete in the AI-cloud market. The company's early success with Amazon Web Services (AWS) was built on designing its own data-center chips, avoiding the high costs of Intel components. Now, in the generative AI era, Amazon aims to sidestep expensive Nvidia GPUs while still delivering powerful AI services to its cloud customers. However, if AWS customers insist on using Nvidia hardware, it could significantly dent Amazon's future cloud profits.
Here’s the kicker: customer complaints, highlighted internally, reveal that startups—a core market for AWS—are hesitant to adopt Amazon's chips. For instance, Typhoon, another startup, found Nvidia's older A100 GPUs to be three times more cost-efficient than AWS's Inferentia 2 chips for certain workloads. Similarly, AI Singapore determined that AWS's G6 servers, equipped with Nvidia GPUs, outperformed Inferentia 2 across multiple use cases.
These struggles are reflected in market share. While Nvidia dominates with over 78%, AWS's chips hold a mere 2%, ranking sixth behind Google and AMD. Even Amazon's high-profile partnership with OpenAI, valued at $38 billion, exclusively uses Nvidia GPUs, with no mention of Trainium processors. Mizuho analysts called this 'disappointing,' noting that it's 'logical' for OpenAI to rely on Nvidia's superior performance and widely adopted CUDA platform.
But here's the controversial question: Can Amazon's Trainium chips ever truly compete with Nvidia's GPUs? While AWS claims its chips offer 30% to 40% better price performance, investors remain skeptical. Bank of America analysts warned that strong demand for Trainium is 'unclear' outside of Anthropic, Trainium's most high-profile customer. Anthropic, the AI startup behind the Claude models, is expected to deploy over 1 million Trainium 2 chips by year-end. However, Anthropic's recent expansion of its partnership with Google, which offers its own TPUs, raised eyebrows and caused Amazon's stock to slip.
Amazon's spokesperson emphasized the company's commitment to improving its chips and offering customers a range of hardware options. CEO Andy Jassy highlighted AWS's focus on providing 'multiple chip options' rather than replacing Nvidia. Yet, the road ahead is uncertain. As AWS revenue grew 20% to $33 billion last quarter, its slowest pace since 2022, competitors like Microsoft and Google Cloud are gaining ground.
What do you think? Can Amazon's Trainium chips close the gap with Nvidia, or is Nvidia's lead insurmountable? Share your thoughts in the comments below and let’s spark a debate!
