Amazon's new AI-training chip

On Tuesday, Amazon unveiled its new AI-training chip, called Trainium 2. They will use the hundreds of thousands of chips to create a giant supercomputer, which should be ready for use sometime next year. The first user of the supercomputer will be Anthropic, an AI startup which Amazon has invested $8 billion.

On top of the supercomputer, Amazon is linking 64 Trainium 2 chips together in a very efficient configuration, which they call an ultraserver. These will be rented out on AWS to companies looking to train AI programs. One of these users is Apple, who has been a beta-tester of the new chips. According to Apple, they have seen up to 50% efficiency savings by switching to Trainium 2 chips.

In the fast-growing world of AI, companies are having to find a balance between chip costs, availability, performance and heat generation. On top of their new Trainium 2 chips, Amazon still gives their customers access to Nvidia chips. It is worth noting that many AI programs are designed to run on Nvidia infrastructure, so using a competitor's chips means software engineers need to spend time making changes to the programs. Leaving Nvida infrastructure isn't a simple switch.

We are happy to see Amazon innovating to keep AWS useful and relevant in our fast-changing world.