On Monday, in a new artificial intelligence research paper, Apple revealed that its AI models were pre-trained on Google’s custom chips, hinting that big tech companies like Apple are looking for NVIDIA alternatives in training their AI models.
The technical paper, “Apple Intelligence Foundation Language Models,” disclosed that the company chose Google’s homegrown Tensor Processing Unit (TPU) for training and also gave its detailed description.
The paper hints that Google was using Google’s hardware in early development. Apple said, “This system allows us to train the AFM models efficiently and scalably, including AFM-on-device, AFM-server, and larger models.”
It added that the AFM on-device was trained on a single “slice” of 2048 TPU v5p chips working together, while the AFM server was trained on 8192 TPU v4 chips that were configured to work together as eight slices through a data center network
Although Google TPUs are among the most mature chips, Google is one of Nvidia’s top customers and uses Nvidia’s GPUs in its TPUs for training AI systems.
NVIDIA’s graphics processing units (GPUs) are dominating the market for high-end AI training chips and have been in high demand in recent years, but they are difficult to access due to their pricey nature. This can be a solid reason behind Apple’s move.
Although Apple hasn’t named NVIDIA in the 47-page paper, it did mention that the Apple Foundation Model (AFM) and AFM server are trained on “Cloud TPU clusters,” hinting that they rented servers from a cloud provider to perform the calculations.
However, reports say that Apple training its model on Google-designed hardware doesn’t mean much in the long run. The company said that it derived its hardware from Apple Silicon in its data centers and is reportedly planning to optimize AI applications within its data centers.
