Google adds two TPU systems: one sealed, one doubled
TPU 8tx keeps data protected while in use for confidential inference; TPU 8i Max puts two 8i chips on one package for long contexts.
2 minDePIN Compute & BandwidthFresh · 9 Nov
Google added two systems to its eighth generation of Tensor Processing Units on 9 November, making four TPU systems announced in a single year, which the company says is a first. TPU 8t for training and TPU 8i for low-latency reasoning came in April; the new pair, TPU 8tx and TPU 8i Max, are built from the same chips but packaged for two specific demands: hardware-level data security and million-token contexts.
TPU 8tx is aimed at regulated enterprises and governments that want to run frontier or proprietary models, and agents, next to their most sensitive data. Data is normally encrypted at rest and in transit but has to be decrypted in memory to be processed. Built on the TPU 8t chip with fifth-generation AMD EPYC hosts, TPU 8tx keeps data and model weights protected and isolated while in use. It also borrows the low-latency Boardfly network made for reasoning, and Google claims up to 2x lower latency than TPU 8t with a near-zero performance penalty. Teams can fine-tune, distill and run reinforcement learning on private data without moving it between clusters.
TPU 8i Max takes on the memory wall that appears when many agents work over a whole codebase or hours of video. Two TPU 8i chips sit on one package, giving double the compute, double the SRAM and double the HBM bandwidth of TPU 8i. Because twice the silicon fits in the same rack footprint, data centres can place more compute per square foot and per watt.
Google also says TPUs are now a fast-growing business with nearly 100 customers, from frontier labs and trading firms to retailers and universities, and that customers can now deploy TPUs in their own data centres. On the software side it open-sourced TPU Sync, a layer that tiers KV caches off the chip and which Google says cuts KV transfer latency and time to first token by more than 90%, and it made Cloud Pathways and GKE dynamic slicing generally available.
Retold from Google. This is a summary in our own words; follow the link for the original reporting.