Google says TPU 8tx keeps data and model weights protected while in use; TPU 8i Max puts two TPU 8i chips in one package to double compute and memory bandwidth.
Google on Monday expanded its eighth generation of Tensor Processing Units with two systems, the first time it has introduced four TPU systems in a single year. TPU 8tx, built on the same silicon as TPU 8t and paired with fifth-generation AMD EPYC host processors, is designed for confidential inference, keeping sensitive data and model weights isolated even while they are being processed, Google said. Google did not say in the post how that isolation is attested or who can verify it. It pitches it at regulated enterprises and governments. ServeTheHome reported that Google selected AMD EPYC Turin hosts to match TPU 8t’s PCIe Gen5 x16 interface, and that Google said users should keep confidential inference and serving on TPU 8tx if data must remain in the confidential computing domain. TPU 8i Max combines two TPU 8i chips in a single package, doubling compute, on-chip memory (SRAM) and high-bandwidth memory bandwidth for workloads with very large context windows, Google said. Google says TPUs now have nearly 100 customers. It also pointed to software and tools aimed at making TPUs easier to use from PyTorch and other open-source software.
Sources: Google · ServeTheHome · Reuters (via CNA)