Confidential Inference

Inference inside a hardware-isolated environment where data and model weights stay encrypted and inaccessible to the platform operator even while they are being processed. Ordinary encryption protects data at rest and in transit; confidential inference closes the gap in between, when data normally has to be decrypted in memory to be used.

Google made it a product line on November 9, 2026 with TPU 8tx, its third eighth-generation TPU system. It is built on the same silicon as the TPU 8t training chip, paired with 5th Generation AMD EPYC CPU hosts, and built for regulated enterprises and governments running frontier or proprietary models and autonomous agents alongside their most sensitive data. Google says the hardware keeps data and weights protected and isolated even while in active use, and claims up to 2x lower latency than TPU 8t with a near-zero performance penalty, using the Boardfly network built for real-time reasoning. The same system can also fine-tune on private data, distill large models into smaller ones, and train agents through reinforcement learning, without moving sensitive data between clusters.

The honest limits are in what Google did not publish. There is no price, no availability date, no regions and no per-chip specifications — the 2027 timing in trade coverage comes from what Amin Vahdat told Bloomberg, not from Google’s post. And the post does not describe the trusted execution environment or how a customer would verify the isolation holds, so the security claim is currently a vendor statement, not a checkable design. Until that design is published, TPU 8tx is best read as a direction of travel: the first TPU built for confidential work, not yet a system you can buy or audit.

Confidential inference is the inference half of confidential computing, which is older than the AI boom — the new part is the accelerator. CPUs have had trusted execution environments for years; the hard problem was extending the same protection to the chip doing the matrix math, because the model weights and the activations live there. TPU 8tx pairs AMD EPYC hosts with TPU silicon, and Google’s pitch is that the protection costs almost nothing: up to 2x lower latency than TPU 8t, which is a claim about the Boardfly network, not about the security features being free.

The tell is what Google did not publish. No price, no date, no regions, no per-chip figures — and no description of the trusted execution environment or the attestation a customer would use to confirm the isolation actually holds. Until that design is published, treat TPU 8tx as a direction of travel: the first TPU built for confidential work, not yet a system you can buy or audit.

Treating confidential inference as a substitute for access control or data governance. It protects data from the platform operator while it runs; it says nothing about who can query the model or what the model does with what it sees.

• blogIntroducing TPU 8tx and TPU 8i Max ↗Google · 2026

  The announcement: TPU 8tx pairs TPU 8t silicon with 5th Generation AMD EPYC hosts for hardware-level confidential inference, up to 2x lower latency than TPU 8t with a near-zero performance penalty, per Google. No price, date, regions or trusted-execution-environment design were published.