Google Tensor SDK Beta with LiteRT

2026-05-18 · Source: Google Developers Blog - AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Internet of Things (IoT) & Connected Devices · Depth: Intermediate, medium

Summary

Google has released the Tensor ML SDK Beta, enabling developers to build and deploy on-device AI experiences on Google Pixel 10 family devices, leveraging the custom Google Tensor System-on-Chip (SoC) and its dedicated Tensor Processing Unit (TPU) inference accelerator. This beta launch introduces two key advantages: a unified developer workflow with LiteRT and access to a Model Garden featuring over 100 classic ML and Generative AI models, including Gemma 3 1B. LiteRT, Google's on-device framework, abstracts vendor-specific SDKs, offering a streamlined API for model conversion, compilation, deployment, and inference on Pixel's TPU. The Model Garden supports diverse applications like small language models, intelligent content creation, vision & understanding, and audio & accessibility features, with precompiled models available on Hugging Face.

Key takeaway

For AI Engineers developing mobile applications, the Tensor ML SDK Beta provides a robust platform for on-device AI. You should explore integrating LiteRT to streamline your model deployment workflow and leverage the extensive Model Garden to accelerate development of features like real-time translation or advanced computational photography on Pixel 10 devices.

Key insights

The Tensor ML SDK Beta unifies on-device AI development on Pixel devices via LiteRT and a comprehensive model garden.

Principles

• On-device ML enhances privacy and real-time interaction.
• Hardware-software co-design optimizes edge AI performance.

Method

Convert PyTorch/TFLite models to optimized binaries using LiteRT Torch, deploy via Play Feature Delivery and AI Packs, then run inference with LiteRT Runtime, specifying CPU/GPU fallbacks.

In practice

• Utilize Function Gemma for local app actions.
• Implement object detection for camera applications.
• Develop secure, low-latency audio transcription.

Topics

• Google Tensor SDK
• LiteRT Framework
• On-device Machine Learning
• Tensor Processing Unit
• Google Pixel 10

Code references

• google-ai-edge/litert-torch
• google-ai-edge/litert-samples
• google-ai-edge/LiteRT

Best for: AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• The Future of AI on the Web: How TensorFlow Powers Browser-Based Machine Learning — TensorFlow.js enables client-side machine learning in browsers, enhancing privacy, speed, and scalability for web applications.
• Building real-world on-device AI with LiteRT and NPU — LiteRT enables efficient, cross-platform on-device AI by unifying NPU, GPU, and CPU acceleration.
• Building Visual ML Pipelines to Python with H2O Driverless AI | Part 22 — H2O platforms enable seamless ML workflow composition, bridging visual development and auditable code generation for diverse working styles.
• What's new in TensorFlow 2.20 — LiteRT replaces `tf.lite` for on-device ML, offering unified NPU support and performance gains.
• Image Captioning with Prodigy & PyTorch — Prodigy's scriptable recipes enable highly customizable, model-assisted data annotation workflows for diverse ML tasks.
• MLCommons Releases MLPerf Mobile v6.0 with New Generative AI Benchmarks for On-Device LLMs — MLPerf Mobile v6.0 introduces on-device LLM benchmarks, standardizing mobile generative AI performance measurement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Google Developers Blog - AI.