FUTO Swipe: Layout-Agnostic Neural Swipe Decoding

2026-06-24 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, medium

Summary

FUTO Swipe introduces a novel neural swipe decoding approach that overcomes the limitation of traditional decoders being tied to specific keyboard layouts. Developed by David Lee Miller and Aleksandras Kostarevas, this method trains models capable of functioning on any contiguous mobile keyboard layout. The encoder predicts character location and indication during a swipe, with the keyboard layout provided only at inference time, rather than being learned during training. This layout-agnostic design is supported by geometric augmentations applied to both swipe trajectories and keyboard layouts during training, forcing the model to generalize based on gesture characteristics. To facilitate this, the authors released swipe.futo.org, an MIT-licensed corpus containing over 1M donated swipes from more than 12k donor sessions. This approach allows the model to generalize to layouts absent from training, sometimes with higher accuracy, effectively combining algorithmic flexibility with neural model precision. Trained models are publicly available.

Key takeaway

For Machine Learning Engineers developing mobile input systems, FUTO Swipe offers a significant shift by enabling layout-agnostic neural swipe decoding. You can now train a single model to support diverse keyboard layouts without retraining for each, drastically reducing development and maintenance overhead. Consider integrating these publicly available models or adopting their geometric augmentation strategy to enhance your system's adaptability and reduce data dependency for new layouts. This approach improves user experience across varied linguistic and regional keyboard preferences.

Key insights

FUTO Swipe enables neural swipe decoders to operate on any mobile keyboard layout by decoupling layout learning from training via inference-time layout provision.

Principles

• Decouple layout learning from neural swipe training.
• Use geometric augmentations for layout generalization.
• Inference-time layout provision enhances flexibility.

Method

The encoder predicts character location and indication from swipe points. Keyboard layout is supplied at inference, mapping spatial/temporal predictions to key logits. Geometric augmentations on swipes and layouts during training force generalization.

In practice

• Utilize swipe.futo.org for swipe data.
• Apply geometric augmentations to training data.
• Integrate trained models for layout-agnostic input.

Topics

• Neural Swipe Decoding
• Layout-Agnostic Models
• Geometric Augmentation
• Mobile Keyboard Layouts
• Swipe Data Corpus
• swipe.futo.org

Code references

• KevinYuLei/ROAP
• andreagemelli/doc2graph

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• SpanCat with spaCy and Prodigy on real data — Span classification with spaCy and Prodigy enables robust extraction of nested entities from messy OCR data, crucial for complex information retrieval.
• Fuzzy-Geometric Branch-Point Modeling for Structure-Aware Augmentation of Handwritten Chinese Characters — Fuzzy geometry-driven augmentation robustly enhances handwritten Chinese character recognition by preserving structural fidelity.
• Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition — Decoupling spatial and temporal adaptation with adaptive augmentation significantly improves micro-gesture recognition in imbalanced datasets.
• DODO: Discrete OCR Diffusion Models — DODO uses block discrete diffusion to achieve faster, accurate OCR by enabling parallel token generation while maintaining structural integrity.
• NIV: Neural Axis Variations for Variable Font Generation — NIV automates variable font generation by predicting neural axis variations on vector glyphs, generalizing across diverse inputs.
• Decoupled Object-Centric Video Understanding for Generating Robotic Manipulation Commands — Decoupling action recognition from object identification improves robotic command generation from video.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.