Any-Dimensional Learning by Sampling

2026-07-08 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, medium

Summary

A new unified approach, "Any-Dimensional Learning by Sampling," addresses how machine learning models generalize from small to larger, unseen inputs and how to efficiently sketch large inputs into smaller ones. This framework, developed by Eitan Levin and Venkat Chandrasekaran, utilizes random sampling maps, including generalizations of sampling with replacement, random binning, and species sampling. It characterizes the appropriate application domains for each sampling type based on problem instance symmetries and relations across different sizes. The approach provides explicit generalization and sketching rates for function classes continuous with respect to the chosen sampling notion, covering large families of functions defined on sequences, graphs, and tensors of varying sizes. Specific examples include moment polynomials on measures, homomorphism densities, permutation-invariant transformers, and graph neural networks.

Key takeaway

For Machine Learning Engineers developing models for variable-sized data, this framework provides a principled approach to address generalization and sketching challenges. You can select appropriate random sampling maps, like species sampling or random binning, based on your problem's symmetries. This enables more robust models that generalize better from small training data to larger unseen inputs, and allows efficient approximation of large inputs for faster evaluation.

Key insights

A unified sampling map framework enables generalization and sketching for machine learning models across varying input sizes.

Principles

• Sampling maps compare different-sized inputs.
• Domain symmetries guide sampling type selection.
• Continuity w.r.t. sampling yields generalization rates.

Method

The approach uses random sampling maps (e.g., sampling with replacement, random binning, species sampling) to compare and approximate inputs of different sizes, characterizing their applicability based on domain symmetries.

In practice

• Evaluate permutation-invariant transformers.
• Analyze graph neural networks.
• Sketch large point clouds efficiently.

Topics

• Any-Dimensional Learning
• Random Sampling Maps
• Generalization Rates
• Input Sketching
• Graph Neural Networks
• Permutation-Invariant Transformers

Code references

• ChenJoya/sampling-free

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Any-Dimensional Learning by Sampling — Random sampling maps unify generalization and sketching for models handling variable-sized inputs.
• Information-geometric adaptive sampling for graph diffusion — DVS adaptively adjusts graph diffusion sampling steps based on information-geometric curvature for improved fidelity and efficiency.
• Stochastic Gradient Optimization with Model-Assisted Sampling — A model-assisted sampling framework reduces stochastic gradient variance by integrating survey sampling theory with ML optimization.
• Separating Geometry from Probability in the Analysis of Generalization — Generalization can be analyzed deterministically via sensitivity of optimization solutions to data perturbations.
• Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration — A unified graph-based framework prunes datasets by modeling samples as nodes and relations as edges, solving a Maximum Weight Clique Problem.
• Meta-Learning Transformers to Improve In-Context Generalization — Training transformers on diverse small datasets improves in-context generalization and modularity over large, uncurated corpora.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.