Amortized Inference for Correlated Discrete Choice Models via Equivariant Neural Networks

2026-07-09 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A novel amortized inference approach utilizes a neural network emulator to rapidly approximate choice probabilities for discrete choice models, including those with correlated errors like Multinomial Probit (MNP). This method addresses the computational demands of MNP models, which lack closed-form solutions and typically rely on slow GHK simulation. The proposed architecture incorporates group-theoretic foundations, enforcing invariance properties and achieving universal approximation. Employing Sobolev training, the emulator learns both choice probabilities and their derivatives, enabling fast likelihood and gradient computations. Simulations demonstrate significant gains in accuracy and speed over the GHK simulator for MNP models with 3, 5, and 10 alternatives. Training for K=3, 5, and 10 took 2.1, 4.6, and 16.6 hours, respectively, with multi-K training also proving effective.

Key takeaway

For Machine Learning Engineers or Econometricians developing discrete choice models, this amortized inference framework offers a compelling solution to the computational bottlenecks of models with correlated errors. You can now efficiently estimate complex models like Multinomial Probit or correlated Gumbel, which previously required slow simulation methods. Integrate this pre-trained emulator to accelerate likelihood and gradient computations, enabling faster maximum likelihood or Bayesian inference. This allows for exploring richer, theoretically grounded choice behaviors without sacrificing interpretability, provided your application's input distribution aligns with the emulator's training data.

Key insights

An equivariant neural network emulator enables rapid, accurate estimation of discrete choice probabilities for models with complex correlated errors.

Principles

• Discrete choice models exhibit inherent location, scale, and permutation invariances.
• Amortized inference pre-computes complex probabilities via one-time training.
• Sobolev training ensures accurate gradient approximation for model fitting.

Method

Train a DeepSet-based neural network emulator on simulated data, using preprocessing to enforce invariance and Sobolev training to match choice probabilities and their derivatives.

In practice

• Substitute emulator evaluations for GHK simulations in MNP models.
• Extend to correlated Gumbel or multivariate-t error distributions.
• Integrate into gradient-based MLE or Hamiltonian Monte Carlo workflows.

Topics

• Amortized Inference
• Discrete Choice Models
• Neural Network Emulators
• Multinomial Probit
• Sobolev Training
• Permutation Equivariance

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators — Neural operators can universally approximate probabilistic conditioning, enabling amortized inference across diverse joint distributions.
• Deep Doubly Debiased Longitudinal Effect Estimation with ICE G-Computation — D³-Net combines SDR-based training with LTMLE targeting to robustly estimate longitudinal treatment effects, mitigating error propagation.
• Simulation-Based Empirical Bayes — SBEB combines empirical Bayes and simulation-based inference to learn priors for intractable likelihoods.
• py/cuTAGI: An Open-Source Library for Tractable Approximate Gaussian Inference in Bayesian Neural Networks — py/cuTAGI enable analytic Bayesian learning in neural networks by treating all quantities as Gaussian random variables, bypassing gradient descent.
• Probabilistic ML - 22 - Factorization, EM, and Responsibility — Latent variables simplify complex probabilistic model optimization by factorizing joint distributions.
• Equivariance and Augmentation for Bayesian Neural Networks — Data augmentation can achieve exact equivariance in Bayesian Neural Networks under specific conditions, enhanced by novel symmetrization techniques.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.