Cautious optimism for deep parameterized quantum circuits

2021-11-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

A new study reveals that gradient-based Parameterized Quantum Circuits (PQCs) can exhibit the "double descent" phenomenon, where performance on unseen data improves as the model size increases beyond the interpolation threshold. This finding challenges the traditional machine learning view that larger models inevitably lead to degraded generalization. The research provides analytical results, rigorously supported by add-one-in perturbation techniques and spectral properties of random matrices. Numerical experiments on re-uploading PQCs, using datasets like MNIST-1D, Fashion MNIST, and a synthetic multidimensional regression task, consistently observed this predicted double descent behavior. The interpolation threshold was identified at $p=NK$, where $p$ is the number of parameters, $N$ is the training set size, and $K$ is the output dimension. While acknowledging existing trainability obstacles, this work offers cautious optimism for the scalability and generalization prospects of deep PQCs.

Key takeaway

For Research Scientists developing Quantum Machine Learning (QML) models, this research suggests that increasing the number of parameters in Parameterized Quantum Circuits (PQCs) beyond the interpolation threshold ($p=NK$) does not necessarily degrade generalization. You should reconsider the traditional bias-variance trade-off and explore overparameterized PQC architectures, focusing on maintaining trainability. This opens new avenues for scaling QML, but remember that trainability remains a prerequisite for practical benefits.

Key insights

Deep Parameterized Quantum Circuits can exhibit double descent, improving generalization with increased overparameterization.

Principles

• Overparameterization in PQCs can lead to improved generalization.
• Double descent extends to quantum machine learning models.
• Trainability is crucial for realizing overparameterization benefits.

Method

The study combines add-one-in perturbation analysis with random matrix theory to derive and analyze expected risk bounds for PQCs.

In practice

• Consider overparameterized PQCs for potential generalization gains.
• Rescale PQC outputs to enhance optimization convergence.
• Evaluate PQC performance around the $p=NK$ interpolation threshold.

Topics

• Parameterized Quantum Circuits
• Quantum Machine Learning
• Double Descent
• Generalization Theory
• Random Matrix Theory

Code references

• bpcarlos/double_descent_PQCs

Best for: AI Scientist, Research Scientist

Related on AIssential

• Cautious optimism for deep parameterized quantum circuits — Gradient-based parameterized quantum circuits can exhibit double descent, improving generalization with increased model size, challenging traditional views.
• Quantum Machine Learning for Industrial Applications — QML theory is advancing to address classical ML limits, focusing on circuit trainability, expressivity, and simulation resistance for industrial applications.
• Adaptive directional gradients for parameterised quantum circuits — Forward gradient estimators significantly reduce measurement cost for training PQCs by averaging random directional derivatives.
• Benign Overfitting with Quantum Kernels — Local-global quantum kernels enable benign overfitting by combining local and global measurement components.
• The Quantum Leap in AI: Building Hybrid Classical-Quantum Neural Networks — Hybrid classical-quantum neural networks combine classical data processing with quantum classification for advanced AI.
• Rethinking Quantum Continual Learning with Quantum Fisher Information — QEWC uses Quantum Fisher Information to mitigate catastrophic forgetting in quantum continual learning by leveraging quantum-state geometry.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.