Subject to: Jorge Nocedal

2026-03-18 · Source: Subject to · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Mathematics & Computational Sciences · Depth: Advanced, extended

Summary

Jorge Nocedal, a distinguished professor at Northwestern University, shares insights into his early life, career, and contributions to optimization. Born in Mexico City in 1950, Nocedal's multicultural upbringing included a German education and exposure to indigenous Mexican culture. He recounts a near-death experience at age nine and his early interests in math, physics, and architecture. Nocedal's career began with an unexpected promotion to industrial manager at 19, followed by studies at UNAM and a PhD at Rice University. He is renowned for developing the Limited-memory BFGS (LBFGS) method in 1978, a significant advancement in nonlinear optimization, and co-authored the highly cited "Numerical Optimization" book with Steve Wright. His work also includes a major practical impact on weather forecasting through variational assimilation and the development of the KNITRO software package. Nocedal reflects on the evolution of optimization, its interplay with machine learning, and the challenges of academic publishing.

Key takeaway

For AI and Research Scientists navigating complex optimization problems, Nocedal's experience underscores the value of deep theoretical understanding combined with practical application. Your work on seemingly "failed" ideas can be foundational for future breakthroughs, as seen with LBFGS. Consider how architectural changes can simplify optimization in machine learning, and don't shy away from interdisciplinary projects like weather forecasting, which can yield immense real-world impact. Actively seek out collaborations and diverse intellectual environments to foster innovation and career growth.

Key insights

Nocedal's journey highlights the impact of diverse experiences and persistent exploration in shaping foundational contributions to optimization.

Principles

• Failures are stepping stones to breakthroughs.
• Interdisciplinary collaboration drives major impact.
• Curiosity and exposure are crucial for career growth.

Method

The development of LBFGS involved identifying the limitations of existing methods, simplifying the approach, and rigorous testing. For weather forecasting, a multi-level Gauss-Newton method with spectral preconditioning was key.

In practice

• Prioritize understanding problem structure for algorithm design.
• Embrace continuous learning and new course development.
• Seek out diverse networks and opportunities.

Topics

• Nonlinear Optimization
• LBFGS Algorithm
• Numerical Optimization
• Machine Learning Optimization
• Variational Data Assimilation

Best for: AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, AI Student

Related on AIssential

• Subject to: Tamás Terlaky — Tamas Taki's career highlights the evolution of optimization, from early linear programming to modern quantum computing applications.
• Subject to: Bissan Ghaddar — Resilience and adaptability are crucial for navigating complex academic and professional paths, especially in optimization.
• Calmcode, Explosion, Data Science — Effective data science prioritizes problem rephrasing, system thinking, and community engagement over solely optimizing algorithms.
• The Non-Optimality of Scientific Knowledge: Path Dependence, Lock-In, and The Local Minimum Trap — Science often converges to local optima due to historical, cognitive, formal, and institutional lock-in.
• "Something Big is Happening." This quarter has a start-up vibe: LLM experiments, great projects, and new optimization material — Rapid AI advancements necessitate immediate curriculum retooling to provide students with practical, hands-on LLM experience.
• Weak-Link Optimization for Multi-Agent Reasoning and Collaboration — Optimizing weak links in multi-agent systems enhances overall robustness and stability more effectively than solely reinforcing strengths.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Subject to.