142 - Science Of Science, with Kyle Lo

2023-12-28 · Source: NLP Highlights · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, extended

Summary

Kyle Lo, a lead scientist at the Allen Institute for AI (AI2) Semantic Scholar team, discussed the "Science of Science" (SciSci) on the NLP Highlights podcast. SciSci is a computational sociology field that uses large-scale methods to study how science is conducted, including community interactions, idea transmission, publishing, and funding influences. Lo emphasized that SciSci helps inform the development of tools that genuinely benefit scientists and the broader scientific community, rather than just individual users. He highlighted his team's contributions, such as SciBERT for NLP, S2ORK for open data, and PaperMage, which recently won a Best Paper Award at EMNLP 2023, for manipulating scientific documents. The discussion also covered challenges like data access and the slow iterative loop in scientific data analysis, suggesting NLP's role in creating intermediate artifacts and efficient tools.

Key takeaway

For AI scientists and researchers developing tools for the scientific community, understanding SciSci principles is crucial. Your efforts should focus on building systems that address systemic barriers and promote equitable participation in science, rather than solely optimizing for individual scientist convenience. Consider contributing to open data initiatives and developing efficient, scalable NLP tools that shorten the iterative loop of scientific inquiry, ensuring your work fosters broader scientific progress.

Key insights

SciSci uses computational methods to study scientific processes, guiding tool development for community-wide benefit.

Principles

• Understanding science's process informs effective tool building.
• Large teams develop science, small teams deconstruct it.
• Field-specific SciSci studies offer more actionable insights.

Method

SciSci involves rapid iteration: posing a question, analyzing data, implementing solutions, and refining models, often leveraging NLP for information extraction and summarization at scale.

In practice

• Utilize open-access scientific literature datasets like S2ORK.
• Employ tools like PaperMage for structured PDF parsing.
• Use Spectre embeddings for scientific document similarity.

Topics

• Science of Science
• Scientific NLP
• Open Scientific Data
• Large Language Models
• Scientific Document Processing

Best for: AI Scientist, AI Researcher, NLP Engineer, Research Scientist

Related on AIssential

• I am worried by NLP research culture — NLP research culture in 2026 prioritizes publication volume over scientific rigor, fostering fraud and limiting openness.
• What I’ve learned from 25 years of automated science, and what the future holds: an interview with Ross King — AI-driven automated science and DNA computing offer pathways to dramatically accelerate scientific discovery and address global challenges.
• Catalyzing scientific impact through global partnerships and open resources — Open science, driven by global partnerships and shared resources, accelerates high-impact discoveries across diverse scientific disciplines.
• Mar 23, 2026ScienceVibe physics: The AI grad student — AI can significantly accelerate expert-guided theoretical physics research, but requires rigorous human oversight.
• The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers — AI research documentation practices improved significantly from 2014-2024, driven by a broader open science movement.
• AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation — AI tools are increasingly enhancing the scientific research lifecycle from discovery to peer review.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP Highlights.