Industry Classification of GitHub Repositories Using the North American Industry Classification System (NAICS)

2026-07-07 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

NAICS-GH is a newly released corpus of 6,588 GitHub repositories, each labeled with a 2-digit sector from the North American Industry Classification System (NAICS 2022). This dataset addresses the lack of standardized industry mapping for GitHub repositories, which hinders empirical research on innovation geography and open-source production. The corpus was generated using a retrieve-and-verify pipeline combining BAAI/bge-large-en embeddings, FAISS retrieval, and GPT-4.1 rubric scoring. This pipeline processed approximately 1.37 million source repositories, narrowing them to 31,178 candidate pairs and retaining 6,588 high-confidence labels with a score of at least 8. The pipeline's reproducibility is high, with candidate sets reproducing to within 0.03 percent. Human validation on a 2,421-repository random sample confirmed 96.98 percent precision, with a Wilson 95 percent confidence interval of [96.23, 97.59]. Benchmarking six pretrained encoders on NAICS-GH showed RoBERTa-large achieving 86.45 percent F1 and 86.35 percent accuracy on a held-out 20 percent test set. The dataset, code, and fine-tuned checkpoint are available under CC-BY-4.0 and MIT licenses.

Key takeaway

For data scientists and economists studying innovation or open-source trends, NAICS-GH offers a critical resource for industry-specific analysis. You can now directly map GitHub repositories to 2-digit NAICS sectors, enabling more precise empirical work on technology diffusion and industrial composition. Consider integrating this corpus into your research to gain granular insights into sector-specific open-source contributions and benchmark your own classification models against a high-precision, human-validated dataset.

Key insights

A new corpus, NAICS-GH, provides 6,588 GitHub repositories with 2-digit NAICS labels, enabling industry-specific open-source analysis.

Principles

• Hybrid AI pipelines enhance data labeling precision.
• High-confidence thresholds improve corpus reliability.
• Standardized industry classification aids economic research.

Method

A retrieve-and-verify pipeline uses BAAI/bge-large-en embeddings, FAISS retrieval, and GPT-4.1 rubric scoring to label GitHub repositories with NAICS 2022 sectors.

In practice

• Use NAICS-GH for innovation geography studies.
• Analyze open-source production by industry sector.
• Benchmark encoders on industry-specific codebases.

Topics

• GitHub Repository Classification
• NAICS Industry Codes
• Large Language Models
• Vector Embeddings
• Open-Source Data
• Empirical Economics

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• Accelerating researchers and developers building multilingual AI with a new open dataset — The GitHub Multilingual Repositories Dataset provides metadata to discover non-English content in over 40 million public repositories.
• General Supervised Learning Framework for Open World Classification — A data- and model-agnostic framework automates unknown class identification, categorization, and model updates in open-world classification.
• Bindu Reddy: Navigating the Path to AGI — AI's future lies in specialized open-source models, ethical development, and agentic systems that automate complex tasks.
• All fundamental knowledge in ML Course by Andrew NG that I noted and create into a repo github [R] — Detailed, beginner-friendly notes for Andrew Ng's ML Specialization are available on GitHub.
• Learning the chemical language of natural products — The NaFM foundation model enhances natural product mining by learning their chemical language.
• So you’ve heard these AI terms and nodded along; let’s fix that — This glossary demystifies key AI terms, from AGI to Weights, for technical and professional readers.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.