# I Built a System That Scores Oil Company Climate Claims Against NGO Evidence.

2026-05-30 · Source: NLP on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, long

Summary

Claimify ESG is a RAG pipeline designed to score corporate climate claims from sustainability reports against a curated corpus of NGO evidence, revealing that only 3 out of 710 climate pledges made by major oil companies in 2021 were kept. The system automates the auditing of complex sustainability reports, providing traceable reasoning for each verdict. Its five-stage pipeline involves PDF ingestion, NLP filtering with ClimateBERT and GPT-4o for claim extraction, two-stage retrieval using SBERT and a cross-encoder, LLM scoring with GPT-4o-mini, and materiality adjustment. This approach achieved 86.7% accuracy on a 60-claim hand-labelled evaluation set, demonstrating a robust method for distinguishing falsifiable claims from aspirations and grounding verdicts in specific evidence.

Key takeaway

For ESG analysts or AI engineers building auditable compliance systems, this RAG pipeline offers a blueprint for robust claim verification. You should adopt a multi-stage approach, pre-filtering irrelevant text and explicitly separating classification logic to improve accuracy and reduce costs. Grounding LLM verdicts in specific, retrieved evidence ensures traceability, allowing you to provide concrete, defensible scores for complex corporate statements rather than relying on vague sentiment analysis or manual review.

Key insights

A RAG pipeline can provide traceable, evidence-backed verdicts for corporate climate claims, overcoming limitations of keyword matching or manual review.

Principles

• Ground LLM outputs with curated domain evidence.
• Pre-filter irrelevant data to optimize cost and speed.
• Separate complex classification into explicit steps.

Method

The pipeline ingests PDFs, filters non-climate sentences with ClimateBERT, extracts structured claims using GPT-4o, retrieves top-5 evidence passages via SBERT and a cross-encoder, scores claims with GPT-4o-mini, and adjusts risk scores by category materiality.

In practice

• Use ClimateBERT (distilroberta-base-climate-detector) for local climate-relevance filtering.
• Implement a two-step LLM prompt for nuanced claim classification.
• Chunk long source documents into ~350-word segments with overlap for RAG.

Topics

• RAG Pipeline
• ESG Reporting
• Climate Claims
• LLM Classification
• Corporate Sustainability
• Semantic Search
• PDF Processing

Best for: AI Engineer, Data Scientist, Consultant

Related on AIssential

• Baseline Enterprise RAG, From PDF to Highlighted Answer — A minimal RAG pipeline can provide verifiable, sourced answers by structuring outputs and linking directly to document evidence.
• Grounded in Law: A Multi-Stage Anti-Hallucination Pipeline for Legal RAG Systems in Brazilian Portuguese — A multi-stage RAG pipeline significantly reduces legal citation hallucinations in Brazilian Portuguese LLMs.
• Building a RAG System That Knows When It’s Wrong — A robust RAG system must know when to refuse answers to prevent confident, silently false generations.
• Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety — Combining LLMs with dynamically updated Knowledge Graphs enhances trustworthiness and verifiability in safety-critical aviation analytics.
• LLM Summarizers Skip the Identification Step — LLM summarization requires an "identification step" to prevent unsupported claims by linking outputs to verifiable source evidence.
• Claim-Selective Certification for High-Risk Medical Retrieval-Augmented Generation — High-risk medical RAG benefits from claim-selective certification to manage mixed evidence and ensure auditable, risk-calibrated responses.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP on Medium.