ChinAI #343: AI Safety/Security Governance Research Report (CAICT 2025)

2026-01-19 · Source: ChinAI Newsletter · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, AI Governance · Depth: Intermediate, extended

Summary

A China Academy of Information and Communications Technology (CAICT) research report on AI safety and security governance, published in 2025, reveals Chinese scholars' deep engagement with global AI governance developments. The report, authored by affiliates of CAICT, the China AI Industry Alliance, the Ministry of Information and Industry Technology, and the Ministry of Public Security, cites international sources like CISPA Helmholtz Center's "JailbreakRadar" and OWASP's LLM vulnerabilities list. It highlights emerging risks such as "value alignment lapses" and the misuse potential of open-source AI, exemplified by "KawaiiGPT" and "WormGPT 4." The report also details information pollution risks, noting Jinri Toutiao blocked over 930,000 pieces of low-quality AI-generated content in 2024. CAICT's AI Safety benchmark, which consistently updates, shows all fifteen models tested in December 2025 had over a 10% hallucination rate.

Key takeaway

For AI/ML Directors evaluating governance strategies, the CAICT 2025 report underscores the critical need to integrate international AI safety and security research. Your teams should actively track global benchmarks and vulnerability lists, such as OWASP's LLM top ten, to anticipate and mitigate risks like value alignment failures and information pollution. Prioritize robust evaluation of open-source models given their potential for misuse and monitor hallucination rates as a key safety metric.

Key insights

Chinese AI governance research actively integrates global findings on safety, security, and emerging risks.

Principles

• Open-source AI lowers misuse barriers.
• AI capabilities raise "unknown risks."
• Information pollution is a societal risk.

Method

The CAICT AI Safety benchmark evaluates models, with the 2025 Q1 update focusing on hallucination rates across fifteen models.

In practice

• Monitor CAICT's AI Safety benchmark for updates.
• Review OWASP's LLM vulnerabilities list.
• Consider "JailbreakRadar" for new AI security risks.

Topics

• AI Safety Governance
• Large Language Model Vulnerabilities
• AI Hallucination
• Information Pollution
• CAICT AI Safety Benchmark

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Researcher, AI Ethicist, Policy Maker

Related on AIssential

• ChinAI #351: CAICT launches 2026 AI Safety Evaluations — CAICT's AI safety benchmarks reveal significant risks in Chinese AI models, including content security and infinite output vulnerabilities.
• ChinAI #344: AI Safety/Security Governance Report (Part II) — Global AI safety governance shows inconsistent benchmarking and varying approaches to vulnerability reporting.
• OpenAI Offers A New Policy Blueprint — OpenAI's blueprint proposes a federal framework for frontier AI safety, emphasizing CAISI's role in evaluations and risk mitigation.
• Promising Signals on AI Governance from China — China advocates for international cooperation and robust governance frameworks to mitigate AI's existential risks and ensure human control.
• Global Standards, Local Ground Truths: Piloting Multilingual, Multimodal AI Safety Understanding in APAC — AI safety and appropriateness are culturally subjective, necessitating localized ground truths and multimodal understanding.
• RiskNet: A large-scale dataset of AI risk incidents from news with alignment and multi-dimensional annotations — A large-scale, structured dataset of real-world AI risk incidents is crucial for empirical AI safety and governance research.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ChinAI Newsletter.