Research: Literature & Standards References (2024-2026)

[PipFoweraker]

Purpose

Centralized reference list from landscape survey (Jan 2026). These inform the design and should be incorporated into references.md and cited in the LW/AF publication.

---

Model Cards & AI Documentation

Academic Papers

• Mitchell et al. 2019 - Model Cards for Model Reporting (arXiv) - foundational paper
• AI Cards (APF 2024) - Machine-readable AI/risk documentation for EU AI Act (Springer, arXiv)
• Policy Cards (2025) - Runtime governance for AI agents (arXiv)
• Automated Model Card Generation (NAACL 2024) - LLM-based card generation (arXiv, GitHub)

Industry Implementations

• Anthropic System Cards - Claude model documentation (Hub, Claude 4)
• NVIDIA Model Card++ - Extended transparency fields (Blog)
• Red Hat AI System Cards - Security/governance extension (Blog)
• HuggingFace Model Cards - Most widely adopted (Docs, Guidebook)

---

Standards & Schemas

Machine-Readable Formats

• Croissant (MLCommons) - JSON-LD dataset metadata (Site, Spec, Paper)
• CycloneDX ML-BOM - Supply chain BOM with ML support (Site, GitHub)

Management Systems & Certification

• ISO/IEC 42001:2023 - AI Management Systems (ISO, BSI)
• NIST AI RMF - Risk Management Framework (NIST)

---

Regulatory

EU AI Act

• Article 11 - Technical Documentation (Text)
• Annex IV - Documentation Requirements (Text)
• Implementation Timeline - Feb 2025 (bans), Aug 2025 (GPAI), Aug 2026 (full), Aug 2027 (grace period ends)
• National Implementation Plans (Overview)

---

Evaluation Science

Core Resources

• Apollo Research - "We Need a Science of Evals" (Blog)
• Apollo Research - "The Evals Gap" (Blog)
• Apollo Research - Opinionated Evals Reading List (Blog)
• METR - Frontier model evaluations (Site)
• UK AISI Inspect Evals (Site)
• HuggingFace OpenEvals Guidebook (Space)

Papers

• Observational Scaling Laws (Ruan et al. 2024) - predictive eval methodology
• Evaluating AI Evaluation (arXiv)

---

Tooling & Infrastructure

• huggingface_hub - Model card creation/validation (Docs)
• Croissant Editor - Visual JSON-LD editor for datasets
• Croissant + MCP (Blog)

---

To Investigate Further

• IEEE P2894 - AI Safety Standards (status?)
• OECD AI Principles implementation guidance
• Singapore Model AI Governance Framework updates
• Partnership on AI documentation recommendations
• AI incident databases (AIID, OECD) - relationship to model cards

---

Action Items

• Update references.md with categorized links
• Add BibTeX entries for academic citations
• Identify gaps requiring additional research

[PipFoweraker]

Tooling Research Additions

Schema Integration Points (Priority Order)

1. JSON-LD + schema.org extension - like Croissant; enables semantic queries, search engine compatibility
2. HuggingFace model-index - existing adoption for eval results; Papers with Code integration
3. CycloneDX ML-BOM - supply chain context; modelCard field exists
4. SPDX 3.0 AI Profile - regulatory alignment; ISO standard backing

Existing Tools to Integrate With

Tool	Use Case	Integration
huggingface_hub	Card creation/push	Export to HF format
mlcroissant	Dataset metadata	Link dataset provenance
cyclonedx-python	Supply chain BOM	Embed card in ML-BOM
MLflow / W&B	Experiment tracking	runHash evidence links
DVC	Data/model versioning	Git-native provenance

Key Schema Fields Missing Everywhere

These should be required in interrogatory cards:

performanceClaims[].datasetRef     # versioned dataset link
performanceClaims[].evalScriptRef  # commit hash of eval code  
performanceClaims[].runHash        # reproducible run identifier
limitations[].worseContext         # where model underperforms baseline
safetyTesting[].domain             # jailbreaks, autonomy, persuasion, cyber, bio
safetyTesting[].methodology        # link to red-team protocol
attestations[]                     # third-party verification

Reference URLs

• HuggingFace Model Cards: https://huggingface.co/docs/hub/model-cards
• Croissant Spec: https://docs.mlcommons.org/croissant/docs/croissant-spec.html
• CycloneDX ML-BOM: https://cyclonedx.org/capabilities/mlbom/
• SPDX 3.0 AI Profile: https://spdx.github.io/spdx-spec/v3.0/model/AI/
• TF Model Card Toolkit: https://github.com/tensorflow/model-card-toolkit

[PipFoweraker]

Evals Science Research Additions

Key Organizations

Organization	Focus	URL
METR	Dangerous capabilities, autonomous agents	https://metr.org
Apollo Research	Deception, scheming, alignment faking	https://apolloresearch.ai
UK AISI	Government eval framework, Inspect	https://gov.uk/.../ai-safety-institute
HELM (Stanford)	Standardized evaluation	https://crfm.stanford.edu/helm/
Eleuther LM Harness	Open-source eval framework	https://github.com/EleutherAI/lm-evaluation-harness

What Makes Eval Claims Falsifiable

From synthesis of evals science discourse:

1. Reproducibility artifacts - model hash, exact prompts, sampling params, random seeds, eval code commit, dataset version
2. Statistical rigor - confidence intervals, multiple runs, significance tests
3. Methodology transparency - what capability claimed, threat model, elicitation attempts, negative results
4. Adversarial robustness - was eval methodology itself reviewed? alternative elicitations tried?

Gaps Interrogatory Cards Should Fill

Currently missing in most model cards:

• Eval code links → evalScriptRef (MUST)
• Run hashes → runHash (MUST)
• Negative results → "worse than baseline context" (MUST)
• Elicitation documentation → consider adding: "What elicitation strategies were attempted?"

Key Papers

• Shevlane et al. (2023) "Model evaluation for extreme risks" - https://arxiv.org/abs/2305.15324
• Liang et al. (2022) "HELM" - https://arxiv.org/abs/2211.09110
• Hubinger et al. (2024) "Sleeper Agents" - https://arxiv.org/abs/2401.05566

Benchmark Saturation Problem

Models quickly saturate benchmarks (MMLU, HumanEval, GSM8K all hit >90% within ~2 years). Solutions being explored:

• Private held-out sets (trust problem)
• Dynamic/procedural generation
• Real-world grounding (SWE-bench, WebArena)
• Expert-level tasks (GPQA, FrontierMath)

Implication for interrogatory cards: Claims about benchmark performance should note benchmark saturation status and whether private held-out variants were used.

[PipFoweraker]

Model Cards Literature Additions

Formal Standards

IEEE 2936-2024: Standard for Model Card [VERIFY - significant if confirmed]

• First formal IEEE standard specifically for model cards
• URL: https://standards.ieee.org/ieee/2936/10791/
• Based on Mitchell et al. 2019 framework
• Should review for alignment/differentiation

What's Working in Current Practice

1. Standardization momentum - HuggingFace adoption created de facto norms
2. Metadata machine-readability - YAML frontmatter enables filtering/search
3. Evaluation results format - Papers with Code integration
4. Community adoption - thousands of models now have some card

What's Failing (Gaps We Address)

1. Quality variance / "checkbox compliance" - boilerplate, copy-paste
2. Selective disclosure - emphasize strengths, generic limitations
3. Missing provenance - performance claims without reproducible eval scripts
4. Lack of operational information - model vs deployed system gap
5. Stale documentation - no changelog requirements

Key Industry System Cards

Provider	URL Pattern	Notable Features
Anthropic	anthropic.com/system-cards	RSP evals, dangerous capabilities
OpenAI	cdn.openai.com/papers/gpt-4-system-card.pdf	Red-team methodology
Meta	Bundled with Llama releases	System-level focus

Domain-Specific Extensions

• Medical AI: FDA SaMD guidance, TRIPOD-AI reporting guidelines
• Financial Services: SR 11-7 model risk management (predates ML)
• Autonomous Vehicles: ODD specifications

Additional Standards Bodies

• Partnership on AI ABOUT ML: https://partnershiponai.org/workstream/about-ml/
• ISO/IEC 5338: AI System Life Cycle Processes (under development)