Skip to content

Conversation

@dsillman2000
Copy link
Owner

@dsillman2000 dsillman2000 commented Feb 7, 2026

Merge Request Summary: Drastically Reduce Features and Improve Deterministic CLI Output

Overview

This MR significantly simplifies the yaml-reference library by removing complex features that added maintenance burden while keeping the core functionality intact. The changes focus on making the library more maintainable and easier to understand, with additional improvements to ensure deterministic output from the CLI tool.

Key Changes

1. Removed Complex Features

  • Anchor references: Removed the ability to reference specific anchors (&anchor) within YAML files
  • JMESPath support: Eliminated JMESPath expression filtering capabilities
  • Round-trip serialization: Removed unresolve() functionality and related serialization features

2. Simplified API

  • Renamed main class: YAML -> YAMLReference for clarity
  • Removed optional parameters: anchor and jmespath parameters removed from Reference and ReferenceAll constructors
  • Simplified error handling: Removed JMESPath-specific import errors

3. Code Reduction

  • Deleted files: yaml_reference/anchor.py (100 lines)
  • Removed code: 567 lines deleted across 12 files
  • Dependency removal: Removed jmespath dependency from pyproject.toml

4. Updated Documentation

  • Updated README to reflect simplified feature set
  • Removed examples and documentation for anchor references and JMESPath functionality
  • Added uv installation instructions

Technical Details

Breaking Changes

  1. Class rename: Users must update imports from from yaml_reference import YAML to from yaml_reference import YAMLReference
  2. Feature removal: Anchor references and JMESPath expressions are no longer supported
  3. API simplification: Reference and ReferenceAll no longer accept anchor or jmespath parameters
  4. CLI behavior change: The CLI tool now uses SafeLoader/SafeDumper for deterministic output, which sorts mapping keys alphabetically

New CLI Deterministic Behavior

The CLI tool (yref-compile) has been updated to use SafeLoader/SafeDumper for consistent, deterministic output:

Implementation Details:

  • SafeLoader: The CLI now uses YAMLReference(typ="safe") for loading, which provides predictable parsing behavior
  • SafeDumper: A separate YAMLRuamel(typ="safe") instance is used for dumping with specific configurations:
    • ignore_aliases = True: Ensures all anchor references are fully resolved in output
    • default_flow_style = False: Forces block formatting for all data structures
    • Automatic key sorting: Safe dumper naturally sorts mapping keys alphabetically

Technical Benefits:

  • Deterministic output: Same input always produces identical output, enabling reliable testing
  • Specification compliance: CLI output serves as authoritative test fixtures for the yaml-reference-specs project
  • Simplified testing: Removed separate CLI test file as behavior is now covered by specification tests
  • Clear contract: The CLI demonstrates exactly how documents are loaded into memory after reference resolution

Example of deterministic behavior:

# Input with references
services: !reference { path: ./services.yml }
networkConfigs: !reference { path: ./networks.yml }
version: '3.1'

# CLI output (keys sorted alphabetically)
networkConfigs:
- network: vpn
  version: 1.1
- network: nfs
  version: 1.0
services:
- website
- database
version: '3.1'

Maintained Core Functionality

  • ✅ Basic file references with !reference tag
  • ✅ Glob-based references with !reference-all tag
  • ✅ Recursive reference resolution
  • ✅ CLI compilation tool (yref-compile) with deterministic output
  • ✅ IDE YAML tag recognition support

Dependency Changes

  • Removed: jmespath dependency
  • Updated: ruamel-yaml version constraint relaxed from <0.19.0 to <0.20.0

Test Suite Updates

  • Removed: Separate CLI test file (test_compile_cli.py) as CLI behavior is now covered by specification tests
  • Enhanced: CLI now serves as the primary tool for testing against the yaml-reference-specs project via the spec-test.sh integration
  • Deterministic: Test outputs are now predictable and consistent due to SafeLoader/SafeDumper usage
  • Integration testing: The CLI's deterministic output enables reliable comparison against expected results in specification tests

Migration Path

Users relying on removed features should:

  1. Anchor references: Extract referenced data into separate files
  2. JMESPath filtering: Perform filtering in application code after loading
  3. Class rename: Update import statements as shown above
  4. CLI output ordering: Be aware that CLI output now sorts mapping keys alphabetically for deterministic behavior

The `YAML` class is renamed to `YAMLReference` and no longer supports
JMESPath queries or anchor references. The `!reference` and
`!reference-all`
tags now only accept a `path` or `glob` parameter respectively.

Dependency on `jmespath` is removed and `ruamel-yaml` version constraint
is relaxed to <0.20.0.
reader/dumper even though it means I have to change a lot of tests
@dsillman2000 dsillman2000 merged commit 646b719 into main Feb 7, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant