[WIP] Refactor/output section

[kellrott]

Updating the playbook file format so that there is one common output section, rather then emit methods in various pipelines

[Copilot]

Pull request overview

Refactors the playbook format toward a single, common output section and shifts templating/configuration from config.* to params.*, while aligning extractor/output naming and updating documentation/examples accordingly.

Changes:

• Rename extractor keys (e.g., tableLoad → table, jsonLoad → json, etc.) and standardize extractor field naming (input → path).
• Replace Configurable discovery (GetConfigFields) with parameter requirements (GetRequiredParams) and update template context to {{params.*}}.
• Refresh docs/examples and remove generated Hugo HTML/XML artifacts from docs/.

Reviewed changes

Copilot reviewed 95 out of 231 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
extractors/transpose_load.go	Renames loader field and updates required-param discovery; adjusts table read call.
extractors/tabular_load.go	Renames loader field and required-param discovery; tweaks error-handling block.
extractors/sqlite_load.go	Switches config-field discovery to required-param discovery.
extractors/sqldump_step.go	Renames loader field and required-param discovery.
extractors/plugin_load.go	Switches config-field discovery to required-param discovery.
extractors/json_load.go	Renames loader field and required-param discovery.
extractors/interface.go	Renames extractor JSON keys and switches to required-param discovery aggregation.
extractors/glob_load.go	Renames glob input field, nested extractor keys, and required-param discovery.
extractors/avro_load.go	Renames loader field and required-param discovery.
examples/vcfload.yaml	Removes outdated example playbook.
examples/hugo-ensembl.yaml	Updates example to new params/inputs/pipelines structure.
examples/genome.yaml	Updates example to new params/inputs/pipelines/emit structure and schema validation.
examples/gene-table.yaml	Updates example to new params/inputs/pipelines structure.
examples/gdc-convert.yaml	Updates example to new inputs + pipelines + emit outputs model.
examples/cbio.yaml	Updates example to new params/inputs/pipelines structure and inline transforms.
evaluate/template.go	Changes template context root from config to params.
docs/tags/index.xml	Removes generated Hugo artifact.
docs/sitemap.xml	Removes generated Hugo artifact.
docs/index.md	Removes embedded image from docs landing page content.
docs/index.html	Removes generated Hugo artifact.
docs/docs/transforms/uuid/index.html	Removes generated Hugo artifact.
docs/docs/transforms/uuid.md	Adds/updates Markdown doc for uuid transform.
docs/docs/transforms/tablewrite/index.html	Removes generated Hugo artifact.
docs/docs/transforms/split/index.html	Removes generated Hugo artifact.
docs/docs/transforms/regexreplace/index.html	Removes generated Hugo artifact.
docs/docs/transforms/objectvalidate/index.html	Removes generated Hugo artifact.
docs/docs/transforms/objectValidate.md	Updates example to use {{params.*}} templating.
docs/docs/transforms/lookup.md	Updates doc wording/examples from config.* to params.*.
docs/docs/transforms/index.html	Removes generated Hugo artifact.
docs/docs/transforms/graphbuild/index.html	Removes generated Hugo artifact.
docs/docs/transforms/from.md	Adds Markdown doc for from transform.
docs/docs/transforms/flatmap/index.html	Removes generated Hugo artifact.
docs/docs/transforms/flatmap.md	Adds Markdown doc for flatMap transform.
docs/docs/transforms/fieldparse/index.html	Removes generated Hugo artifact.
docs/docs/transforms/fieldParse.md	Adds Markdown doc for fieldParse transform.
docs/docs/transforms/emit/index.html	Removes generated Hugo artifact.
docs/docs/transforms/distinct/index.html	Removes generated Hugo artifact.
docs/docs/transforms/debug/index.html	Removes generated Hugo artifact.
docs/docs/transforms/debug.md	Updates Markdown frontmatter for debug transform.
docs/docs/transforms/accumulate/index.html	Removes generated Hugo artifact.
docs/docs/schema.md	Adds comprehensive playbook schema documentation for new format.
docs/docs/outputs/json.md	Adds Markdown doc for JSON output.
docs/docs/outputs/graphBuild.md	Updates graphBuild output doc to use {{params.*}}.
docs/docs/inputs/xmlload/index.html	Removes generated Hugo artifact.
docs/docs/inputs/xml.md	Renames xml input docs to new naming/fields.
docs/docs/inputs/table.md	Renames table input docs to new naming/fields and updates examples.
docs/docs/inputs/sqlite.md	Renames sqlite input docs to new naming/fields and updates examples.
docs/docs/inputs/sqldump.md	Updates docs to use path and {{params.*}}.
docs/docs/inputs/plugin.md	Updates docs to use {{params.*}}.
docs/docs/inputs/json.md	Renames json input docs to new naming/fields and updates example.
docs/docs/inputs/index.md	Adds new Markdown index for Inputs section.
docs/docs/inputs/index.html	Removes generated Hugo artifact.
docs/docs/inputs/gripperload/index.html	Removes generated Hugo artifact.
docs/docs/inputs/glob.md	Updates glob docs to new nested extractor names and path.
docs/docs/inputs/embedded/index.html	Removes generated Hugo artifact.
docs/docs/inputs/avroload/index.html	Removes generated Hugo artifact.
docs/docs/inputs/avro.md	Renames avro input docs to new naming/fields.
docs/docs/index.md	Updates docs landing page to params terminology and new outputs model.
docs/docs/example.md	Updates walkthrough example to new params/inputs/pipelines/outputs model.
docs/docs/developers/source_mapping.md	Adds mapping of docs pages to source files.
docs/docs/config.md	Adds documentation for parameter configuration.
docs/docs/.nav.yml	Adds docs navigation configuration listing major sections.
docs/categories/index.xml	Removes generated Hugo artifact.
config/config.go	Refactors config types into Params, Param, and ParamRequest; updates helpers.
cmd/scan/main.go	Removes scan CLI command implementation.
cmd/run/run.go	Renames Execute argument from inputs→params (API-level rename).
cmd/run/main.go	Updates CLI flags and parsing to use params instead of config inputs.
cmd/root.go	Removes graph-plan and scan commands from root CLI.
cmd/inspect/main.go	Updates inspect output to report required params; adjusts outputs reporting.
cmd/graphplan/main.go	Removes graph-plan CLI command implementation.
README.md	Updates README examples to use params templating terminology.
Makefile	Bumps version and removes docs generation target.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The struct tag uses json:"Path" (capital P). JSON/YAML key matching via tags is case-sensitive, so playbooks using path (lowercase) won’t unmarshal into this field, and this also conflicts with other extractors that use json:"path". Standardize this to json:"path" to avoid breaking configs and to match documentation.

Suggested change

	Path string `json:"Path" jsonschema_description:"TSV to be transformed"`
	Path string `json:"path" jsonschema_description:"TSV to be transformed"`

[Copilot]

Same issue as TableLoadStep: the tag json:"Path" requires a Path key rather than path, which is inconsistent with other loaders and the docs. Use json:"path" for consistent decoding and schema generation.

Suggested change

	Path string `json:"Path" jsonschema_description:"Path to the SQL dump file"`
	Path string `json:"path" jsonschema_description:"Path to the SQL dump file"`

[Copilot]

ReadAt can return (n > 0, io.EOF) on the final read; treating any non-nil error as “skip this row” risks dropping the last value. Also, the current code ignores n and always splits the full buffer, which can include trailing zero/garbage beyond the actual bytes read. Capture n, operate on buf[:n], and consider io.EOF as a successful read when n > 0.

[Copilot]

IsDir() currently returns true only when type is "path", but the docs in this PR mention Dir in places (and the name IsDir implies a directory type). This will cause directory params declared as Dir to be treated as non-directories. Either (a) treat both "dir" and "path" as directory/path-like types, or (b) rename the method and docs to consistently use Path.

[Copilot]

Correct spelling: “Paramaters” → “Parameters” (title and heading).

Suggested change

	title: Paramaters
	---
	## Paramaters Variables
	title: Parameters
	---
	## Parameters Variables

[Copilot]

This example uses value: for params, but other docs/examples in this PR use default: (and config/config.go defines Param.Default). To avoid confusing users and to match the implemented struct, update this example to use default: instead of value:.

Suggested change

	value: ../../source/go/goa_human.gaf.gz
	default: ../../source/go/goa_human.gaf.gz

[Copilot]

The schema doc for transpose still documents the field as input, but the code changes in this PR rename it to path (e.g., TransposeLoadStep.Path). Update the schema text to - \path`: Path to the file.` for consistency with the new playbook format.

Suggested change

	- `input`: Path to the file.
	- `path`: Path to the file.

[kellrott]

@copilot open a new pull request to apply changes based on the comments in this thread

[Copilot]

@kellrott I've opened a new pull request, #74, to work on those changes. Once the pull request is ready, I'll request review from you.