Skip to content

Allow variable values to be passed in on command line #153

@lisad

Description

@lisad

Issue #152 suggested this feature idea

When working with data from different sources, different days, different URLs etc, the pipeline should be able to be launched with variable names

E.g: data team periodically downloads "employees.csv" from multiple companies and it's also important what day it's downloaded

> python3 -m phaser run employee_pipeline output "sources/employees.csv" --var company_name="Example" --var date="2024-06-06"

I guess this would be done like this:

  • in the CLI, add an argument name="--var", with the 'append' action (using python's argparse library)
  • split each value of 'var' on the equal sign
  • after the pipeline is created, get its context and use add_variable for each value

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions