WHY
The following readers/writers are using implicit config.<VAR> variables:
gcs_reader, via the object_storage_reader
s3_reader, via the object_storage_reader
config.REGION_NAME, config.AWS_ACCESS_KEY_ID, config.AWS_SECRET_ACCESS_KEY
gcs_writer
bq_writer
These variables are not provided by the user as click options, but as environment variables. These environment variables are retrieved from the following snippet that appears in the nck/config.py module:
for key, var in os.environ.items():
locals()[key] = var
This can be quite confusing for the user, and does not match the common NCK logic.
HOW
Explicitly implement these variables as click options