More friendly version of config files

[minhptx]

@raunaqtri1 @binh-vu

We need to integrate our transformation pipelines into [DAME]
(http://mint-project.info/products/model/#desktop-appliation-for-model-execution-dame). So we need a more user-friendly version of our YAML so that DAME can show to the users.

Can we add some new fields to the YAML file ?

1. description: should be the description for the whole pipeline
2. description for each input of the pipeline since we have the input section now

I'll update the list after I get more requests from them.

[raunaqtri1]

@minhptx, there is already a description key for the whole pipeline. It is optional and that's why never used. For example, you can have a YAML config like this:

version: "1"
description: Transform ISRIC soil files to rts/rti files that can be used as input for Topoflow. Perform cropping, regridding and resampling using GDAL.
adapters:
  tf_soil:
    comment: My topoflow soil write adapter
    adapter: funcs.Topoflow4SoilWriteFunc
    inputs:
      layer: 1
      input_dir: "/tmp/demo/input/isric_soil"
      output_dir: "/tmp/demo/output/isric_soil"
      DEM_bounds: "34.221249999999, 7.362083333332, 36.446249999999, 9.503749999999"
      DEM_xres_arcsecs: "30"
      DEM_yres_arcsecs: "30"

[minhptx]

@raunaqtri1 Thanks. I will update the existing pipelines to include this. How about the second bullet ?(just updated)

[mosoriob]

Can you add a field dataset? A link where the user can find the input_files
For example, the topoflow climate transformation:

description: ""Data transformation to generate TopoFlow-ready precipitation files (RTS) from Global Precipitation Measurement (GPM) data sources" 
datasets: https://data-catalog.mint.isi.edu/datasets/ea0e86f3-9470-4e7e-a581-df85b4a7075d

[mosoriob]

Right now, DAME has only one transformation. If you add the description and the datasets, I can add more.
PS: We're late because the deadline of this integration was yesterday.

[binh-vu]

@sirspock There can be multiple datasets in the pipeline, if you want to list all datasets in the pipeline, you can iterate through the DcatReadFunc in the adapters, and get the dataset_id there.