Skip to content

Clarify the support of custom filenames#94

Open
kimlee87 wants to merge 3 commits intomainfrom
custom_name
Open

Clarify the support of custom filenames#94
kimlee87 wants to merge 3 commits intomainfrom
custom_name

Conversation

@kimlee87
Copy link
Collaborator

This PR resolves issue #42

  • update script and tests for downloading data to check file existing cases
  • rename get_filename to suggest_filename to clarify that the returned string is only a suggestion and not mandatory, update tests accordingly
  • update tutorials A and B accordingly

Filenames

Users have two options to name a file for downloaded data:

  • Choose an arbitrary name but the file extension should match data_format in the request, e.g. .nc for netcdf, .grib for grib files
  • Use inout.suggest_filename() (previously inout.get_filename()) to generate a filename with key metadata about the dataset.
    • Structure of the filename
    {base_name}_{year_str}_{month_str}_{day_str}_{time_str}_{var_str}_{ds_type}_{area_str}_raw.{ext}
    

Unique tag when preprocess data files

By default if the unique_tag parameter of the preprocess.preprocess_data_file() function is set to None, unique_tag would be generated as "ts{timestamp}_h{hostname}"

However, users can provide any non-empty string for the unique_tag, particularly when rerunning the preprocessing scripts multiple times on the same raw data, and then use the resulting preprocessed file in the next pipeline. This addresses the comment mentioned in issue #42.

@codecov
Copy link

codecov bot commented Jan 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.76%. Comparing base (3bfea3c) to head (b12925a).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #94   +/-   ##
=======================================
  Coverage   98.75%   98.76%           
=======================================
  Files           9        9           
  Lines        2405     2422   +17     
=======================================
+ Hits         2375     2392   +17     
  Misses         30       30           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kimlee87 kimlee87 marked this pull request as ready for review January 27, 2026 08:55
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants