Update download data

CHANGELOG.md

@@ -0,0 +1,60 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2025-07-29
+
+### Added
+- Updated CDSAPI syntax in `get_era5_data.py` to support latest API version
+- Added troubleshooting documentation for "request too large" errors
+- Added CDO merge command example for combining multiple NetCDF files
+- Added `sphinx-rtd-theme` for improved documentation rendering
+- Added development dependencies: `flake8`, `black`, and `twine`
+
+### Changed
+- Updated data download script to use modern CDSAPI syntax
+- Improved documentation with additional error handling guidance
+- Enhanced requirements.txt with documentation and development tools
+- Updated `cdsapi` from 0.6.1 to 0.7.6
+
+### Fixed
+- Fixed deprecated CDSAPI syntax that was causing compatibility issues
+- Fixed NumPy deprecation warnings in `src/utils.py` by using `.item()` method for scalar conversion
+- Fixed Sphinx/docutils dependency conflict by pinning docutils to compatible version (<0.19)
+
+## [0.0.1] - 2025-06-19
+
+### Added
+- Initial release of ATMOS-BUD
+- Heat, vorticity and humidity budget analysis for atmospheric regions
+- Support for ERA5 reanalysis data
+- Command-line interface for budget calculations
+- Documentation with getting started guide
+- Example scripts for visualization
+- Automated data download functionality
+- Support for fixed, tracking, and choose domain frameworks
+
+### Features
+- **Data Handling**: Load and preprocess atmospheric data from NetCDF files
+- **Budget Calculations**: Compute heat, vorticity, and humidity budgets
+- **Domain Selection**: Support for fixed regions, storm tracking, and user-defined domains
+- **Visualization**: Generate maps, vertical profiles, and Hovmöller diagrams
+- **Output Management**: Save results in CSV and NetCDF formats
+- **ERA5 Integration**: Direct download from Copernicus Climate Data Store
+
+### Documentation
+- Complete user guide with tutorials
+- API documentation for all modules
+- Examples for different analysis frameworks
+- Installation and setup instructions
+
+### Requirements
+- Python >= 3.10
+- NetCDF4, xarray, numpy, matplotlib
+- cartopy for map projections
+- cdsapi for ERA5 data download

docs/downloading_data.rst

@@ -8,55 +8,69 @@ In the following example, we will guide you through the process of downloading d
 Method 1: Copernicus Interface
 -------------------------------
 
-1. Access the Copernicus Climate Data Store (CDS) website at `Copernicus CDS <https://cds.climate.copernicus.eu/#!/home>`_. Create an account or log into your existing account.
+To download ERA5 data from the Copernicus Climate Data Store (CDS) using their web interface, follow these steps:
 
-.. image:: _static/images/tutorial_ERA5_copernicus_1.png
-   :align: center
-   :alt: Copernicus CDS Home
+1. Access the Copernicus Climate Data Store (CDS) website at `Copernicus CDS <https://cds.climate.copernicus.eu/#!/home>`_. On the top right, click to create an account or log into your existing account.
 
-2. Visit the ERA5 dataset page at `ERA5 dataset page <https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-pressure-levels?tab=overview>`_ and click on "Download Data".
+2. Visit the ERA5 dataset page at `ERA5 dataset page <https://cds.climate.copernicus.eu/datasets/reanalysis-era5-pressure-levels?tab=overview>`_ and click on "Download".
 
-.. image:: _static/images/tutorial_ERA5_copernicus_2.png
-   :align: center
-   :alt: ERA5 Dataset Download
+3. Select the necessary variables for your analysis:
 
-3. Select the necessary variables for your analysis.
+   - **Geopotential**
+   - **Temperature**
+   - **Specific Humidity**
+   - **Vertical Velocity**
+   - **U-Component of Wind**
+   - **V-Component of Wind**
 
-.. image:: _static/images/tutorial_ERA5_copernicus_3.png
-   :align: center
-   :alt: Variable Selection
+4. Choose the time range of interest (year, month, day). Make sure to select the dates that correspond to the atmospheric event you wish to analyze.
 
-4. Select the desired vertical levels for your analysis. For this example, we are using data from the surface to 10 hPa.
+5. Define the temporal resolution. Generally, 3-hourly data is suitable for most atmospheric analyses, but you can choose hourly or daily data depending on your needs.
 
-.. image:: _static/images/tutorial_ERA5_copernicus_4.png
-   :align: center
-   :alt: Vertical Level Selection
+6. Select the desired vertical levels for your analysis. Generally, the pressure levels of interest are from 1000 hPa to 100 hPa.
 
-5. Choose the time range of interest. Make sure to select the dates that correspond to the atmospheric event you wish to analyze.
+7. Select sub-region extraction and specify the spatial extent by selecting the geographic region that covers the atmospheric system you are studying.
 
-.. image:: _static/images/tutorial_ERA5_copernicus_5.png
-   :align: center
-   :alt: Date Selection
+8. Choose the output file format. Select 'NetCDF' as the preferred format for the output data.
 
-6. Define the temporal resolution. In this case, we will opt for 3-hourly data for simplicity.
+9. Select download format as "Unarchived" to ensure the data is downloaded in a single NetCDF file without compression.
 
-.. image:: _static/images/tutorial_ERA5_copernicus_6.png
-   :align: center
-   :alt: Temporal Range Selection
+10. After all selections have been made, submit the data retrieval request. You will receive the dataset in the specified format once the request has been processed.
 
-7. Specify the spatial extent by selecting the geographic region that covers the atmospheric system you are studying.
+11. Once you have downloaded the dataset, it is advisable to rename the file for easier identification and organization. Use a convention that includes the system identifier and dataset name, such as ``system-identifier_dataset-name.nc``. For example, rename your downloaded file to ``system-20050808_ERA5.nc`` to reflect the system date and data source.
 
-.. image:: _static/images/tutorial_ERA5_copernicus_7.png
-   :align: center
-   :alt: Spatial Range Selection
+Method 3: NCAR's Research Data Archive
+---------------------------------------
 
-8. Choose the output file format. Select 'NetCDF' as the preferred format for the output data.
+For users who prefer to access data from the National Center for Atmospheric Research (NCAR), you can download ERA5 data from their Research Data Archive. Follow these steps:
+
+1. Visit the ERA5 dataset page on the NCAR Research Data Archive website at `NCAR RDA <https://rda.ucar.edu/datasets/d633000/dataaccess/>`_.
+
+2. Click on the "Get a Subset" from the "ERA5 atmospheric pressure level analysis [netCDF4]" dataset
+
+3. On "Temporal Selection", select the time range (start and end date) that corresponds to the atmospheric event you are interested in. You can specify the year, month, and day.
+ - Here, we cannot select the temporal resolution, only the start and end hours. So, if the user wishes to use a distinct temporal resolution, they will need to download the data and process it using Climate Data Operators (CDO).
+
+4. Select the necessary variables for your analysis:
 
-9. After all selections have been made, submit the data retrieval request. You will receive the dataset in the specified format once the request has been processed.
+   - **Geopotential**
+   - **Temperature**
+   - **Specific Humidity**
+   - **Vertical Velocity**
+   - **U Component of Wind**
+   - **V Component of Wind**
 
-10. Once you have downloaded the dataset, it is advisable to rename the file for easier identification and organization. Use a convention that includes the system identifier and dataset name, such as ``system-identifier_dataset-name.nc``. For example, rename your downloaded file to ``system-20050808_ERA5.nc`` to reflect the system date and data source.
+5. Click on "Continue" to proceed to the next step.
 
-Method 2: Automated Download Using `get_era5_data.py`
+6. Check whether the selected variables are correct and select the vertical levels you need for your analysis. The pressure levels of interest are generally from 1000 hPa to 100 hPa.
+
+7. Make a spatial selection by clicking on "DRAW BOX" bellow the map and either select the desired region or enter the latitude and longitude coordinates manually. Ensure that the selected region covers the atmospheric system you are studying.
+
+8. Click on "Submit Request" to initiate the data retrieval process.
+
+ - Note that the system might retrieve multiple files, depending on the time range and spatial extent you selected. If you encounter multiple files, you can merge them using Climate Data Operators (CDO) or similar tools.
+
+Method 3: Automated Download Using `get_era5_data.py`
 ------------------------------------------------------
 
 For those who prefer an automated approach to downloading ERA5 data, ATMOS-BUD includes a Python script that utilizes the ECMWF's `cdsapi` library. This method streamlines the data retrieval process, allowing you to specify your data requirements directly within the script.
@@ -67,24 +81,30 @@ For those who prefer an automated approach to downloading ERA5 data, ATMOS-BUD i
 
     cd src
 
-2. Before running the script, it's recommended to create a copy and edit the copy instead. This ensures that the original script remains unaltered for future reference.
-
-.. code-block:: bash
-
-    cp get_era5_data.py custom_get_era5_data.py
-
-3. Open the `custom_get_era5_data.py` script in your preferred text editor. Inside the script, locate the section where input parameters are specified, such as pressure levels, dates, and spatial coverage. Modify these inputs to match your specific data requirements.
+2. Open the `get_era5_data.py` script in your preferred text editor. Inside the script, locate the section where input parameters are specified, such as pressure levels, dates, and spatial coverage. Modify these inputs to match your specific data requirements.
 
 4. Once you have configured the script with your desired parameters, save the changes, and run the script from the terminal:
 
 .. code-block:: bash
 
-    python custom_get_era5_data.py
+    python get_era5_data.py
 
 The script will automatically communicate with the Copernicus Climate Data Store, request the specified data, and download it in NetCDF format to your local machine. This method is particularly useful for automating the data retrieval process and can be easily integrated into larger data processing workflows.
 
+
+.. note::
+   Ensure you have the `cdsapi` library installed and configured with your CDS API key before running the script. For more information on setting up `cdsapi`, visit the official `cdsapi` installation guide and user documentation: `https://cds.climate.copernicus.eu/how-to-api`
+
 .. note::
-   Ensure you have the `cdsapi` library installed and configured with your CDS API key before running the script. For more information on setting up `cdsapi`, visit the official `cdsapi` installation guide and user documentation: `https://cds.climate.copernicus.eu/api-how-to`
+    Regardless of the method used, it is advisable to rename the downloaded file, choosing a name that facilitates easy identification and organization of your datasets. We recommend adopting a naming convention that incorporates the system identifier and the dataset name. For example, naming your output file as ``system-20050808_ERA5.nc`` can be beneficial, where ``20050808`` denotes the date of the atmospheric event under analysis, and ``ERA5`` represents the dataset source. This practice ensures your files are organized systematically, enhancing the efficiency of data management and retrieval for future analyses.
+
+.. note::
+    If you encounter the error message "Your request is too large, please reduce your selection" when downloading data, this indicates that you are trying to download too much data in a single request. To resolve this issue, it is recommended to download data for each day separately in a loop and then merge the individual files using the Climate Data Operators (CDO) tool. You can merge multiple NetCDF files chronologically using the following command:
+
+    .. code-block:: bash
+
+        cdo -f nc mergetime system-*_ERA5.nc system-merged_ERA5.nc
+
+    This command will merge all files matching the pattern ``system-*_ERA5.nc`` in chronological order into a single output file named ``system-merged_ERA5.nc``. Make sure to have CDO installed on your system before using this command. For installation instructions and more information about CDO, visit: `https://code.mpimet.mpg.de/projects/cdo <https://code.mpimet.mpg.de/projects/cdo>`_
 
 .. note::
-    The script `get_era5_data.py` includes the option to specify the output file name directly within the script. It is advisable to choose a name that facilitates easy identification and organization of your datasets. We recommend adopting a naming convention that incorporates the system identifier and the dataset name. For example, naming your output file as ``system-20050808_ERA5.nc`` can be beneficial, where ``20050808`` denotes the date of the atmospheric event under analysis, and ``ERA5`` represents the dataset source. This practice ensures your files are organized systematically, enhancing the efficiency of data management and retrieval for future analyses.

requirements.txt

@@ -1,7 +1,5 @@
-alabaster==0.7.16
-babel==2.17.0
 Cartopy==0.22.0
-cdsapi==0.6.1
+cdsapi==0.7.6
 certifi==2024.2.2
 cftime==1.6.3
 charset-normalizer==3.3.2
@@ -11,20 +9,16 @@ cmocean==3.1.3
 contourpy==1.2.0
 cycler==0.12.1
 dask==2024.2.0
-docutils==0.18.1
 exceptiongroup==1.3.0
 fonttools==4.49.0
 fsspec==2024.2.0
 idna==3.6
-imagesize==1.4.1
 importlib-metadata==7.0.1
 iniconfig==2.1.0
-Jinja2==3.1.6
 joblib==1.3.2
 kiwisolver==1.4.5
 locket==1.0.0
 lorenz-phase-space==1.2.1
-MarkupSafe==3.0.2
 matplotlib==3.8.3
 MetPy==1.6.1
 netCDF4==1.6.5
@@ -38,7 +32,6 @@ platformdirs==4.2.0
 pluggy==1.6.0
 pooch==1.8.0
 pyarrow==15.0.0
-Pygments==2.19.1
 pyparsing==3.1.1
 pyproj==3.6.1
 pyshp==2.3.1
@@ -53,13 +46,6 @@ scipy==1.12.0
 shapely==2.0.3
 six==1.16.0
 snowballstemmer==3.0.1
-Sphinx==5.0.0
-sphinxcontrib-applehelp==2.0.0
-sphinxcontrib-devhelp==2.0.0
-sphinxcontrib-htmlhelp==2.1.0
-sphinxcontrib-jsmath==1.0.1
-sphinxcontrib-qthelp==2.0.0
-sphinxcontrib-serializinghtml==2.0.0
 threadpoolctl==3.3.0
 tomli==2.2.1
 toolz==0.12.1
@@ -70,3 +56,23 @@ tzdata==2024.1
 urllib3==2.2.0
 xarray==2025.4.0
 zipp==3.17.0
+
+# Development and documentation dependencies
+alabaster==0.7.16
+babel==2.17.0
+black==23.12.1
+docutils>=0.14,<0.19
+flake8==6.0.0
+imagesize==1.4.1
+Jinja2==3.1.6
+MarkupSafe==3.0.2
+Pygments==2.19.1
+Sphinx==5.0.0
+sphinx-rtd-theme==1.3.0
+sphinxcontrib-applehelp==2.0.0
+sphinxcontrib-devhelp==2.0.0
+sphinxcontrib-htmlhelp==2.1.0
+sphinxcontrib-jsmath==1.0.1
+sphinxcontrib-qthelp==2.0.0
+sphinxcontrib-serializinghtml==2.0.0
+twine==4.0.2

setup.py

@@ -1,31 +1,52 @@
 from setuptools import setup, find_packages
+import os
 
 # Lê as dependências diretamente do requirements.txt
-with open('requirements.txt') as f:
-    required = f.read().splitlines()
+def read_requirements():
+    with open('requirements.txt') as f:
+        return [line.strip() for line in f if line.strip() and not line.startswith('#')]
+
+# Lê o README
+def read_readme():
+    if os.path.exists('README.md'):
+        with open('README.md', encoding='utf-8') as f:
+            return f.read()
+    return ''
 
 setup(
     name='atmos-bud',
-    version='0.0.0',
+    version='0.1.0',
     description='Program for analyzing the heat, vorticity and humidity budgets of limited regions on the atmosphere.',
-    long_description=open('README.md').read(),
+    long_description=read_readme(),
     long_description_content_type='text/markdown',
     author='Danilo Couto de Souza',
     author_email='danilo.oceano@gmail.com',
     url='https://github.com/daniloceano/ATMOS-BUD',
     packages=find_packages(where='src'),
     package_dir={'': 'src'},
-    install_requires=required,
+    install_requires=read_requirements(),
     entry_points={
         'console_scripts': [
-            'atmos-bud=atmos_bud:main',
+            'atmos-bud=atmos_bud.main:main',  # Verificar se este path está correto
         ],
     },
     classifiers=[
+        'Development Status :: 3 - Alpha',
+        'Intended Audience :: Science/Research',
+        'Topic :: Scientific/Engineering :: Atmospheric Science',
+        'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
         'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.10',
+        'Programming Language :: Python :: 3.11',
         'Operating System :: OS Independent',
     ],
     license='GPL-3.0',
     python_requires='>=3.10',
     license_files=('LICENSE',),
-)
+    keywords='atmospheric science, meteorology, budget analysis, ERA5',
+    project_urls={
+        'Documentation': 'https://github.com/daniloceano/ATMOS-BUD/docs',
+        'Source': 'https://github.com/daniloceano/ATMOS-BUD',
+        'Tracker': 'https://github.com/daniloceano/ATMOS-BUD/issues',
+    },
+)