Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 0 additions & 59 deletions .flake8

This file was deleted.

2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Select the appropriate type(s) that describe this PR
## Checklist
<!-- Put an `x` in the boxes that apply. -->

- [ ] I have run `flake8` and `black` to make sure the Python code adheres to PEP-8 and is consistently formatted
- [ ] I have run `ruff check` and `ruff format` to make sure the Python code adheres to PEP-8 and is consistently formatted
- [ ] I have formatted the Fortran code with `fprettify` or C/C++ code with `clang-format` as applicable
- [ ] I have run unit and regression tests which pass locally with my changes
- [ ] I have added new tests that prove my fix is effective or that my feature works
Expand Down
17 changes: 0 additions & 17 deletions .github/workflows/black.yaml

This file was deleted.

34 changes: 0 additions & 34 deletions .github/workflows/flake8.yaml

This file was deleted.

32 changes: 32 additions & 0 deletions .github/workflows/format-and-lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
on:
workflow_call:

jobs:
format-and-lint:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.12
- name: Install dependencies
run: |
wget https://raw.githubusercontent.com/mdolab/.github/main/.pre-commit-config.yaml
pip install pre-commit
# If there's already a ruff.toml file in the repo, we should rename the file we're downloading to ruff.toml-global, then put the line `extend = "ruff.toml-global"` at the top of the local ruff.toml so that ruff will use both configurations.
url=https://raw.githubusercontent.com/mdolab/.github/main/ruff.toml
if [[ -f "ruff.toml" ]]; then
wget $url -O ruff.toml-global;
sed -i '1s/^/extend = "ruff.toml-global" \n/' ruff.toml
else
wget $url
fi
echo "Ruff config:"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to the above comment, for users that are developing locally and want to run the same thing, its seems like they would need to manually add extend = <path-to-global-ruff.toml> to all project files? We could add/have a script that does this automatically that could be used both here and locally.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's annoying, but currently they'd have to do the same thing for our repos with local flake8 configs so I don't think this is any worse.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not quite as difficult with flake8 since on the CLI you can just add the --append-config flag with the extra config file. No file editing needed. This does not seem to exist in ruff.

if [[ -f "ruff.toml-global" ]]; then
cat ruff.toml-global
fi
cat ruff.toml
- name: Run pre-commit checks
run: |
pre-commit run --all-files --show-diff-on-failure
20 changes: 20 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: trailing-whitespace
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the idea that we check all files with all these? This seems to be going through everything, including fortran files and other source and config files, generating loads of errors. Maybe this is fine since these are mostly whitespace issues, but ideally we should have tools specifically for fortran to avoid any clashes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we should be running this on all files. I often find myself unwittingly commiting a bunch of whitespace changes to files I didn't mean to edit because my editor automatically trims whitespace and fixes the ends of files. Enforcing these checks will avoid that.

- id: end-of-file-fixer
- id: check-yaml
- id: check-json
- id: mixed-line-ending
- id: check-merge-conflict
- id: debug-statements
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.12.10
hooks:
# Run the linter.
- id: ruff-check
args: [ --fix, --exit-non-zero-on-fix ]
# Run the formatter.
- id: ruff-format
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
This repo stores the following shared repository settings/configurations/templates:
- Issue and pull request templates on GitHub
- Shared configurations for
- `flake8`
- `pre-commit`
- `ruff`
- `isort`
- `pylint`
- `codecov`
Expand Down
47 changes: 24 additions & 23 deletions azure/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,30 @@ The templates are organized into the following files:
- `azure_template.yaml` which handles the job itself, calling the necessary sub-templates.

## Template Options
| Name | Type | Default | Description. |
| :------------------ | :------ | :--------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------ |
| `REPO_NAME` | string | | Name of repository |
| `COMPLEX` | boolean | `false` | Flag for triggering complex build and tests |
| `TIMEOUT_BUILD` | number | `120` | Runtime allowed for each build and test job, in minutes |
| `GCC_CONFIG` | string | `None` | Path to GCC configuration file (from repository root) |
| `INTEL_CONFIG` | string | `None` | Path to Intel configuration file (from repository root) |
| `BUILD_REAL` | string | `.github/build_real.sh` | Path to Bash script with commands to build real code. Using `None` will skip this step. |
| `TEST_REAL` | string | `.github/text_real.sh` | Path to Bash script to run real tests. Using `None` will skip this step. |
| `BUILD_COMPLEX` | string | `.github/build_complex.sh` | Path to Bash script with commands to build complex code. Using `None` will skip this step. |
| `TEST_COMPLEX` | string | `.github/text_complex.sh` | Path to Bash script with commands to run complex tests. Using `None` will skip this step. |
| `IMAGE` | string | `public` | Select Docker image. Can be `public`, `private`, or `auto`. `auto` uses the private image on trusted builds and the public image otherwise. |
| `COVERAGE` | boolean | `false` | Flag to report test coverage to `codecov` |
| `TIMEOUT_STYLE` | number | `10` | Runtime allowed for each style check, in minutes |
| `IGNORE_STYLE` | boolean | `false` | Flag to allow `black` and `flake8` checks to fail without failing the pipeline |
| `ISORT` | boolean | `false` | Flag to trigger the `isort` check |
| `PYLINT` | boolean | `false` | Flag to trigger the `pylint` check |
| `CLANG_FORMAT` | boolean | `false` | Flag to trigger the `clang-format` check |
| `FPRETTIFY` | boolean | `false` | Flag to trigger the `fprettify` check |
| `TAPENADE` | boolean | `false` | Flag to trigger the Tapenade check |
| `TIMEOUT_TAPENADE` | number | `10` | Runtime allowed for the Tapenade check, in minutes |
| `TAPENADE_SCRIPT` | string | `.github/build_tapenade.sh` | Path to Bash script with commands to run Tapenade |
| `TAPENADE_VERSION` | string | `3.10` | Version of Tapenade to use |
| Name | Type | Default | Description. |
| :----------------- | :------ | :-------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------ |
| `REPO_NAME` | string | | Name of repository |
| `COMPLEX` | boolean | `false` | Flag for triggering complex build and tests |
| `TIMEOUT_BUILD` | number | `120` | Runtime allowed for each build and test job, in minutes |
| `GCC_CONFIG` | string | `None` | Path to GCC configuration file (from repository root) |
| `INTEL_CONFIG` | string | `None` | Path to Intel configuration file (from repository root) |
| `BUILD_REAL` | string | `.github/build_real.sh` | Path to Bash script with commands to build real code. Using `None` will skip this step. |
| `TEST_REAL` | string | `.github/text_real.sh` | Path to Bash script to run real tests. Using `None` will skip this step. |
| `BUILD_COMPLEX` | string | `.github/build_complex.sh` | Path to Bash script with commands to build complex code. Using `None` will skip this step. |
| `TEST_COMPLEX` | string | `.github/text_complex.sh` | Path to Bash script with commands to run complex tests. Using `None` will skip this step. |
| `IMAGE` | string | `public` | Select Docker image. Can be `public`, `private`, or `auto`. `auto` uses the private image on trusted builds and the public image otherwise. |
| `COVERAGE` | boolean | `false` | Flag to report test coverage to `codecov` |
| `TIMEOUT_STYLE` | number | `10` | Runtime allowed for each style check, in minutes |
| `IGNORE_STYLE` | boolean | `false` | Flag to allow `formatting and linting` checks to fail without failing the pipeline |
| `RUFF` | boolean | `false` | Flag to trigger the `ruff` check |
| `ISORT` | boolean | `false` | Flag to trigger the `isort` check |
| `PYLINT` | boolean | `false` | Flag to trigger the `pylint` check |
| `CLANG_FORMAT` | boolean | `false` | Flag to trigger the `clang-format` check |
| `FPRETTIFY` | boolean | `false` | Flag to trigger the `fprettify` check |
| `TAPENADE` | boolean | `false` | Flag to trigger the Tapenade check |
| `TIMEOUT_TAPENADE` | number | `10` | Runtime allowed for the Tapenade check, in minutes |
| `TAPENADE_SCRIPT` | string | `.github/build_tapenade.sh` | Path to Bash script with commands to run Tapenade |
| `TAPENADE_VERSION` | string | `3.10` | Version of Tapenade to use |

## Setting up a pipeline
### Step 1: Setup Azure Pipelines YAML File:
Expand Down
63 changes: 29 additions & 34 deletions azure/azure_style.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ parameters:
- name: IGNORE_STYLE
type: boolean
default: false
- name: RUFF
type: boolean
default: true
- name: ISORT
type: boolean
default: false
Expand All @@ -20,61 +23,48 @@ parameters:
- name: FPRETTIFY
type: boolean
default: false

jobs:
- job: flake8
- job: ruff
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
strategy:
matrix:
"py39":
PYTHON_VERSION: "3.9"
"py311":
PYTHON_VERSION: "3.11"
condition: ${{ parameters.RUFF }}
steps:
- checkout: self
- checkout: azure_template
- task: UsePythonVersion@0
inputs:
versionSpec: $(PYTHON_VERSION)
versionSpec: "3.9"
- script: |
cd ${{ parameters.REPO_NAME }}
pip install wheel
pip install -r ../.github/flake8-requirements.txt

if [[ -f ".flake8" ]]; then
export FL8=.flake8-project
mv .flake8 $FL8; # rename the file from code repo; should have higher precedence in merge
fi

cp ../.github/.flake8 .
pip install ruff==0.12.10

if [[ -f "$FL8" ]]; then
flake8 . --append-config $FL8 --count --show-source --statistics;
# If there's already a ruff.toml file in the repo, we should rename the global config to ruff.toml-global, then put the line `extend = "ruff.toml-global"` at the top of the local ruff.toml so that ruff will use both configurations.
globalConfigFile="../.github/ruff.toml"
if [[ -f "ruff.toml" ]]; then
cp $globalConfigFile ruff.toml-global
sed -i '1s/^/extend = "ruff.toml-global" \n/' ruff.toml
else
flake8 . --count --show-source --statistics;
cp $globalConfigFile .
fi
echo "Ruff config:"
if [[ -f "ruff.toml-global" ]]; then
cat ruff.toml-global
fi
cat ruff.toml

- job: black
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: "3.9"
- script: |
pip install wheel
pip install black==23.1.0
black . --check --diff -l 120 --target-version py39 --target-version py311
# Run the linting
ruff check .

# Run the formatting
ruff format --check .

- job: isort
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
condition: ${{ parameters.ISORT }}
steps:
- checkout: self
Expand All @@ -98,6 +88,7 @@ jobs:
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
condition: ${{ parameters.PYLINT }}
steps:
- checkout: self
Expand All @@ -118,6 +109,7 @@ jobs:
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
condition: ${{ parameters.CLANG_FORMAT }}
steps:
- checkout: self
Expand All @@ -141,6 +133,7 @@ jobs:
pool:
vmImage: "ubuntu-22.04"
timeoutInMinutes: ${{ parameters.TIMEOUT }}
continueOnError: ${{ parameters.IGNORE_STYLE }}
condition: ${{ parameters.FPRETTIFY }}
steps:
- checkout: self
Expand All @@ -165,3 +158,5 @@ jobs:

# Exit with an error if any of the tracked files changed
git diff --summary --exit-code


3 changes: 3 additions & 0 deletions azure/azure_template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ parameters:
- name: FPRETTIFY
type: boolean
default: false
- name: RUFF
type: boolean
default: false
# Tapenade
- name: TAPENADE
type: boolean
Expand Down
7 changes: 0 additions & 7 deletions flake8-requirements.txt

This file was deleted.

Loading