Human 2.0

.github/pull_request_template.md

@@ -5,9 +5,9 @@ Is it an additional test/function/dataset?
 e.g. This PR improves/fixes # by ...
 -->
 
-
 **I hereby confirm that I have:**
 <!-- *Note: replace [ ] with [X] to check the box. -->
-- [ ] Tested my code on my own computer for running the model
-- [ ] Selected `develop` as a target branch
-- [ ] Any removed reactions and metabolites have been moved to the corresponding deprecated identifier lists
+- [ ] Any removed reactions and metabolites have been moved to the corresponding deprecated identifier lists in `data/deprecatedIdentifiers/`.
+<!-- Chose ONE of the following two options. -->
+- [ ] This PR has `develop` as target branch, and will be resolved with a **squash-merge**.
+- [ ] This PR has `main` as target branch, and will be resolved with a **merge commit**.

.github/workflows/check-metabolictasks.yml

@@ -1,6 +1,6 @@
 name: Test metabolic tasks
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   check-metabolictasks:

.github/workflows/commentGeneEssential.md

@@ -0,0 +1,7 @@
+This PR has been [automatically tested with GH Actions](https://github.com/SysBioChalmers/Human-GEM/actions/runs/{GH_ACTION_RUN}). Here is the output of the gene essentiality test:
+
+<pre>
+{TEST_RESULTS}
+</pre>
+
+> _Note: In the case of multiple test runs, this post will be edited._

.github/workflows/commentMacaw.md

@@ -0,0 +1,9 @@
+This PR has been [automatically tested with GH Actions](https://github.com/SysBioChalmers/Human-GEM/actions/runs/{GH_ACTION_RUN}). Here is the output of the [MACAW](https://github.com/Devlin-Moyer/macaw) test:
+
+<pre>
+{TEST_RESULTS}
+</pre>
+
+This and a more detailed output from MACAW are also committed to `data/macawResults/`.
+
+> _Note: In the case of multiple test runs, this post will be edited._

.github/workflows/gene-essentiality.yml

@@ -0,0 +1,80 @@
+name: Check gene essentiality with Hart 2015
+
+on:
+  pull_request:
+    branches:
+      - "main"
+      - "develop"
+
+jobs:
+  gene-essentiality:
+    runs-on: self-hosted
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Fetch RAVEN
+        uses: actions/checkout@v4
+        with:
+          repository: "SysBioChalmers/RAVEN"
+          path: "RAVEN"
+
+      - name: Run gene essentiality
+        id: essentiality
+        run: >
+          TEST_RESULTS=$(/usr/local/bin/matlab -batch
+          "warning('off', 'MATLAB:rmpath:DirNotFound');
+          rmpath(genpath('/home/m/ecModels-dependencies/RAVEN'));
+          rmpath(genpath('/home/m/actions-runner'));
+          addpath(genpath('.'));
+          setRavenSolver('gurobi');
+          ihuman = readYAMLmodel('model/Human-GEM.yml');
+          taskStruct = parseTaskList('data/metabolicTasks/metabolicTasks_Essential.txt');
+          [~, eGenes] = evalc('estimateEssentialGenes(ihuman, ''Hart2015_RNAseq.txt'', taskStruct);');
+          output = transpose(evaluateHart2015Essentiality(eGenes));
+          fid = fopen('data/testResults/gene-essential.csv','w');
+          fprintf(fid,[repmat('%s,',1,9) '%s\n'],output{:,1});
+          fprintf(fid,['%s,%d,%d,%d,%d' repmat(',%.4g',1,5) '\n'],output{:,2:end});
+          fclose(fid);
+          disp(cell2table(transpose(output(:,2:end)),'VariableNames',output(:,1)));") &&
+          echo "$TEST_RESULTS" &&
+          PARSED_RESULTS="${TEST_RESULTS//$'\n'/'<br>'}" &&
+          PARSED_RESULTS="${PARSED_RESULTS//$'\r'/'<br>'}" &&
+          echo "results=$PARSED_RESULTS" >> $GITHUB_OUTPUT
+
+      - name: Mention PR# in README.md
+        env:
+          PR_NUMBER: ${{ github.event.number }}
+        run: sed -i -e "s/[[:digit:]]\{3,4\}\*\* (gene /$PR_NUMBER\*\* (gene /" data/testResults/README.md
+
+      - name: Update local branch before committing changes
+        env:
+          BRANCH_NAME: ${{ github.head_ref || github.ref_name }} 
+        run: |
+          git stash
+          git fetch
+          git checkout $BRANCH_NAME
+          if git stash list | grep -q 'stash@{'; then
+            git stash pop
+          fi
+
+      - name: Auto-commit results
+        uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_user_name: memote-bot
+          commit_message: "chore: add gene essentiality test result"
+          file_pattern: data/testResults/*
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUMBER: ${{ github.event.number }}
+
+      - name: Post comment
+        uses: NejcZdovc/comment-pr@v2
+        with:
+          file: "commentGeneEssential.md"
+          identifier: "GITHUB_COMMENT_GENE"
+        env:
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+          TEST_RESULTS: ${{steps.essentiality.outputs.results}}
+          GH_ACTION_RUN: ${{github.run_id}}

.github/workflows/macaw-tests.yml

@@ -0,0 +1,71 @@
+name: Run macaw tests
+
+on: [pull_request]
+
+jobs:
+  macaw-tests:
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Test checkout
+        run: ls -la && cd code && ls -la
+
+      - name: Set up Python 3
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install macaw
+        run: pip install git+https://github.com/Devlin-Moyer/macaw.git@main numpy==1.26.4
+
+      - name: Run macaw
+        id: macaw-run
+        run: |
+          TEST_RESULTS=$(python code/test/macawTests.py)
+          echo $TEST_RESULTS
+          PARSED_RESULTS="${TEST_RESULTS//$'\n'/'<br>'}"
+          PARSED_RESULTS="${PARSED_RESULTS//$'\r'/'<br>'}"
+          echo $PARSED_RESULTS
+          echo "results=$PARSED_RESULTS" >> $GITHUB_OUTPUT
+          printf "$TEST_RESULTS" > data/testResults/macaw_summary.md
+
+      - name: Mention PR# in README.md
+        env:
+          PR_NUMBER: ${{ github.event.number }}
+        run: sed -i -e "s/[[:digit:]]\{3,4\}\*\* (MACAW)/$PR_NUMBER\*\* (MACAW)/" data/testResults/README.md
+
+      - name: Update local branch before committing changes
+        env:
+          BRANCH_NAME: ${{ github.head_ref || github.ref_name }} 
+        run: |
+          git stash
+          git fetch
+          git checkout $BRANCH_NAME
+          if git stash list | grep -q 'stash@{'; then
+            git stash pop
+          fi
+
+      - name: Auto-commit results
+        uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_user_name: memote-bot
+          commit_message: "chore: add macaw test result"
+          file_pattern: data/testResults/*
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUMBER: ${{ github.event.number }}
+
+      - name: Post comment
+        uses: NejcZdovc/comment-pr@v2
+        with:
+          file: "commentMacaw.md"
+          identifier: "GITHUB_COMMENT_MACAW"
+        env:
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+          TEST_RESULTS: ${{steps.macaw-run.outputs.results}}
+          GH_ACTION_RUN: ${{github.run_id}}
+

.github/workflows/yaml-conversion.yml

@@ -1,6 +1,6 @@
 name: Test YAML conversion
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   yaml-conversion:

.github/workflows/yaml-validation.yml

@@ -1,6 +1,6 @@
 name: YAML validation
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   yaml-validation:

README.md

@@ -1,30 +1,22 @@
 # Human-GEM: The generic genome-scale metabolic model of _Homo sapiens_
-
 [![Join the chat at https://gitter.im/SysBioChalmers/Human-GEM](https://badges.gitter.im/SysBioChalmers/Human-GEM.svg)](https://gitter.im/SysBioChalmers/Human-GEM?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GitHub version](https://badge.fury.io/gh/sysbiochalmers%2FHuman-GEM.svg)](https://badge.fury.io/gh/sysbiochalmers%2FHuman-GEM)
 [![DOI](https://zenodo.org/badge/105752644.svg)](https://zenodo.org/badge/latestdoi/105752644)<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
 [![All Contributors](https://img.shields.io/badge/all_contributors-31-success.svg)](#contributors)
 <!-- ALL-CONTRIBUTORS-BADGE:END --> 
 
-### Brief Model Description
-
+### Brief model description
 This repository contains the latest version of Human-GEM, a human genome-scale metabolic model. We encourage [contributions](#contributing).
 
 ### Cite us:
-
 If you use Human1 in your research, please cite:  
-
  > J. L. Robinson, P. Kocabasÿ, H. Wang, P.-E. Cholley, et al. An atlas of human metabolism. _Sci. Signal._ 13, eaaz1482 (2020). [doi:10.1126/scisignal.aaz1482](https://doi.org/10.1126/scisignal.aaz1482)
 
 Starting from Human-GEM v1.5.0, all the releases are also archived in [Zenodo](https://doi.org/10.5281/zenodo.4099692) from which specific version can be cited if used.
 
 If you use Mouse1, Rat1, Zebrafish1, Fruitfly1, or Worm1 in your research, please cite:   
-
   > H. Wang, J. L. Robinson, P. Kocabasÿ, J. Gustafsson, M. Anton, P.-E. Cholley, et al. Genome-scale metabolic network reconstruction of model animals as a platform for translational research. _PNAS_ 118, e2102344118 (2021). [doi.org/10.1073/pnas.2102344118](https://doi.org/10.1073/pnas.2102344118)
 
-
-
-### Model Keywords
-
+### Model keywords
 **Utilisation:** predictive simulation, multi-omics integrative analysis, model template  
 **Field:** metabolic-network reconstruction  
 **Type of Model:** reconstruction, curated  
@@ -34,38 +26,43 @@ If you use Mouse1, Rat1, Zebrafish1, Fruitfly1, or Worm1 in your research, pleas
 **Metabolic System:** general metabolism  
 **Condition:** generic metabolism  
 
-
-### Model Overview
-
+### Model overview
 |Taxonomy | Template Model | Reactions | Metabolites| Genes |
 | ------------- |:-------------:|:-------------:|:-------------:|:-----:|
 |_Homo sapiens_ |   HMR2, Recon3D, iHsa|    {{nRXN}}|  {{nMET}}|  {{nGENE}}|
 
-
 ## Contributing
-
 Contributions are always welcome! Read more about the project's philosophy in our [wiki](https://github.com/SysBioChalmers/Human-GEM/wiki) or have a look at the [Contributing guidelines](https://github.com/SysBioChalmers/Human-GEM/blob/main/.github/CONTRIBUTING.md) before starting.
 
-
-## User Guide
-
+## User guide
 Detailed instructions on the installation and use of the Human-GEM model and repository can be found in the [Human-GEM user guide](https://sysbiochalmers.github.io/Human-GEM-guide/).
 
+# Installation
+## Required software
+### Basic user
+If you want to use the model for your own model simulations, you can use any software that accepts **SBML L3V1 FBCv3** formatted model files. This includes any of the following:
+
+#### MATLAB-based
+* [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN) v2.10.3 or later (recommended, see [Installation instructions](https://github.com/SysBioChalmers/RAVEN/wiki/Installation#installation-instructions))
+* [COBRA Toolbox](https://github.com/opencobra/cobratoolbox)
 
-## Installation
+#### Python-based
+* [cobrapy](https://github.com/opencobra/cobrapy)  
 
-### Required Software
-* A functional MATLAB installation (MATLAB 7.3 and higher).
-* The [RAVEN toolbox](https://github.com/SysBioChalmers/RAVEN).
-* The [COBRA toolbox](https://github.com/opencobra/cobratoolbox) (not necessary for most functionality).
+Please see the installation instructions for each software package.
 
+### Developer
+#### MATLAB-based  
+If you want to contribute to the development of Human-GEM, or otherwise want to run any of the [provided](https://github.com/SysBioChalmers/Human-GEM/tree/main/code) MATLAB functions, then the following software is required:
+* [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN) v2.10.3 or later (recommended, see [Installation instructions](https://github.com/SysBioChalmers/RAVEN/wiki/Installation#installation-instructions))
 
-### Dependencies - Recommended Software
-* The libSBML MATLAB API (version [5.13.0](https://sourceforge.net/projects/sbml/files/libsbml/5.13.0/stable/MATLAB%20interface/) is recommended).
-* [Gurobi Optimizer](http://www.gurobi.com/registration/download-reg) for any simulations.
+#### Python-based  
+You can also contribution to the development of Human-GEM via python (e.g. cobrapy), even if you would not be able to run any of the model-specific MATLAB functions. To curate the model, you can still edit `Human-GEM.yml`, `genes.tsv`, `metabolites.tsv` and `reactions.tsv`, all located in the `model/` folder.
 
+### Recommended solver
+* When performing simulations with Human-GEM, you are encouraged to use [Gurobi Optimizer](https://www.gurobi.com/academia/academic-program-and-licenses/).
 
-### Installation Instructions
+## Installation instructions
 * Clone the [main branch](https://github.com/SysBioChalmers/Human-GEM/tree/main) of this repository, or [download the latest release](https://github.com/SysBioChalmers/Human-GEM/releases/latest).
 * Add the directory to your MATLAB path either by using the lines below or manually (instructions [here](https://se.mathworks.com/help/matlab/ref/addpath.html?requestedDomain=www.mathworks.com)).
 ```matlab
@@ -77,47 +74,35 @@ HumanGEMInstaller.install
 HumanGEMInstaller.uninstall
 ```
 
-## Model Files
-
-The model is available as `.xml`, `.xlsx`, `.txt`, `.yml`, and `.mat` in the `model/` directory. Note that only the `.yml` version is available on branches other than `main` (e.g., `develop`), to facilitate tracking of model changes.
+## Model files
+The model is available as `.xml`, `.xlsx`, `.txt`, `.yml`, and `.mat` in the `model/` directory.
 
+The model development is based on the `yml` file (to facilitate tracking of model changes), so that this is the only model format available on non-`main` branches (such as `develop`). See also the [location of annotation information](#reaction-metabolite-and-gene-annotations).
 
 ## Usage
-
-#### Loading/saving the model
-
-`Human-GEM.mat` (Recommended if on `main` branch)
+#### Loading/saving the model in MATLAB
+`Human-GEM.mat` (Recommended if on `main` branch):
 * Load and save using the built-in MATLAB `load()` and `save()` functions.
 
-`Human-GEM.yml` (Recommended if on `develop` or other branches)
-* Load using the `importYaml.m` function (in `code/io/`)
-* Save using the `exportYaml.m` function (in `code/io/`)
-
-`Human-GEM.xml` (SBML format)
-* Load using the `importModel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN))
-* Save using the `exportModel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN))
-
-
-## Reaction, Metabolite, and Gene Annotations
-
-Additional annotation information and external identifiers for Human-GEM reactions, metabolites, and genes are provided as `tsv` files in the `model/` directory (`reactions.tsv`, `metabolites.tsv`, and `genes.tsv`, respectively).  
+`Human-GEM.yml` (Recommended if on `develop` or other branches):
+* Load using the `readYAMLmodel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN)).
+* Save using the `writeYAMLmodel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN)).
 
-To import/export this annotation data to/from MATLAB, use the `importTsvFile` and `exportTsvFile` functions, respectively.
+`Human-GEM.xml` (SBML format):
+* Load using the `importModel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN)).
+* Save using the `exportModel.m` function (from [RAVEN Toolbox](https://github.com/SysBioChalmers/RAVEN)).
 
+## Reaction, metabolite, and gene annotations
+By default, additional annotation information and external identifiers for Human-GEM reactions, metabolites, and genes are **not** kept in the `yml` file, but rather provided as `tsv` files in the `model/` directory (`reactions.tsv`, `metabolites.tsv`, and `genes.tsv`, respectively). The `annotateGEM` function can add those annotations in MATLAB, while direct import/export of this annotation data is done through the `importTsvFile` and `exportTsvFile` functions, respectively.
 
 ## Websites
-
 - [Metabolic Atlas](https://metabolicatlas.org/) enables visualization and exploration of Human-GEM content.
 - The [Human-GEM user guide](https://sysbiochalmers.github.io/Human-GEM-guide/) provides detailed instructions and examples for using the Human-GEM model and repository.
 
-
-## Metabolic Maps
-
+## Metabolic maps
 A collection of manually curated 2D metabolic maps associated with Human-GEM are stored in the [Human-maps repository](https://github.com/SysBioChalmers/Human-maps). These maps can be downloaded from the repository or explored interactively using [Metabolic Atlas](https://metabolicatlas.org/explore/map-viewer/human1).
 
-
 ## Contributors
-
 <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
 <!-- prettier-ignore-start -->
 <!-- markdownlint-disable -->
@@ -164,8 +149,6 @@ A collection of manually curated 2D metabolic maps associated with Human-GEM are
     </tr>
   </tbody>
 </table>
-
 <!-- markdownlint-restore -->
 <!-- prettier-ignore-end -->
-
 <!-- ALL-CONTRIBUTORS-LIST:END -->

code/DepMapGeneEss/PrepDepMapData.m

@@ -37,7 +37,7 @@
 save('data/arrayDataDepMap.mat','arrayDataDepMap');
 
 %Generate ftINIT prepData - only needs to be done once. Can take up to an hour to run
-model = importYaml('../../model/Human-GEM.yml');
+model = readYAMLmodel('../../model/Human-GEM.yml');
 [model.grRules, skipped] = simplifyGrRules(model.grRules, true);%takes a few minutes to run
 prepData = prepHumanModelForftINIT(model, true, '../../data/metabolicTasks/metabolicTasks_Essential.txt', '../../model/reactions.tsv');
 save('data/prepDataGeneSymbols.mat', 'prepData')