Skip to content

Fix crash when using old InterPro data version#295

Open
TheLostLambda wants to merge 6 commits intoebi-pf-team:mainfrom
TheLostLambda:main
Open

Fix crash when using old InterPro data version#295
TheLostLambda wants to merge 6 commits intoebi-pf-team:mainfrom
TheLostLambda:main

Conversation

@TheLostLambda
Copy link

The crash can be reproduced by just setting --interpro 107.0 on the test command:

interproscan6 on  HEAD (4e76f7c)
❯ NXF_APPTAINER_CACHEDIR=/mnt/ARCHIVE/Bioinformatics/Nextflow nextflow run main.nf -profile apptainer,test --datadir /mnt/ARCHIVE/Bioinformatics/Nextflow --interpro 107.0

 N E X T F L O W   ~  version 25.10.3

Launching `main.nf` [trusting_varahamihira] DSL2 - revision: 822dfbf6eb

# InterProScan6 6.0.0
# Genome-scale protein function classification

executor >  local (49)
[69/b5ebcf] process > INTERPROSCAN:PREPARE_DATABASES:FIND_MISSING_DATA                [100%] 1 of 1 ✔
[-        ] process > INTERPROSCAN:PREPARE_DATABASES:DOWNLOAD_DATABASE                -
[56/6968d6] process > INTERPROSCAN:PREPARE_DATABASES:VALIDATE_DATA                    [100%] 1 of 1 ✔
[8a/30922e] process > INTERPROSCAN:PREPARE_SEQUENCES:VALIDATE_FASTA (1)               [100%] 1 of 1 ✔
[c2/4acbb2] process > INTERPROSCAN:PREPARE_SEQUENCES:LOAD_SEQUENCES (1)               [100%] 1 of 1 ✔
[b0/972824] process > INTERPROSCAN:PREPARE_SEQUENCES:SPLIT_FASTA (1)                  [100%] 1 of 1 ✔
[fe/4ce6a6] process > INTERPROSCAN:LOOKUP:PREPARE_LOOKUP                              [100%] 1 of 1 ✔
[-        ] process > INTERPROSCAN:LOOKUP:LOOKUP_MATCHES                              -
[2a/551fbe] process > INTERPROSCAN:SCAN_REMAINING:ANTIFAM:SEARCH_ANTIFAM (1)          [100%] 1 of 1 ✔
[67/a6da18] process > INTERPROSCAN:SCAN_REMAINING:ANTIFAM:PARSE_ANTIFAM (1)           [100%] 1 of 1 ✔
[ad/5d505c] process > INTERPROSCAN:SCAN_REMAINING:CATH:SEARCH_GENE3D (1)              [100%] 1 of 1 ✔
[cc/b40734] process > INTERPROSCAN:SCAN_REMAINING:CATH:PARSE_CATHGENE3D (1)           [100%] 1 of 1 ✔
[bb/4cb607] process > INTERPROSCAN:SCAN_REMAINING:CATH:PREPARE_FUNFAM (1)             [100%] 1 of 1 ✔
[67/4d7c46] process > INTERPROSCAN:SCAN_REMAINING:CATH:SEARCH_FUNFAM (1)              [100%] 1 of 1 ✔
[b7/dcda19] process > INTERPROSCAN:SCAN_REMAINING:CATH:PARSE_FUNFAM (1)               [100%] 1 of 1 ✔
[6d/d501b8] process > INTERPROSCAN:SCAN_REMAINING:CDD:SEARCH_CDD (1)                  [100%] 1 of 1 ✔
[53/12e2e5] process > INTERPROSCAN:SCAN_REMAINING:CDD:PARSE_CDD (1)                   [100%] 1 of 1 ✔
[0b/eff7c8] process > INTERPROSCAN:SCAN_REMAINING:COILS:RUN_COILS (1)                 [100%] 1 of 1 ✔
[db/98b5c4] process > INTERPROSCAN:SCAN_REMAINING:COILS:PARSE_COILS (1)               [100%] 1 of 1 ✔
[5e/4c98cb] process > INTERPROSCAN:SCAN_REMAINING:HAMAP:RUN_PFSEARCH (1)              [100%] 1 of 1 ✔
[19/88bbb3] process > INTERPROSCAN:SCAN_REMAINING:HAMAP:PARSE_PFSEARCH (1)            [100%] 1 of 1 ✔
[20/a6fcd5] process > INTERPROSCAN:SCAN_REMAINING:MOBIDBLITE:RUN_MOBIDBLITE (1)       [100%] 1 of 1 ✔
[c2/997ffd] process > INTERPROSCAN:SCAN_REMAINING:MOBIDBLITE:PARSE_MOBIDBLITE (1)     [100%] 1 of 1 ✔
[d1/e67a72] process > INTERPROSCAN:SCAN_REMAINING:NCBIFAM:SEARCH_NCBIFAM (1)          [100%] 1 of 1 ✔
[26/3f013c] process > INTERPROSCAN:SCAN_REMAINING:NCBIFAM:PARSE_NCBIFAM (1)           [100%] 1 of 1 ✔
[4f/3ac4a3] process > INTERPROSCAN:SCAN_REMAINING:PANTHER:SEARCH_PANTHER (1)          [100%] 1 of 1 ✔
[9d/82d354] process > INTERPROSCAN:SCAN_REMAINING:PANTHER:PREPARE_TREEGRAFTER (1)     [100%] 1 of 1 ✔
[69/04ba41] process > INTERPROSCAN:SCAN_REMAINING:PANTHER:RUN_TREEGRAFTER (1)         [100%] 1 of 1 ✔
[04/e50080] process > INTERPROSCAN:SCAN_REMAINING:PANTHER:PARSE_PANTHER (1)           [100%] 1 of 1 ✔
[e6/c66e6f] process > INTERPROSCAN:SCAN_REMAINING:PFAM:SEARCH_PFAM (1)                [100%] 1 of 1 ✔
[cd/b81372] process > INTERPROSCAN:SCAN_REMAINING:PFAM:PARSE_PFAM (1)                 [100%] 1 of 1 ✔
[37/e24ce2] process > INTERPROSCAN:SCAN_REMAINING:PIRSF:SEARCH_PIRSF (1)              [100%] 1 of 1 ✔
[b1/876449] process > INTERPROSCAN:SCAN_REMAINING:PIRSF:PARSE_PIRSF (1)               [100%] 1 of 1 ✔
[89/a09e92] process > INTERPROSCAN:SCAN_REMAINING:PIRSR:SEARCH_PIRSR (1)              [100%] 1 of 1 ✔
[56/c6a150] process > INTERPROSCAN:SCAN_REMAINING:PIRSR:PARSE_PIRSR (1)               [100%] 1 of 1 ✔
[ad/b19efc] process > INTERPROSCAN:SCAN_REMAINING:PRINTS:RUN_PRINTS (1)               [100%] 1 of 1 ✔
[7b/77b9a5] process > INTERPROSCAN:SCAN_REMAINING:PRINTS:PARSE_PRINTS (1)             [100%] 1 of 1 ✔
[19/0b9db2] process > INTERPROSCAN:SCAN_REMAINING:PROSITE_PATTERNS:RUN_PSSCAN (1)     [100%] 1 of 1 ✔
[4f/4a9160] process > INTERPROSCAN:SCAN_REMAINING:PROSITE_PATTERNS:PARSE_PSSCAN (1)   [100%] 1 of 1 ✔
[0d/47889b] process > INTERPROSCAN:SCAN_REMAINING:PROSITE_PROFILES:RUN_PFSEARCH (1)   [100%] 1 of 1 ✔
[f5/70a015] process > INTERPROSCAN:SCAN_REMAINING:PROSITE_PROFILES:PARSE_PFSEARCH (1) [100%] 1 of 1 ✔
[37/1706a3] process > INTERPROSCAN:SCAN_REMAINING:SFLD:SEARCH_SFLD (1)                [100%] 1 of 1 ✔
[26/8d040b] process > INTERPROSCAN:SCAN_REMAINING:SFLD:PARSE_SFLD (1)                 [100%] 1 of 1 ✔
[ab/601954] process > INTERPROSCAN:SCAN_REMAINING:SMART:PREFILTER_SMART (1)           [100%] 1 of 1 ✔
[0a/e430dc] process > INTERPROSCAN:SCAN_REMAINING:SMART:PREPARE_SMART (1)             [100%] 1 of 1 ✔
[c5/2348f2] process > INTERPROSCAN:SCAN_REMAINING:SMART:SEARCH_SMART (1)              [100%] 1 of 1 ✔
[b5/65a434] process > INTERPROSCAN:SCAN_REMAINING:SMART:PARSE_SMART (1)               [100%] 1 of 1 ✔
[b0/5ca862] process > INTERPROSCAN:SCAN_REMAINING:SUPERFAMILY:SEARCH_SUPERFAMILY (1)  [100%] 1 of 1 ✔
[75/ebde20] process > INTERPROSCAN:SCAN_REMAINING:SUPERFAMILY:PARSE_SUPERFAMILY (1)   [100%] 1 of 1 ✔
[ac/59bd55] process > INTERPROSCAN:SCAN_REMAINING:REPORT_NO_MATCHES (1)               [100%] 1 of 1 ✔
[-        ] process > INTERPROSCAN:SCAN_LOCALLY:REPORT_NO_MATCHES                     -
[57/22d6c4] process > INTERPROSCAN:COMBINE:COMBINE_MATCHES_LOCAL (1)                  [  0%] 0 of 1 ✘
[-        ] process > INTERPROSCAN:COMBINE:XREFS_LOCAL                                -
[-        ] process > INTERPROSCAN:COMBINE:REPRESENTATIVE_LOCATIONS_LOCAL             -
[-        ] process > INTERPROSCAN:OUTPUT:WRITE_GFF3_LOCAL                            -
[-        ] process > INTERPROSCAN:OUTPUT:WRITE_JSON_LOCAL                            -
[-        ] process > INTERPROSCAN:OUTPUT:WRITE_JSONL_LOCAL                           -
[-        ] process > INTERPROSCAN:OUTPUT:WRITE_TSV_LOCAL                             -
[-        ] process > INTERPROSCAN:OUTPUT:WRITE_XML_LOCAL                             -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'INTERPROSCAN:COMBINE:COMBINE_MATCHES_LOCAL (1)'

Caused by:
  null (No such file or directory) -- Check script 'modules/combine/main.nf' at line: 15


Source block:
  String outputFilePath = task.workDir.resolve('combined.json')
  ProcessCombine.run(members_matches.collect { it.toString() }, outputFilePath)

Work dir:
  /home/tll/Documents/Coding/Languages/Groovy/interproscan6/work/57/22d6c4877e3c82be3f245951bcb781

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

There is a super simple work-around where you can filter out null paths in COMBINE_MATCHES_LOCAL by replacing members_matches.collect { it.toString() } with (members_matches - null).collect { it.toString() }, but that felt like a bit of a hacky solution.

This PR moves the check for Match API compatibility earlier in the pipeline so we can branch at the same place we would for --no-matches-api. I think this makes things a bit easier to follow, and it overall shrinks the size of the codebase.

I've tried to split the PR into easy-to-understand, self-contained commits to make things easier to review!

Let me know if you have any questions or would like any changes!

This is instead of the `no_matches_api` boolean and lays the groundwork
for Match API version checking in a future commit.
If the Match API version doesn't match the requested InterPro version,
then behave as if `--no-match-api` was set.
An alternative solution here would be moving the Match API version
compatibility check to `PREPARE_APPLICATIONS`, but that requires a _lot_
of code restructuring.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant