Validation errors

[lcoutos]

Hi, I'm working with a large variants dataset, and some of them raised some validation bugs. I have four types of issues (I can make a separate issue for each of them if it is easier for you):

NM_012234.7 errors

I have three variants on NM_012234.7 that are normalized as 5'UTR variants by VariantValidator, but missense variants by Mutalyzer and VEP:

Input	assembly	url	output	expected (Mutalyzer)
NC_000003.12:g.72446581A>C	GRCh38	https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh38/NC_000003.12:g.72446581A%3EC/select?content-type=application%2Fjson	NM_012234.7:c.-141=	NC_000003.12(NM_012234.7):c.43T>G, p.(Trp15Gly)
NC_000003.12:g.72446533A>C	GRCh38	https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh38/NC_000003.12:g.72446533A%3EC/select?content-type=application%2Fjson	NM_012234.7:c.-93=	NC_000003.11(NM_012234.7):c.91T>G
NC_000003.12:g.72446553T>C	GRCh38	https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh38/NC_000003.12:g.72446553T%3EC/select?content-type=application%2Fjson	NM_012234.7:c.-113=	NC_000003.12(NM_012234.7):c.71A>G

"N" as Ref base

For two variants on NM_001017915.3, the validated result as "N" as Reference sequence instead of the input ref:

Input	assembly	url	output	expected (Mutalyzer)
chr2:233998842:C:T	GRCh37	https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh37/chr2:233998842:C:T/select?content-type=application%2Fjson	NC_000002.11:g.234052046N>T - NM_001017915.3:c.739-2749N>T	NC_000002.11:g.233998842C>T - NM_001017915.3:c.663+3486C>T
chr2:233998852:G:A	GRCh37	https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh37/chr2:233998852:G:A/select?content-type=application%2Fjson	NC_000002.11:g.234052056N>A - NM_001017915.3:c.739-2739N>A	NC_000002.11:g.233998852G>A - NM_001017915.3:c.663+3496G>A

Correct validation by VariantValidator but not VariantFormatter

I have a set of dup variants with a correct HGVSp prediction in VariantValidator, but not in VariantFormatter (Alt amino acids added after "dup"):

Input	assembly	url	output (VariantFormatter)	expected (VariantValidator)
NC_000012.12:g.50086346_50086348dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12%3Ag.50086346_50086348dup/refseq/NM_003076.4/False?content-type=application%2Fjson	NP_003067.3:p.(Asn122dupN)	NP_003067.3:p.(Asn122dup)
NC_000012.12:g.50086346_50086348dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.50086346_50086348dup/refseq/NM_003076.4/False?content-type=application%2Fjson	NP_003067.3:p.(Asn122dupN)	NP_003067.3:p.(Asn122dup)
NC_000012.12:g.132643541_132643543dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.132643541_132643543dup/refseq/NM_006231.2/False?content-type=application%2Fjson	NP_006222.2:p.(Ala1437dupA)	NP_006222.2:p.(Ala1437dup)
NC_000001.11:g.26697128_26697130dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000001.11:g.26697128_26697130dup/refseq/NM_006015.4/False?content-type=application%2Fjson	NP_006006.3:p.(Gly242dupG)	NP_006006.3:p.(Gly242dup)
NC_000005.10:g.168762604_168762612dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000005.10:g.168762604_168762612dup/refseq/NM_003062.3/False?content-type=application%2Fjson	NP_003053.1:p.(Gly513_Ile515dupGTI)	NP_003053.1:p.(Gly513_Ile515dup)
NC_000023.11:g.67546511_67546519dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000023.11:g.67546511_67546519dup/refseq/NM_000044.3/False?content-type=application%2Fjson	NP_000035.2:p.(Gly471_Gly473dupGGG)	NP_000035.2:p.(Gly471_Gly473dup)
NC_000013.11:g.109785992_109785994dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000013.11:g.109785992_109785994dup/refseq/NM_003749.2/False?content-type=application%2Fjson	NP_003740.2:p.(Asn28dupN)	NP_003740.2:p.(Asn28dup)
NC_000014.9:g.37592322_37592324dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000014.9:g.37592322_37592324dup/refseq/NM_004496.3/False?content-type=application%2Fjson	NP_004487.2:p.(Gly157dupG)	NP_004487.2:p.(Gly157dup)
NC_000001.11:g.26696529_26696531dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000001.11:g.26696529_26696531dup/refseq/NM_006015.4/False?content-type=application%2Fjson	NP_006006.3:p.(Ala45dupA)	NP_006006.3:p.(Ala45dup)
NC_000012.12:g.25245353_25245358dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.25245353_25245358dup/refseq/NM_004985.3%7CNM_033360.2/False?content-type=application%2Fjson	NP_004976.2:p.(Ala11_Gly12dupAG)	NP_004976.2:p.(Ala11_Gly12dup)
NC_000012.12:g.45849629_45849631dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.45849629_45849631dup/refseq/NM_152641.2/False?content-type=application%2Fjson	NP_689854.2:p.(Asn589dupN)	NP_689854.2:p.(Asn589dup)
NC_000007.14:g.55181309_55181317dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000007.14:g.55181309_55181317dup/refseq/NM_005228.3/False?content-type=application%2Fjson	NP_005219.2:p.(Ala767_Val769dupASV)	NP_005219.2:p.(Ala767_Val769dup)
NC_000023.11:g.67546558_67546566dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000023.11:g.67546558_67546566dup/refseq/NM_000044.3/False?content-type=application%2Fjson	NP_000035.2:p.(Gly471_Gly473dupGGG)	NP_000035.2:p.(Gly471_Gly473dup)
NC_000017.11:g.7673790_7673792dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000017.11:g.7673790_7673792dup/refseq/NM_001126112.2/False?content-type=application%2Fjson	NP_001119584.1:p.(Cys277dupC)	NP_001119584.1:p.(Cys277dup)
NC_000023.11:g.154428803_154428808dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000023.11:g.154428803_154428808dup/refseq/NM_001183.4/False?content-type=application%2Fjson	NP_001174.2:p.(Ala40_Ala41dupAA)	NP_001174.2:p.(Ala40_Ala41dup)
NC_000009.12:g.21974775_21974780dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000009.12:g.21974775_21974780dup/refseq/NM_000077.4/False?content-type=application%2Fjson	NP_000068.1:p.(Thr18_Ala19dupTA)	NP_000068.1:p.(Thr18_Ala19dup)
NC_000007.14:g.140753340_140753342dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000007.14:g.140753340_140753342dup/refseq/NM_004333.4/False?content-type=application%2Fjson	NP_004324.2:p.(Thr599dupT)	NP_004324.2:p.(Thr599dup)
NC_000007.14:g.140753338_140753340dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000007.14:g.140753338_140753340dup/refseq/NM_004333.4/False?content-type=application%2Fjson	NP_004324.2:p.(Thr599dupT)	NP_004324.2:p.(Thr599dup)
NC_000017.11:g.7670666_7670671dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000017.11:g.7670666_7670671dup/refseq/NM_001126112.2/False?content-type=application%2Fjson	NP_001119584.1:p.(Ala347_Leu348dupAL)	NP_001119584.1:p.(Ala347_Leu348dup)
NC_000007.14:g.98910180_98910212dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000007.14:g.98910180_98910212dup/refseq/NM_001244580.1/False?content-type=application%2Fjson	NP_001231509.1:p.(Ala492_Pro502dupAAPGPAPSPAP)	NP_001231509.1:p.(Ala492_Pro502dup)
NC_000005.10:g.112841957_112841959dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000005.10:g.112841957_112841959dup/refseq/NM_000038.5/False?content-type=application%2Fjson	NP_000029.2:p.(Ala2122dupA)	NP_000029.2:p.(Ala2122dup)
NC_000004.12:g.54726014_54726019dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000004.12:g.54726014_54726019dup/refseq/NM_000222.2/False?content-type=application%2Fjson	NP_000213.1:p.(Ala502_Tyr503dupAY)	NP_000213.1:p.(Ala502_Tyr503dup)
NC_000012.12:g.25245346_25245348dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.25245346_25245348dup/refseq/NM_004985.3%7CNM_033360.2/False?content-type=application%2Fjson	NP_004976.2:p.(Gly13dupG)	NP_004976.2:p.(Gly13dup)
NC_000017.11:g.7674245_7674250dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000017.11:g.7674245_7674250dup/refseq/NM_001126112.2/False?content-type=application%2Fjson	NP_001119584.1:p.(Asn239_Ser240dupNS)	NP_001119584.1:p.(Asn239_Ser240dup)
NC_000017.11:g.1656537_1656545dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000017.11:g.1656537_1656545dup/refseq/NM_006445.3/False?content-type=application%2Fjson	NP_006436.3:p.(Asn1881_Val1883dupNIV)	NP_006436.3:p.(Asn1881_Val1883dup)
NC_000012.12:g.25245355_25245357dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000012.12:g.25245355_25245357dup/refseq/NM_004985.3%7CNM_033360.2/False?content-type=application%2Fjson	NP_004976.2:p.(Gly10dupG)	NP_004976.2:p.(Gly10dup)
NC_000016.10:g.31185099_31185101dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000016.10:g.31185099_31185101dup/refseq/NM_004960.3/False?content-type=application%2Fjson	NP_004951.1:p.(Gly231dupG)	NP_004951.1:p.(Gly231dup)
NC_000019.10:g.11012988_11012990dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000019.10:g.11012988_11012990dup/refseq/NM_001128844.1%7CNM_001128849.1%7CNM_003072.3/False?content-type=application%2Fjson	NP_001122316.1:p.(Asn772dupN)	NP_001122316.1:p.(Asn772dup)
NC_000004.12:g.54726014_54726019dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000004.12:g.54726014_54726019dup/refseq/NM_000222.2/False?content-type=application%2Fjson	NP_000213.1:p.(Ala502_Tyr503dupAY)	NP_000213.1:p.(Ala502_Tyr503dup)
NC_000004.12:g.54727489_54727527dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000004.12:g.54727489_54727527dup/refseq/NM_000222.2/False?content-type=application%2Fjson	NP_000213.1:p.(Thr574_Arg586dupTQLPYDHKWEFPR)	NP_000213.1:p.(Thr574_Arg586dup)
NC_000004.12:g.54727487_54727522dup	GRCh38	https://rest.variantvalidator.org/VariantFormatter/variantformatter/GRCh38/NC_000004.12:g.54727487_54727522dup/refseq/NM_000222.2/False?content-type=application%2Fjson	NP_000213.1:p.(Thr574_Pro585dupTQLPYDHKWEFP)	NP_000213.1:p.(Thr574_Pro585dup)

Simple SNP to large delins

The GRCh37 chr11:118650341:C:T variant is normalized as simple SNP for most of transcripts, and very large delins for some of them (included new versions of previous ones): https://rest.variantvalidator.org/VariantValidator/variantvalidator/GRCh37/chr11%3A118650341%3AC%3AT/select?content-type=application%2Fjson
Mutalyzer gives me a correct SNP for all transcripts (input: "GRCh37(chr11):g.118650341C>T" | output: "NC_000011.9(NM_004397.6):c.369G>A")

Those errors are all raised by the python library and REST api.
Thanks a lot for your help!

[Peter-J-Freeman]

@lcoutos . Thanks for these. Will investigate.

For these

I have three variants on NM_012234.7 that are normalized as 5'UTR variants by VariantValidator, but missense variants by Mutalyzer and VEP:

The mappings used by VEP and Mutalyzer are often incorrect, but I will check. VV is likely to be much more reliable. We get all our data including alignment data from source at the NIH and Ensembl

[Peter-J-Freeman]

@John-F-Wagstaff to take the first 2 points, I will take the 2nd too

[lcoutos]

The mappings used by VEP and Mutalyzer are often incorrect, but I will check. VV is likely to be much more reliable. We get all our data including alignment data from source at the NIH and Ensembl

Good to know!
For those ones I quicly checked on UCSC before submitting the issue, and the variant is supposed to be on the CDS region (if I didn't make any mistake in the process).
Thanks for your quick response

[Peter-J-Freeman]

@lcoutos. @John-F-Wagstaff is going to check the alignment. We will give you a complete report on this.

So far I have solved

Simple SNP to large delins

in dev at the moment. As always, once we are ready for testing and acceptance, will let you know

[Peter-J-Freeman]

NP_003067.3:p.(Asn122dupN)

OK, this is odd, should not happen. Likely a very easy fix!!!

[Peter-J-Freeman]

Fixed the ref bases in p. descriptions now. In dev anyway. @lcoutos Good eye. I had never noticed. Thanks. Will push changes ASAP. We are still working on a few more little treats for the community. Keep you posted.

@John-F-Wagstaff what are the alignments looking like r.e. the first couple of points?

[John-F-Wagstaff]

Hello Pete

Sorry for the delay I had been digging into the first problem, but still can't figure out what Mutalyzer is doing.

For NM_012234.7 against NC_000003.12 the alignment spans 72374596 to 72446623 (in 0 based genomic coordinates) it is also -ve WRT the genome. The CDS starts at base 183 and ends at base 870 within the transcript. The first variant will therfore be 43 bp from the start of the transcript mapping (in 1 based coordinates for the transcript), which, given that we are - strand WRT the genome, is backwards from the end of the span. Accounting for 0 vs 1 based coordinates this all seems to line up with the results.

Since this mapping happens within the '2314=' at the start of the first(within transcript) mapped exon the mapping should be fairly simple. As such I am confidant that we are doing the right thing here, or else a lot of other obviously wrong things would be happening.

The CIGAR includes a few deletions (something like 6 bases) and indels WRT to the genome are something which Mutalyzer will ignore when mapping (until they fix this I do NOT recommend trusting ANY Mutalyzer mappings for any transcripts with gaps). This however should usually only affect bases after the gap, and again only shift bases by ~6 at worst.

Given the length of the shift this gap issue should not cause the level of difference we are seeing here, which is more than 146 bases by default if Mutalyzer puts the variant inside the CDS, even without looking up the Mutalyzer result,. As such I am left a little stumped as to what is going on. The only thing I can think of here is some problem with -ve transcripts?

[Peter-J-Freeman]

@John-F-Wagstaff so it's a gap gene?

If so then I agree and would say

@lcoutos Mutalyzer and VEP will always get things wrong in such genes. Always!!! So if you use them, I recommend using VV or the hgvs python library by GA4GH to validate / double check the nomenclature and rely more on VV of the hgvs tool for accuracy. If you want a full list of affeced genes, we can provide

[John-F-Wagstaff]

For the others is is a lot more simple

from biocommons.seqrepo import SeqRepo
sr = SeqRepo('/home/n95911jw/VV_SR_2024_09/master')
sr['NC_000002.11'][234052040:234052050]

returns
'NNNNNNNNNN'

The base gets corrected to match the reference. There is not a good way of specifying this (asserting that the N in a reference is a specific base, to start with, but then transitions to another base) but there are several types of bad outcome if you try to use these variants as-is for other work. Using a different reference is probably for the best, whenever possible for this type of variant.

[Peter-J-Freeman]

The base gets corrected to match the reference. There is not a good way of specifying this (asserting that the N in a reference is a specific base, to start with, but then transitions to another base) but there are several types of bad outcome if you try to use these variants as-is for other work. Using a different reference is probably for the best, whenever possible for this type of variant.

Ah, I see and also as I suspected

@lcoutos, unlike most tools VV actually checks the reference sequence after liftover and checks the reference base. It assumes the ALT is correct because that was observed by the user, but since reference bases at aligned positions may differ, it will automatically update the reference and the coresponding nomenclature

[lcoutos]

Thanks a lot for your answers! I usually don't use Mutalyzer, only for a quick check when I have something strange in VV, but some of my colleagues use it, I will warn them about those issues.

For the others is is a lot more simple

from biocommons.seqrepo import SeqRepo
sr = SeqRepo('/home/n95911jw/VV_SR_2024_09/master')
sr['NC_000002.11'][234052040:234052050]

returns 'NNNNNNNNNN'

The base gets corrected to match the reference. There is not a good way of specifying this (asserting that the N in a reference is a specific base, to start with, but then transitions to another base) but there are several types of bad outcome if you try to use these variants as-is for other work. Using a different reference is probably for the best, whenever possible for this type of variant.

For this point, I will find a way to manage it in my scripts, it made me realized that the output HGVSc for (intronic) variants should be homogenized on GRCh38 and not depend of the input assembly.

For NM_012234.7 against NC_000003.12 the alignment spans 72374596 to 72446623 (in 0 based genomic coordinates) it is also -ve WRT the genome. The CDS starts at base 183 and ends at base 870 within the transcript. The first variant will therfore be 43 bp from the start of the transcript mapping (in 1 based coordinates for the transcript), which, given that we are - strand WRT the genome, is backwards from the end of the span. Accounting for 0 vs 1 based coordinates this all seems to line up with the results.

Since this mapping happens within the '2314=' at the start of the first(within transcript) mapped exon the mapping should be fairly simple. As such I am confidant that we are doing the right thing here, or else a lot of other obviously wrong things would be happening.

The CIGAR includes a few deletions (something like 6 bases) and indels WRT to the genome are something which Mutalyzer will ignore when mapping (until they fix this I do NOT recommend trusting ANY Mutalyzer mappings for any transcripts with gaps). This however should usually only affect bases after the gap, and again only shift bases by ~6 at worst.

Given the length of the shift this gap issue should not cause the level of difference we are seeing here, which is more than 146 bases by default if Mutalyzer puts the variant inside the CDS, even without looking up the Mutalyzer result,. As such I am left a little stumped as to what is going on. The only thing I can think of here is some problem with -ve transcripts?

For this one, I tried to understand what is happening, but what I see on NCBI viewer is discordant with VV and mutalyzer, so I'm afraid I am missing something.

Here is a set of screenshots of the viewer. According to the alignment, a whole part of the transcript is not aligned. All the beginning of the transcript and the protein are absent (first panel, the first aa of the protein is noted as 11 and second panel for the transcript, the first base is at position 214 for mRNA and 31 for CDS). If I select the 72446581 position (last panel) the CDS position is 73 and protein position 25. If I'm correct on this, my HGVSp should be p.(Trp25Gly), which is incoherent with VV AND Mutalyzer. I tried to look at a close variant from dbSNP and for a chr3:72446585 (GRCh38.p14) variant, the HGVS descriptions are NM_012234.7:c.69= and NP_036366.3:p.Gly23=, which seems coherent with my observations.

And for the other points, thanks for your help, I will try to test it as soon as possible

[ifokkema]

Some observations / context:

• @lcoutos Please note that NC(NM) does not equal NM. As such, Mutalyzer's NC_000003.12(NM_012234.7):c.43T>G has nothing to do with NM_012234.7:c.43T>G. This is super confusing, but unfortunately, it is how it currently is. I'm currently discussing with the Mutalyzer team how this can be resolved, but it's not simple. We will discuss this more broadly in two weeks with the rest of the HVNC (HGVS Variant Nomenclature Committee). In the meantime, whenever Mutalyzer produces something like NC_000003.12(NM_012234.7), note that this uses a certain alignment of the NM onto the NC, but it uses only the NC sequence. As such, the cDNA sequence as well as the protein sequence sometimes make no sense whatsoever, compared with the actual NM cDNA and protein sequence. Particularly, it is out of frame whenever there are gapped alignments.
• all: have a look at this genome browser view. According to the UCSC genome browser, NC_000003.12:g.72446581A>C indeed overlaps with NM_012234.7's p.Trp15, as Mutalyzer suggested. So whatever the UCSC is using, Mutalyzer is using it too. Most importantly, zoom out a bit and notice that this transcript mapping makes no sense. It has no 5' UTR and starts with an Arginine.

[John-F-Wagstaff]

Saying that NC(NM) does not equal NM is putting it kindly, Mutalyzer has strong opinions on the matter, which clash with ours, and I think most other's. For us NC(NM) is, if it is not intronic, exactly equal to NM, in terms of sequence. I believe that most users think that this is the case, and would be quite unpleasantly surprised to find about Mutalyzer's opinion on the matter.

To get some idea why it helps to realise that the whole coordinates for a transcript change if the coordinate origin is the other side of even the smallest ins/dels between NM and NC(NM), if Mutalyzer is correct. This effect, combined with the fact that some kind of NC(NM) is needed for introns, makes reading and understanding transcript changes near intron/exon boundaries quite unpleasant if you agree with them. This is true even if you know exactly what you are looking at, but things could easily end up incomprehensible if you do not already know the exact size and location of any ins/dels in the transcript alignment you are using, before you start. Worse sometimes the genomic version of the CDS does not even work (start or end properly etc) for some transcripts giving rise to the "fun" question of where to hang a :c. coordinate off in this case, or how to handle post CDS coordinates in the case of different CDS lengths (including frame-shift derived early termination for the genomic version).

Despite this Mutalyzer's stance has confused the issue, so avoiding NC_000003.12(NM_012234.7) type statements where possible for exonic data, or at least supplementing them with pure NC coordinates for introns, may be good practice until this is sorted out.

From what I remember from my last look at the code Mutalyzer uses UCSC data, which is based on blat alignments, we use the official RefSeq alignments, which are done with NCBI's splign.