Allow partial comparison in nvbench_compare.py

[bernhardmgruber]

Fixes: #295

Can compare the files in #295:

['base.json', 'base_bif.json']
# mul

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 253.092 us |       0.42% | 259.197 us |       0.35% | 6.104 us |   2.41% |   SLOW   |
|   I16   |      I64      |      2^28      | 253.069 us |       0.42% | 259.680 us |       0.37% | 6.612 us |   2.61% |   SLOW   |

# add

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 371.395 us |       0.22% | 374.366 us |       0.24% | 2.972 us |   0.80% |   SLOW   |
|   I16   |      I64      |      2^28      | 371.890 us |       0.21% | 374.705 us |       0.25% | 2.815 us |   0.76% |   SLOW   |

# triad

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 373.139 us |       0.21% | 374.479 us |       0.24% | 1.340 us |   0.36% |   SLOW   |
|   I16   |      I64      |      2^28      | 373.672 us |       0.21% | 374.310 us |       0.24% | 0.638 us |   0.17% |   SAME   |

# nstream

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |      Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|-----------|---------|----------|
|   I16   |      I32      |      2^28      | 492.228 us |       0.08% | 488.943 us |       0.15% | -3.285 us |  -0.67% |   FAST   |
|   I16   |      I64      |      2^28      | 492.258 us |       0.13% | 489.330 us |       0.16% | -2.928 us |  -0.59% |   FAST   |

# Summary

- Total Matches: 8
  - Pass    (diff <= min_noise): 1
  - Unknown (infinite noise):    0
  - Failure (diff > min_noise):  7

[oleksandr-pavlyk]

Should we check that intersection of hay["axes"] and need["axes"] is not empty?

[oleksandr-pavlyk]

I had something as follows in mind:

def axes_do_intersect(hay, needle):
    if (len(hay) != len(needle)):
        return False
    for ha, na in zip(hay, needle):
        if any(ha[k] != na[k] for k in ["name", "type", "flags"]):
            return False
        hv, nv = ha["values"], na["values"]
        if not any(v in hv for v in nv):
            return False
    return True


def find_matching_bench(needle, haystack):
    for hay in haystack:
        if hay["name"] == needle["name"] and axes_do_intersect(hay["axes"], needle["axes"]):
            return hay
    return None