Event headers table should be a typed object #51
Description
Currently the event (and lesser extent run) header table that still originates from the Virtex TOS ascii data days is kept as a Table[string, string] after parsing & until writing to the H5 file in raw_data_manipulation.
This is both error prone (in the sense of typos etc.) and very inefficient, as the majority of fields (but not all, e.g. "DateTime") are actually integers.
In addition for other data sources, e.g. Timepix3, data doesn't even arrive in string format. So we have an especially unnecessary back and forth conversions to string land.
Instead we should:
- either define a typed object
EventHeader - or have a table of something like
Table[string, EventHeaderField]
with theoretical implementation ideas like:
type
EventHeader =
dateTime: DateTime
timestamp: int
case timepix: TimepixVersion # or TOS version field?
of Timepix1:
useHvFadc: bool
... or alternatively:
type
EventHeaderFieldKind = enum
ekString, ekBool, ekFloat, ekDateTime
EventHeaderField = object
case kind: EventHeaderFieldKind
of ekString: s: string
of ekBool: b: bool
...etc.
I'd do it now, but it touches too many procedures etc. to quickly make the change.
This would also allow us to get rid of fully optional fields like useHvFadc in detectors / data sources for which it makes no sense etc.
The first solution has the advantage of being fully typed, while the second still allows for trivial extensions (i.e. adding a new scintillator field etc.).
The good thing about the second one is that at that point the code simplifies significantly as we have all type information ready in the table. So we don't have to hardcode the fields that are e.g. int or string anymore and can just check the kind of the field and generate a correct resulting H5Dataset based on that. Maybe that's the best of both worlds.