-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Problem
When the MDL converter resolves GET DIRECT DATA/CONSTANTS/LOOKUPS/SUBSCRIPT expressions via the DataProvider, the resolved equation (baked lookup/constant values) replaces the opaque GET DIRECT string, but the Compat.data_source metadata (file, tab, cell) is not populated. This means models imported from MDL with GET DIRECT references lose their external-data provenance -- they only retain baked values and cannot be exported back with reconstructable GET DIRECT intent.
The DataSource struct and the Compat.data_source field exist and are properly populated by the XMILE, protobuf, and JSON deserializers, but the MDL conversion path skips this step.
Concrete example
In src/simlin-engine/src/mdl/convert/variables.rs, try_resolve_data_equation (around lines 797-834) calls into the DataProvider to resolve expressions like GET DIRECT CONSTANTS('file.xlsx', 'Sheet1', 'B2'). The resolved constant/lookup replaces the equation, but no code sets compat.data_source to preserve the original file path, tab/delimiter, and cell reference.
Why it matters
- Roundtrip fidelity: An MDL model imported and then exported back to MDL (via the MDL writer) will lose GET DIRECT references, replacing them with inlined data. This is a silent semantic change.
- Provenance tracking: Users cannot see where external data came from after import.
- Correctness of related work: Issues Add protobuf support for EXCEPT equations, element-level dimension mappings, and DataSource metadata #341 and Add XMILE support for EXCEPT equations, element-level dimension mappings, and DataSource metadata #342 add protobuf and XMILE serialization support for
DataSource, but those serialization paths will always seeNonefor MDL-imported models since the conversion step never populates the field.
Components affected
src/simlin-engine/src/mdl/convert/variables.rs--try_resolve_data_equationfunction
Possible approach
In try_resolve_data_equation, after successfully resolving a GET DIRECT expression, parse the function arguments (file, tab/delimiter, row/col, cell) into a DataSource struct and set it on variable.compat.data_source. The arguments are already available as parsed AST nodes at the call site.
Context
Identified during review of the mdl-full-compat branch. Related to #341, #342, and #348, but those issues cover serialization formats and simulation support respectively, not the MDL import path's metadata population.