Complete OTTR Engine

[areleu]

Hello,

I find this project very promising and I consider that it can become a very powerful tool. I was wondering if you have intention of integrating OTTR at full in this tool. I would fin particularly useful to be able to define the templates outside of the code in one or more of the possible OTTR serializations and the support of bOTTR for relational database integration. I think for both of those there are direct workarounds directly in python, but given how the language works, it seems to me that modularization may become a pain in moderately sized knowledge bases.

Thanks for the good work!

PS: I see you have an stOTTR parser so probably this solves half of the issue, https://github.com/DataTreehouse/maplib/blob/main/lib/templates/src/parsing.rs

[magbak]

Thanks @areleu , and thanks for reaching out!
There is indeed stOTTR-support, but there is currently no bOTTR-support.
We have considered adding it, but have not prioritized it.

The current way we do relational database integration is to do it via the rich ecosystem surrounding DataFrames, e.g. with Polars like this:

import polars as pl
from maplib import Mapping

...

df = pl.read_database(
    query=my_query,
    connection=my_conn,
)

m = Mapping(my_stottr_doc) 
m.expand(my_template_iri, df)

Where:

• my_query is an SQL query
• my_conn is a connection (e.g. SQLAlchemy) as specified here
• my_stottr_doc is a stOTTR document string
• my_template_iri is a template IRI.

This approach has the advantage that the DataFrame (df) can be manipulated in a flexible way using e.g. Polars (this example) or Pandas, doing things that could be tricky in an SQL, for debugging the mapping and adding asserts that catch certain errors in the input data at an early stage.

But as you imply, there are more moving parts in this setup, and you may prefer to define the inputs and mapping in a single declarative configuration.

Feel free to reach out via email (magnus at data-treehouse.com) if you want to discuss your use case or how we could add bOTTR support!

Best regards,
Magnus

[areleu]

I think I will follow with this approach for now, I actually find your API friendlier than writing stOTTR files. I have a technical question. Is it possible to have multiple mappings and conditionally expand them based on the contents of the query, is this a feature considered in the datalog implementation?

[magbak]

If there are only two different templates, you could do two different queries. On the other hand, if there are multiple templates each for use in a different case, I would look into the partition_by functionality, perhaps using as_dict=True.

Edit: Forgot to answer: Have not considered conditional expansion, will have a think about it :)

[magbak]

In model transformation languages such as Eclipse Transformation Language, there is the concept of a guard. Perhaps that could be something to implement.