Adding Support for Field Device Feeds

README.md

@@ -24,7 +24,7 @@ The build package tar.gz file will be located in the dist folder.
 
 ## Running the Translators Locally
 
-This set of CWZ and WZDx message translators is set up to be implemented in GCP with App Engines and Dataflows. It is also set up with raw, standard, and enhanced data feeds. This means that to take a raw icone document and generate a CWZ or WZDx message, the raw icone xml document must first be converted to 1 or multiple standard json messages (based on CDOT RTDH specification), and then each standard message may be converted into a single enhanced message. At this point, this data can be combined with other CWZ/WZDx messages, through the [combination scripts](wzdx/experimental_combination/)
+This set of CWZ and WZDx message translators is set up to be implemented in GCP with App Engines and Dataflows. It is also set up with raw, standard, and enhanced data feeds. This means that to take a CDOT planned events document and generate a CWZ or WZDx message, the raw planned event document must first be converted to 1 or multiple standard json messages (based on CDOT RTDH specification), and then each standard message may be converted into a single enhanced message. At this point, this data can be combined with other CWZ/WZDx messages, through the [combination scripts](wzdx/experimental_combination/)
 
 ### Prerequisites
 
@@ -109,44 +109,6 @@ Example usage:
 python -m wzdx.standard_to_wzdx.planned_events_translator 'wzdx/sample_files/standard/planned_events/standard_planned_event_OpenTMS-Event20643308360_westbound.json'
 ```
 
-### Execution for iCone translator
-
-#### Raw to Standard Conversion
-
-```
-python -m wzdx.raw_to_standard.icone inputfile.json --outputDir outputDirectory
-```
-
-Example usage:
-
-```
-python -m wzdx.raw_to_standard.icone 'wzdx/sample_files/raw/icone/icone_ftp_20241107-235100.xml'
-```
-
-#### Standard to CWZ Conversion
-
-```
-python -m wzdx.standard_to_cwz.icone_translator inputfile.json --outputFile outputfile.geojson
-```
-
-Example usage:
-
-```
-python -m wzdx.standard_to_cwz.icone_translator 'wzdx/sample_files/standard/icone/standard_icone_U13632784_20241107235100_1731023924_unknown.json'
-```
-
-#### Standard to WZDx Conversion
-
-```
-python -m wzdx.standard_to_wzdx.icone_translator inputfile.json --outputFile outputfile.geojson
-```
-
-Example usage:
-
-```
-python -m wzdx.standard_to_wzdx.icone_translator 'wzdx/sample_files/standard/icone/standard_icone_U13632784_20241107235100_1731023924_unknown.json'
-```
-
 ### Execution for NavJoy 568 translator
 
 This translator reads in a NavJoy 568 speed reduction form and translates it into a WZDx message. Many of the 568 messages cover 2 directions of traffic, and are thus expanded into 2 WZDx messages, one for each direction.
@@ -191,27 +153,27 @@ python -m wzdx.standard_to_wzdx.navjoy_translator 'wzdx/sample_files/standard/na
 
 ### Combine WZDx Messages
 
-These combination scripts take in a base WZDx message and an additional icone/navjoy WZDx or Geotab JSON message, and generate an enhanced WZDx message as output.
+These combination scripts take in a base WZDx message and an additional field device/navjoy WZDx or Geotab JSON message, and generate an enhanced WZDx message as output.
 
-### iCone
+### Field Device Feed
 
-Edit the files read in for iCone and WZDx messages in the main method, then run the combination script:
+Insert names for the files read in for field device feed and WZDx messages in the below combination script and then run:
 
 ```
-python icone.py wzdxFile.geojson ./iconeDirectory --outputDir ./ --updateDates true
+python -m wzdx.experimental_combination.field_devices wzdxFile.geojson deviceFeedFile.geojson --outputDir ./ --updateDates false
 ```
 
 ### Navjoy 568 form
 
-Edit the files read in for navjoy and WZDx messages in the main method, then run the combination script:
+Insert names for the files read in for navjoy-568 and WZDx messages in the below combination script and then run:
 
 ```
 python navjoy.py wzdxFile.geojson navjoyWzdxFile.geojson --outputDir ./ --updateDates true
 ```
 
 ### Geotab Vehicle (ATMA)
 
-Edit the files read in for geotab_avl and WZDx messages in the main method, then run the combination script:
+Insert names for the files read in for geotab_avl and WZDx messages in the below combination script and then run:
 
 ```
 python attenuator.py wzdxFile.geojson geotabFile.json --outputDir ./ --updateDates true

tests/data/experimental_combination/field_devices/field_device_hwy-159.json

@@ -0,0 +1,19 @@
+{
+  "id": "04435BD5-83C9-44C5-968A-7CF54C089530",
+  "type": "Feature",
+  "geometry": {
+    "type": "Point",
+    "coordinates": [-105.43859934199997, 37.19252041900006]
+  },
+  "properties": {
+    "core_details": {
+      "device_type": "arrow-board",
+      "data_source_id": "67899A97-0F3E-4683-B169-75C09C3B8F67",
+      "device_status": "ok",
+      "update_date": "2022-07-27T22:00:00Z",
+      "has_automatic_location": true,
+      "description": "Roadwork - Caution"
+    },
+    "pattern": "four-corners-flashing"
+  }
+}

tests/data/experimental_combination/field_devices/field_device_nowhere.json

@@ -0,0 +1,19 @@
+{
+  "id": "04435BD5-83C9-44C5-968A-7CF54C089530",
+  "type": "Feature",
+  "geometry": {
+    "type": "Point",
+    "coordinates": [-106.94366455078125, 39.455546939606066]
+  },
+  "properties": {
+    "core_details": {
+      "device_type": "arrow-board",
+      "data_source_id": "67899A97-0F3E-4683-B169-75C09C3B8F67",
+      "device_status": "ok",
+      "update_date": "2022-07-27T20:00:00Z",
+      "has_automatic_location": true,
+      "description": "Roadwork - Caution"
+    },
+    "pattern": "four-corners-flashing"
+  }
+}

tests/experimental_combination/field_devices_experimental_test.py

@@ -0,0 +1,139 @@
+from wzdx.experimental_combination import field_devices
+from wzdx.models.field_device_feed.field_device_feature import FieldDeviceFeature
+import json
+import time_machine
+import datetime
+
+
+def test_get_direction_from_route_details():
+    route_details = {"Direction": "a"}
+    expected = "a"
+
+    actual = field_devices.get_direction_from_route_details(route_details)
+
+    assert actual == expected
+
+
+def test_get_direction():
+    street = "I-25N"
+    coords = [
+        [-106.07316970825195, 39.190971392168045],
+        [-106.07331991195677, 39.18659739731203],
+    ]
+    route_details = {"Direction": "northbound"}
+
+    expected = "northbound"
+    actual = field_devices.get_direction(street, [])
+    assert actual == expected
+
+    expected = "southbound"
+    actual = field_devices.get_direction("", coords)
+    assert actual == expected
+
+    expected = "northbound"
+    actual = field_devices.get_direction("", [], route_details)
+    assert actual == expected
+
+    expected = "northbound"
+    actual = field_devices.get_direction(street, coords, route_details)
+    assert actual == expected
+
+
+def test_get_combined_events_valid():
+    icone_msgs = [
+        field_devices.pre_process_field_device_feature(
+            FieldDeviceFeature.model_validate_json(
+                open(
+                    "./tests/data/experimental_combination/field_devices/field_device_hwy-159.json"
+                ).read()
+            )
+        )
+    ]
+    wzdx = [
+        json.loads(
+            open(
+                "./tests/data/experimental_combination/field_devices/wzdx_combination_hwy-159.json"
+            ).read()
+        )
+    ]
+
+    with time_machine.travel(
+        datetime.datetime(2022, 7, 27, 22, 0, 0, 0, tzinfo=datetime.timezone.utc)
+    ):
+        expected = field_devices.get_combined_events(icone_msgs, wzdx)
+    assert len(expected) == 1
+
+
+def test_get_combined_events_invalid():
+    icone_msgs = [
+        field_devices.pre_process_field_device_feature(
+            FieldDeviceFeature.model_validate_json(
+                open(
+                    "./tests/data/experimental_combination/field_devices/field_device_hwy-159.json"
+                ).read()
+            )
+        )
+    ]
+    wzdx = [
+        json.loads(
+            open(
+                "./tests/data/experimental_combination/field_devices/wzdx_combination_nowhere.json"
+            ).read()
+        )
+    ]
+
+    with time_machine.travel(
+        datetime.datetime(2022, 7, 27, 20, 0, 0, 0, tzinfo=datetime.timezone.utc)
+    ):
+        expected = field_devices.get_combined_events(icone_msgs, wzdx)
+    assert len(expected) == 0
+
+
+def test_get_combined_events_invalid_different_routes():
+    icone_msgs = [
+        field_devices.pre_process_field_device_feature(
+            FieldDeviceFeature.model_validate_json(
+                open(
+                    "./tests/data/experimental_combination/field_devices/field_device_hwy-159.json"
+                ).read()
+            )
+        )
+    ]
+    wzdx = [
+        json.loads(
+            open(
+                "./tests/data/experimental_combination/field_devices/wzdx_combination_co-9.json"
+            ).read()
+        )
+    ]
+
+    with time_machine.travel(
+        datetime.datetime(2022, 7, 27, 20, 0, 0, 0, tzinfo=datetime.timezone.utc)
+    ):
+        expected = field_devices.get_combined_events(icone_msgs, wzdx)
+    assert len(expected) == 0
+
+
+def test_get_combined_events_invalid_no_routes():
+    icone_msgs = [
+        field_devices.pre_process_field_device_feature(
+            FieldDeviceFeature.model_validate_json(
+                open(
+                    "./tests/data/experimental_combination/field_devices/field_device_nowhere.json"
+                ).read()
+            )
+        )
+    ]
+    wzdx = [
+        json.loads(
+            open(
+                "./tests/data/experimental_combination/field_devices/wzdx_combination_nowhere_2.json"
+            ).read()
+        )
+    ]
+
+    with time_machine.travel(
+        datetime.datetime(2022, 7, 27, 20, 0, 0, 0, tzinfo=datetime.timezone.utc)
+    ):
+        expected = field_devices.get_combined_events(icone_msgs, wzdx)
+    assert len(expected) == 0