Bug of simulating GPS data

[JiaxinCCC]

Hi @GeorgeEfstathiadis @hackdna,

I followed the code below to simulate GPS data.

import datetime

from forest.bonsai.simulate_gps_data import gps_to_csv, sim_gps_data
from forest.bonsai.simulate_log_data import sim_log_data
from forest.jasmine.traj2stats import Frequency, gps_stats_main
from forest.willow.log_stats import log_stats_main

# 1. If you don't have any smartphone data (yet), you can generate fake data
path_to_synthetic_gps_data = "/home/ubuntu/projects/gps/forest/tutorials/simu_gps_data"
path_to_synthetic_log_data = "/home/ubuntu/projects/gps/forest/tutorials/simu_log_data"
path_to_gps_summary = "/home/ubuntu/projects/gps/forest/tutorials/simu_gps_output"
path_to_log_summary = "/home/ubuntu/projects/gps/forest/tutorials/simu_log_output"

# Generate synthetic GPS data and communication log data as CSV files
# Define parameters for generating the data
# To conserve smartphone battery power, we typically collect location data intermittently: e.g., an on-cycle of 3 minutes followed by an off-cycle of 12 minutes. We'll generate data in this way.
# Number of persons to generate
n_persons = 1
# Location of persons to generate, format: Country_2_letter_ISO_code/City_Name
location = "GB/Bristol"
# Start date for generated trajectories
start_date = datetime.date(2021, 10, 1)
# End date for trajectories
end_date = datetime.date(2021, 10, 5)
# API key for OpenRouteService, generated from https://openrouteservice.org/
#api_key = "5b3ce3597851110001cf6248359f91eec01b4e09ae4ceced5a7d04d1"
api_key = "mock_key"
# Length of off-cycle + length of on-cycle in minutes
cycle = 15
# Length off-cycle / (length off-cycle + length on-cycle)
percentage = 0.8
# Dictionary of personal attributes for each user, set to None if random; check Attributes class for usage in simulate_gps_data module
personal_attributes = {
    "User 1": {
        "main_employment": "none",
        "vehicle": "car",
        "travelling_status": 10,
        "active_status": 7,
    },
    "Users 2-4": {
        "main_employment": "university",
        "vehicle": "bicycle",
        "travelling_status": 8,
        "active_status": 8,
        "active_status-16": 2,
    },
    "User 5": {
        "main_employment": "office",
        "vehicle": "foot",
        "travelling_status": 9,
        "travelling_status-20": 1,
        "preferred_exits": ["cafe", "bar", "cinema"],
    },
}
sample_gps_data = sim_gps_data(
    n_persons,
    location,
    start_date,
    end_date,
    cycle,
    percentage,
    api_key,
    personal_attributes,
)

However, I encountered the following error, which seems like a bug in the code.

File [~/miniconda3/envs/gps/lib/python3.11/site-packages/forest/bonsai/simulate_gps_data.py:895](http://localhost:8888/lab/tree/Desktop/GPS_forest/~/miniconda3/envs/gps/lib/python3.11/site-packages/forest/bonsai/simulate_gps_data.py#line=894), in gen_all_traj(person, switches, start_date, end_date, api_key)
    870 """Generates trajectories for a single person.
    871 
    872 Args:
   (...)
    891     ValueError: if no offices around person's house address
    892 """
    894 if len(person.possible_destinations) < 4:
--> 895     raise ValueError("Not enough possible destinations")
    896 if (
    897     person.attributes.main_occupation != Occupation.NONE
    898     and person.office_coordinates == (0, 0)
    899 ):
    900     raise ValueError("No office coordinates")

ValueError: Not enough possible destinations

Do you have any suggestions on how I might resolve this error? Thanks for your help!

[hackdna]

Hi @JiaxinCCC, I'll let @GeorgeEfstathiadis address the error but I've noticed that you may have exposed your API key for OpenRouteService above. If so, you should invalidate that key and generate a new one.

[JiaxinCCC]

Thank you for catching that! I’ll revoke the API key and generate a new one to secure access. I appreciate the heads-up. I’ll also follow up with George about the error.

[GeorgeEfstathiadis]

Hi @JiaxinCCC , so I ran the code myself as is in the code segmented you provided and was not able to replicate your error. Did you perhaps modify the location or some other parameter to fit your use-case? In order to replicate the error you got I would need the exact script you ran.

This error is raised when the location of the simulated participant doesn't have more than 4 amenity type locations around it to basically simulate GPS trajectories. So it can be that either the location provided is very secluded or wrong spelling, some issue with OSM for that location or some bug in our code, but I can't be definitive right now which one it is.

[JiaxinCCC]

Hi @JiaxinCCC , so I ran the code myself as is in the code segmented you provided and was not able to replicate your error. Did you perhaps modify the location or some other parameter to fit your use-case? In order to replicate the error you got I would need the exact script you ran.

This error is raised when the location of the simulated participant doesn't have more than 4 amenity type locations around it to basically simulate GPS trajectories. So it can be that either the location provided is very secluded or wrong spelling, some issue with OSM for that location or some bug in our code, but I can't be definitive right now which one it is.

@GeorgeEfstathiadis thanks for your reply! My Python version is Python 3.11.10 and the package version is forest==0.1.1. I ran the following code, which comes from README.md:

# Currently, all imports from `forest` must be explicit.  For the below example you need to import the following
# In future, it would be great to have all functions import automatically
import datetime

from forest.bonsai.simulate_gps_data import gps_to_csv, sim_gps_data
from forest.bonsai.simulate_log_data import sim_log_data
from forest.jasmine.traj2stats import Frequency, gps_stats_main
from forest.willow.log_stats import log_stats_main

# 1. If you don't have any smartphone data (yet) you can generate fake data
path_to_synthetic_gps_data = "simu_gps_data"
path_to_synthetic_log_data = "simu_log_data"
path_to_gps_summary = "simu_gps_output"
path_to_log_summary = "simu_log_output"

# Generate fake call and text logs
# Because of the explicit imports, you don't have to precede the functions with forest.subpackage.
sim_log_data(path_to_synthetic_log_data)

# Generate synthetic gps data and communication logs data as csv files
# Define parameters for generating the data
# To save smartphone battery power, we typically collect location data intermittently: e.g. during an on-cycle of 3 minutes, followed by an off-cycle of 12 minutes. We'll generate data in this way
# number of persons to generate
n_persons = 1
# location of person to generate format: Country_2_letter_ISO_code/City_Name
location = "GB/Bristol"
# start date of generated trajectories
start_date = datetime.date(2021, 10, 1)
# end date of trajectories
end_date = datetime.date(2021, 10, 5)
# api key for openroute service, generated from https://openrouteservice.org/
api_key = "mock_api_key"
# Length of off-cycle + length of on-cycle in minutes
cycle = 15
# Length off-cycle / (length off-cycle + length on-cycle)
percentage = 0.8
# dictionary of personal attributes for each user, set to None if random, check Attributes class for usage in simulate_gps_data module.
personal_attributes = {
    "User 1":
    {
        "main_employment": "none", 
        "vehicle" : "car",
        "travelling_status": 10,
        "active_status": 7
    },

    "Users 2-4":
    {
        "main_employment": "university",
        "vehicle" : "bicycle",
        "travelling_status": 8,
        "active_status": 8,
        "active_status-16": 2 
    },

    "User 5":
    {
        "main_employment": "office",
        "vehicle" : "foot",
        "travelling_status": 9,
        "travelling_status-20": 1,
        "preferred_exits": ["cafe", "bar", "cinema"] 
    }
}
sample_gps_data = sim_gps_data(n_persons, location, start_date, end_date, cycle, percentage, api_key, personal_attributes)
# save data in format of csv files
gps_to_csv(sample_gps_data, path_to_synthetic_gps_data, start_date, end_date)

Let me know if more information is needed.

[GeorgeEfstathiadis]

So I'm still not able to replicate your error. For some reason it looks like the user generated does not have locations close enough to visit for amenities (which shouldn't be the case). To debug further I made a smaller script for you to run that simply generates a Person as defined in this function and we take a look at the possible exits assigned to them to see if they match the criteria needed.

Could you run the following script and share what it outputs?

from forest.bonsai.simulate_gps_data import generate_nodes, Person, generate_addresses, load_attributes

location_ctr, location_city = "GB/Bristol".split("/")
nodes = generate_addresses(location_ctr, location_city)
house_address = (float(nodes[0]['lat']), float(nodes[0]['lon']))
attributes_dict =  {
    "User 1": {
        "main_employment": "none",
        "vehicle": "car",
        "travelling_status": 10,
        "active_status": 7,
    },
    "Users 2-4": {
        "main_employment": "university",
        "vehicle": "bicycle",
        "travelling_status": 8,
        "active_status": 8,
        "active_status-16": 2,
    },
    "User 5": {
        "main_employment": "office",
        "vehicle": "foot",
        "travelling_status": 9,
        "travelling_status-20": 1,
        "preferred_exits": ["cafe", "bar", "cinema"],
    },
}

attributes_dictionary, switches_dictionary = load_attributes(
    attributes_dict
)
attrs = attributes_dictionary[1]
all_nodes = generate_nodes(house_address, attrs.main_occupation)

person = Person(house_address, attrs, all_nodes)

print(person.possible_destinations, len(person.possible_destinations))

[JiaxinCCC]

@GeorgeEfstathiadis Thanks for your help.

Below is my output of the above code.

[GeorgeEfstathiadis]

Sorry for the delay, your output is problematic. I was expecting to see at least 5 possible exits, but for some reason you only get one. This is what causes the error to basically state that it can't generate trajectories for a person that doesn't have that many locations nearby to visit according to OSM (need at least 3 possible exits to run).

I'm not sure why this occurs, since the same code segment always seem to work for me. It appears to have something to do with the OpenStreetMap API not collecting many addresses for some reason. Try using a fixed address, i.e. replace house_address in your code with house_address = (51.4565167, -2.5896389).

The expected output should be something like:

[<PossibleExits.CAFE: 'cafe'>, <PossibleExits.BAR: 'bar'>, <PossibleExits.RESTAURANT: 'restaurant'>, <PossibleExits.PARK: 'park'>, <PossibleExits.CINEMA: 'cinema'>, <PossibleExits.DANCE: 'dance'>, <PossibleExits.FITNESS: 'fitness_centre'>] 7

If that's not the case we would have to dive deeper into why the API is not working for you. If it works it might be the case an address was selected that was weird or some temporary issue of the API and I would suggest rerunning the simulate GPS coordinates code.

[JiaxinCCC]

Sorry for the delay, your output is problematic. I was expecting to see at least 5 possible exits, but for some reason you only get one. This is what causes the error to basically state that it can't generate trajectories for a person that doesn't have that many locations nearby to visit according to OSM (need at least 3 possible exits to run).

I'm not sure why this occurs, since the same code segment always seem to work for me. It appears to have something to do with the OpenStreetMap API not collecting many addresses for some reason. Try using a fixed address, i.e. replace house_address in your code with house_address = (51.4565167, -2.5896389).

The expected output should be something like:

[<PossibleExits.CAFE: 'cafe'>, <PossibleExits.BAR: 'bar'>, <PossibleExits.RESTAURANT: 'restaurant'>, <PossibleExits.PARK: 'park'>, <PossibleExits.CINEMA: 'cinema'>, <PossibleExits.DANCE: 'dance'>, <PossibleExits.FITNESS: 'fitness_centre'>] 7
If that's not the case we would have to dive deeper into why the API is not working for you. If it works it might be the case an address was selected that was weird or some temporary issue of the API and I would suggest rerunning the simulate GPS coordinates code.

Thank you for your reply. I ran the following code:

from forest.bonsai.simulate_gps_data import generate_nodes, Person, generate_addresses, load_attributes

location_ctr, location_city = "GB/Bristol".split("/")
nodes = generate_addresses(location_ctr, location_city)
house_address = (51.4565167, -2.5896389)
attributes_dict = {
    "User 1": {
        "main_employment": "none",
        "vehicle": "car",
        "travelling_status": 10,
        "active_status": 7,
    },
    "Users 2-4": {
        "main_employment": "university",
        "vehicle": "bicycle",
        "travelling_status": 8,
        "active_status": 8,
        "active_status-16": 2,
    },
    "User 5": {
        "main_employment": "office",
        "vehicle": "foot",
        "travelling_status": 9,
        "travelling_status-20": 1,
        "preferred_exits": ["cafe", "bar", "cinema"],
    },
}

attributes_dictionary, switches_dictionary = load_attributes(
    attributes_dict
)

attrs = attributes_dictionary[1]

all_nodes = generate_nodes(house_address, attrs.main_occupation)

person = Person(house_address, attrs, all_nodes)

print(person.possible_destinations, len(person.possible_destinations))

I got the following output:

[<PossibleExits.PARK: 'park'>] 1