Observation mismatch when downloading subthreshold triggers #37
Description
With the help of @dmopalmer and @sibasish1 ...
When running download_swift_trigger_data using a utc triggertime, the code searches for any observations with a start time within 300s (by default) of the triggertime parameter. However, this raises 2 possible issues: first, it could return the wrong observation with a closer start time and second, there are many observations that contain the same triggertime from many different observing runs over multiple orbits of the observatory.
Source code with issue from batlib.py:
1848: query = {'start_time': f"{tstart.isot}..{tend.isot}", 'fields': 'all'}
1849: nearest_obs_table = from_heasarc(**query)
1850: dt = Time(float(triggermjd), format="mjd") - Time(nearest_obs_table["START_TIME"], format="mjd")
1851: closest_obsid = nearest_obs_table["OBSID"][np.argmin(np.abs(dt))]
1852: if timewindow > np.abs(dt[np.argmin(np.abs(dt))].to("s")).value:
For an example of the first problem, there are 2 BAT observations on 2022-10-10 from 01:27:31 - 01:46:57 (Obsid 01126853004) and from 01:47:00 - 02:00:27 (Obsid 00014981017). A trigger occuring at 01:39:00 would not return data from either observation because it is more than 300s away from either start time. However, a trigger occuring at 01:43:00 would return the data from 00014981017 because the start time is within 300s of the trigger time, but we are not interested in this data, rather we want the data from 01126853004 because it encloses the trigger time.
For the second problem, we need to select the correct observation from the swiftmastr table similar to how swinfo returns a single Obs Sequence Number.
Here is a brief code snippet to demonstrate the first problem:
import batanalysis as ba
import datetime
datadir = Path("/tmp/sb_test")
t0 = datetime.datetime(2022, 10, 10, 1, 39)
ba.datadir(“/tmp/sb_test", mkdir=True)
downloads = ba.download_swift_trigger_data(triggertime=t0)
print(downloads)This code produces the following files:
Running swinfo for the times of interest produces this result:
Obsids 01126883000 and 01126884000 are between 01:27:31 - 01:46:57 UTC, but the downloaded observation 00014981017 within is from the following observation from 01:47:00 - 02:00:27 UTC. This is because 01126884000 occurs at 01:43, which is within 300s of 01:47, so it takes that observation data instead of the real data with start time 01:27 (more than 300s away). Increasing the default triggerwindow would not change this behavior because it would simply encompass more start times and yet still return the closest one. Some consideration of the observation that actually encloses the triggertime needs to be included.