bug: datetime truncation causes OCO3_L2_Lite_SIF temporal subset failure

[ocsmit]

Steps to reproduce

Granule:

• https://oco2.gesdisc.eosdis.nasa.gov/test/data//OCO3_DATA/OCO3_L2_Lite_SIF.11r/2021/oco3_LtSIF_210808_B11072Ar_240908045505s.nc4

Params:

poetry run l2ss-py --bbox "-180" "-90" "180" "90" --min-time "2021-08-08T06:00:00" --max-time "2021-08-08T18:00:00" "oco3_LtSIF_210808_B11072Ar_240908045505s.nc4" "test_out.nc4"

Version:

• 3.1.0rc4 (this was the version of the latest autotester run when I began work https://github.com/podaac/l2ss-py-autotest/runs/49921842196)

Relevant Autotester PR: podaac/l2ss-py-autotest#3616, but work based off this specific run https://github.com/podaac/l2ss-py-autotest/runs/49921842196

Explanation

This collection has different temporal dimension variables being associated with the root group and its children. The root group's temporal dim is associated with the /Delta_Time variable and all of the child nodes are associated with the /Geolocation/time_tai93.
These two variables differ with the former representing seconds elapsed since 1990-01-01 and the later seconds elapsed since 1993-01-01.

When opening the granule with xarray, the output datatree is able to parse /Geolocation/time_tai93
but not /Delta_Time. However, when manually parsing the /Delta_Time variable, the datetimes are truncated up to the seconds, whereas /Geolocation/time_tai93 does not truncate. I.e. 2021-08-08T23:53:07.3906 becomes 2021-08-08T23:53:07.0000.

This effects the upper bounds of the temporal subset as timestamps that fall between 2021-08-08T18:00:00 and 2021-08-08T18:00:01 are included with the subset when subsetting with /Delta_Time (truncated), but not included when subsetting with the non truncated /Geolocation/time_tai93 dimension. I.e. the root group has ndim = 188776, and child groups have ndim = 188754 resulting in the misalignment error when trying to construct the final dataset.

The trunctaion occurs when multiplying the float64 seconds since array by a np timedelta scalar

l2ss-py/podaac/subsetter/subset.py

Line 634 in c3d02a6

delta = np.array(seconds_since) * np.timedelta64(1, 's')

Replacing the following

l2ss-py/podaac/subsetter/subset.py

Line 635 in c3d02a6

result = epoch_dt64 + delta

with

result = np.array([epoch_dt64 + datetime.timedelta(seconds=i) for i in np.array(seconds_since)], dtype=np.datetime64)

makes sure that the timestamp values are the same as if xarray were to parse it (non truncated).

I tested with this change made to the source of 3.1.0rc4, and it was able to generate a correct subset.

[jamesfwood]

@ocsmit Would you be willing to submit a PR for this fix? You can fork this repo and make the fix and submit a PR. Thanks!