- Posts: 16
- Thank you received: 0
time_position data downloaded from OpenSky appears to be corrupted
3 years 4 months ago #1534
by w3sz
I am seeing that in the downloaded json object for each aircraft the timestamp ("time_position") is corrupted, being offset from the correct value by ~ +60 seconds, thus placing the timestamps for recently acquired aircraft in the future, with the time stamp indicating that the aircraft generated the data at a time in the future that has not yet occurred.
I see this result both when viewing the data downloaded from OpenSky with my own software and when I use an online epoch converter to convert time_position value received in the json object from opensky-network.org/api/states to human-readable form.
My installation is NIST-synchronized, and I have done the test using other NIST-synchronized platforms independent of my installation, and all tests give the same result, that the OpenSky time_position values are corrupted, with an offset of ~+60 seconds. So the issue is not due to my installation having incorrect time synchronization.
So, for example, if I am querying the state of an aircraft whose report was actually generated at UTC 14:00:00, then the OpenSky state data reports time_position (index 3) values for that aircraft to be ~14:01:00 UTC.
I do not know when this offset into the future started, as I had not been actively reviewing the data here for some months due to involvement with other projects. I first noted this on June 22, 2021 as I reinitiated work on a project involving querying the live database.
You can easily prove that this error is present by doing the following:
1. Copy the most recent timestamp contained in the jquery resulting from:
opensky-network.org/api/states/all?lamin...7.8229&lomax=10.5226
2. Paste the timestamp into the textbox under the heading "Convert epoch to human-readable date and vice versa" at:
www.epochconverter.com/ (Obviously you need to have picked a "recent" timestamp from the query and not a stale one).
3. Click the "Timestamp to Human date" button next to the textbox into which you put the timestamp.
4. You will see in the result, "Relative: In a minute" indicating that the downloaded timestamp is approximately one minute into the future.
The incorrect time offset in the data downloaded from OpenSky is a corruption of uploaded aircraft data that occurs at the level of the OpenSky server; it is not present in the time data that observers are uploading to the OpenSky server.
Although I first noticed this problem on data that did not originate at my installation, I have since confirmed the fact that the data corruption is occurring at the level of the OpenSky server by using the plane data that I upload to OpenSky. The timestamps are correct on the plane data that I generate here but when that data returns to me from the OpenSky server, the time data returning from the OpenSky server has been corrupted, with the time offset into the future applied, so that the timestamps are no longer correct when they are received from the OpenSky server.
This is a major issue for those using the data in a manner that requires accurate timestamps, and it is an issue that should be addressed.
I see this result both when viewing the data downloaded from OpenSky with my own software and when I use an online epoch converter to convert time_position value received in the json object from opensky-network.org/api/states to human-readable form.
My installation is NIST-synchronized, and I have done the test using other NIST-synchronized platforms independent of my installation, and all tests give the same result, that the OpenSky time_position values are corrupted, with an offset of ~+60 seconds. So the issue is not due to my installation having incorrect time synchronization.
So, for example, if I am querying the state of an aircraft whose report was actually generated at UTC 14:00:00, then the OpenSky state data reports time_position (index 3) values for that aircraft to be ~14:01:00 UTC.
I do not know when this offset into the future started, as I had not been actively reviewing the data here for some months due to involvement with other projects. I first noted this on June 22, 2021 as I reinitiated work on a project involving querying the live database.
You can easily prove that this error is present by doing the following:
1. Copy the most recent timestamp contained in the jquery resulting from:
opensky-network.org/api/states/all?lamin...7.8229&lomax=10.5226
2. Paste the timestamp into the textbox under the heading "Convert epoch to human-readable date and vice versa" at:
www.epochconverter.com/ (Obviously you need to have picked a "recent" timestamp from the query and not a stale one).
3. Click the "Timestamp to Human date" button next to the textbox into which you put the timestamp.
4. You will see in the result, "Relative: In a minute" indicating that the downloaded timestamp is approximately one minute into the future.
The incorrect time offset in the data downloaded from OpenSky is a corruption of uploaded aircraft data that occurs at the level of the OpenSky server; it is not present in the time data that observers are uploading to the OpenSky server.
Although I first noticed this problem on data that did not originate at my installation, I have since confirmed the fact that the data corruption is occurring at the level of the OpenSky server by using the plane data that I upload to OpenSky. The timestamps are correct on the plane data that I generate here but when that data returns to me from the OpenSky server, the time data returning from the OpenSky server has been corrupted, with the time offset into the future applied, so that the timestamps are no longer correct when they are received from the OpenSky server.
This is a major issue for those using the data in a manner that requires accurate timestamps, and it is an issue that should be addressed.
