Using OpenSky api data in dump1090 web interface

I am operating 3 feeders for Flightaware, OpenSky, ... at two different locations. I combined the data of these feeders in an instance of dump1090-fa with --net-only option and without any feeder on a dedicated raspi combining their ADS-B and MLAT data, receiving about 150 aircraft position at the same time. I like the history option, that traces of observed aircraft are plotted. The next idea was to combine this with data from the opensky API. I changed the location of the JSON data for the web interface and wrote a python script which merges dump1090 JSON data with data from OpenSky producing new aircraft.json and history files for the webinterface. Here is the screenshot of my first try (the web interface is accessible from my LAN only):

Nice! How much data did you use for this? 1 hour? 1 day? Have you ever compared the output of your dump1090 and your script fetching data from our api using the /own instead of /all URL? They should more or less look the same...

Also: are you planning to publish your code? I'd love to try it myself

This is the direct comparison (left: local data from three receivers located at two locations with 60km distance, right: local data combined with opensky data).

The OpenSky data improves my own coverage area by adding more low altitude aircraft, especially for the airport area in Frankfurt and Amsterdam.
The history files are duplicates of the aircraft.json file, which are generated every 30 seconds (first tried 10s). Up to 60 history files are stored(first tried 120). A new aircraft.json is generated every 10s. receiver.json is manipulated to accept this larger refresh interval without timeout warning.
I am filtering the OpenSky data by thresholds for latitude and longitude to be limited to Europe (an option for the API to get a certain radius of data would be helpful). Datasets without position, altitude and course are removed. Additionally, flights containing 'DLH' in the callsign are shown outside of this area.

I am still working on effciency of merging. Generating human readible json files took a few seconds more than those without any spaces or newlines. And I need to reconsider my approach: As I am feeding all my dump1090-fa ADS-B data without MLAT to OpenSky, merging the local MLAT only data with the OpenSky data should bring the same result with less or even without comparisons. The next idea would be to merge OpenSky and ADSBexchange data (containing MLAT) for a further coverage improvement.

I can publish a short howto after tidying up the code.

Requirements for Raspberry Pi running Raspbian:
dump1090-fa (either complete piaware installation or dump1090-fa operated with --net-only option and fed with data from elsewhere)
opensky python api

Enable userdir webhosting:
sudo lighty-enable-mod userdir
Copy contents of /usr/share/dump1090-fa/html/ to ~/public_html/
Create link from ~/public_html/data to /run/shm/dump1090-data

Put your OpenSky login data in json-conversion-test.py, run the file on your Pi and access the website on http://your-hostname-or-ip/~pi/

Currently, an area around Europe + aircraft with 'DLH' callsign are visible worldwide (where OpenSky coverage exists).
Feel free to improve this script.

As I am using the API with username+password, I assumed that I would be allowed to get fresh data every 5 seconds as mentioned on opensky-network.org/apidoc/rest.html . But if I try to decrease the delay to less than 10 seconds between requests, the API prints "WARNING:opensky_api:Blocking request due to rate limit".

Here is another example of coverage comparison (left: usage of OpenSky data for whole Europe area, right: similar modification as described above for using data from ADSB-Exchange in a 1000km circle around my position):

jluebbe wrote:

As I am using the API with username+password, I assumed that I would be allowed to get fresh data every 5 seconds as mentioned on opensky-network.org/apidoc/rest.html . But if I try to decrease the delay to less than 10 seconds between requests, the API prints "WARNING:opensky_api:Blocking request due to rate limit".

I've tested, but cannot reproduce your issue with the current version:

There was an issue conerning rate limits which has been fixed a couple of months ago . Maybe an update helps?

Ok, I found the problem comparing your script to my setup. I tried to shorten the sleep by the data processing time to obtain a 5 second interval in total. That seems to be too short. Now, my timestamp for counting 5 seconds is set after api.get_states is called and everything works.