OpenSky Network Blog
Research (6)
You can now feed voice data to OpenSky -- our public feeding interface is finally live! Go to https://ui.atc.opensky-network.org/ to set up your receiver and also see your receiver's stats. In principle, you can use a similar setup to your traditional ADS-B receiver but you will generally need two different setups to feed both ADS-B and VHF at the same time.
Please note that this requires a separate user account for the moment, this will be remedied in the future when we will move the old accounts to the new system. Right now, you will also have to let us know through our contact form that you are planning to feed. This will also be consolidated soon.
On Air Traffic Control Voice Communication
Although in everyday life people prefer texting over calling, it is the opposite in aviation. Tactical information exchange between air traffic controllers (ATCOs) and pilots is still done over analog radio voice channels. For this purpose the VHF (Very High Frequency) band between 118MHz and 136MHz is used. The band is divided into 8.33kHz channels.
This site is interfacing Opensky Network's Air Traffic Communication (ATC Platform), which has been set up to collect voice recordings through a network of volunteers. The recordings are automatically transcribed and stored on our infrastructure. Of course there is also a handy API available!
Our mission has remained the same over the years - that is, to improve security, reliability, efficiency and sustainability of the air space usage by providing open access of real-world air traffic control data to the public and are available for use for our users. So if you have a specific use case where either recordings or their transcripts are useful, do send us an email This email address is being protected from spambots. You need JavaScript enabled to view it. and tell us about it.
OpenSky: Air Traffic Data for Scientists
Written by Martin StrohmeierOpenSky has been founded by scientists for scientists. The mission of our non-profit association since 2014 has been to support open global air traffic research by universities and other not-for-profit institutions. We can confidently state that this has been an increasingly successful endeavour. In this blog post, I want to briefly look back on recent developments, illustrating the growing influence of OpenSky and air traffic data in general on research of all sorts and our new efforts (helped by our impressive community) in supporting more of these attempts with even better and more tools and data. But everything is only possible thanks to our feeders and supporters around the globe! If you want to improve air traffic research around the world for the common good, do join us!
Most Popular Research Areas
This analysis is just a fraction of the data we have analysed in a recent resarch paper [1]. The figure illustrates the growing number of publications over the past few years based on OpenSky data and divides it also into the most popular research areas over time.
From the paper:
The gray bars illustrate the overall growth trend, which strongly accelerated by 2018 with 22 publications, 144% more than in the previous year. This trend continued in 2019 with a 118% increase. For this year (as of November 2020), 54 publications have been counted, ensuring a new record before the year has ended.
We can break down our analysis further by taking a look at the different scientific subfields in which OpenSky’s data have been used. We identified four large recurring themes: machine learning (ML, 16 papers overall), air traffic management (ATM, 50 papers), (cyber) security (36 papers),and Earth sciences (8 papers). Smaller areas, subsumed under ‘Others’, include research on noise, high-performance computing, sensor networks, and visualization, among others. From the trends ofthe subfields, we can see that security was the largest area of OpenSky data use in the early years up to and including 2018, reflecting the interests of the original community.
You can find every publication using OpenSky data on our Publications page.
Our Users: Social and Research Impact
Our target groups are researchers from two different spheres: academics and non-profit organsiation. The figure below provides an illustration of 100 academic research groups, which have received access to our Impala shell over the course of 2019.
We can see that these are groups from all over the world!
Besides academics, we support and help a lot of non-profit organisations , NGOs and journalists, which have been using our flight data for everything from internal research to publicly released analyses. Example use cases range from gauging Covid-19-related impact to investigative journalism. The full updated list of these can be found on our Social Impact page but they include for example Bellingcat, the United Nations or the Bank of England!
New Data Tools and Datasets
As also discussed in the paper [1], our existing access methods -- the live API and the Impala database -- have served us and many members of the scientific community very well. However, we have also realised that we can do better and provide both tools and datasets tailored more to specific settings and use cases to make it easier for non-experts to read and use our data.
To this end, we have created two new subpages, which collect and curate a) data tools provided by our community and b) datasets based on OpenSky data.
Data Tools
Our data tools page currently introduces five different, community-maintained tools, which can be used to access, process and visualize data from our Impala shell in different programming languages. These tools are already helping many researchers with easier access to our massive amounts of data stored. Go check them out!
Datasets
Likewise, another avenue we have spent more time on in 2020, was to prepare different types of datasets based on OpenSky's whole data. These datasets fulfill different purposes, including better accessibility also without a working knowledge in programming/shell scripting and improved scientific comparability. Other datasets are the artifacts of scientific papers and have been released by their authors. We have collected all these efforts on our scientific dataset page, which currently lists eight different datasets!
Resources
[1] Martin Strohmeier. Research Usage and Social Impact of Crowdsourced Air Traffic Data.
In Proceedings of the 8th OpenSky Symposium 2020. November 2020.
[2] OpenSky Data Tools: https://opensky-network.org/data/data-tools
[3] OpenSky Scientific Datasets: https://opensky-network.org/data/datasets
[4] OpenSky Social Impact: https://opensky-network.org/about/social-impact
Over the past year we have been inundated with requests regarding flight data related to the COVID-19 pandemic. While we have done our best to help researchers around the world using our trusted Impala shell, we have also released a public version of the flight meta data that we have collected over the whole year of 2020 plus the whole year of 2019 for a pre-COVID-19 comparison. If you are only interested in the data, you can find it over at CERN's Zenodo repository: https://doi.org/10.5281/zenodo.3737101
We currently plan to update this dataset monthly during the pandemic. If you have research needs that go beyond this release model, you can apply for full data access with us.
Edit May 2023: We stopped updating the dataset after December 2022. Previous files have been fixed after a thorough sanity check. This is now the final and stable version of all flight metadata from 2019-2022.
The main use cases of flight data related to the pandemic are manifold: First, flight data can be used as input for models analysing and predicting the global spread of the virus. Second, flights as an indicator of economic activity can provide insights into the impact of the pandemic on both countries' economies in general and the aviation industry in particular. Finally, the data has become popular for analysis in Earth Systems Sciences over the course of the year. We have a pre-print discussing this here. [1] The data in this dataset is derived and cleaned from the full OpenSky dataset and made fully publicly available for the first time. It spans metadata for all flights seen by the network's more than 3500 members in 2019 and 2020.
The most important point to remember is that as these data are derived from our awesome feeders, we cannot provide every global flight movement in our dataset but only those ADS-B-equipped aircraft seen within our coverage!
An overview of our coverage is provided for any given day on our Facts page. An example for yesterday (2020-04-02) is shown here:
If you have access to a place that is not yet covered by OpenSky and want to see researchers include that area in the future, please do provide a feed!
[1] Strohmeier, M., Olive, X., Lübbe, J., Schäfer, M., and Lenders, V.: Crowdsourced Air Traffic Data from the OpenSky Network 2019–20, Earth Syst. Sci. Data. [preprint], 2021.
Usage Examples & Tools
Xavier Olive from ONERA has made some initial plots using this dataset, illustrating for example the drop in air traffic at specific airports during the early phase of the pandemic:
More up-to-date visualisations and code for use with this dataset can be found over at Impact of COVID-19 on worldwide aviation.
Since the dataset is naturally large, we recommend using tools such as R, Python or Matlab for processing vast quantities of data. You can find more tools, that interface directly with our Live API and our Impala shell on our data tools subpage.
If you want to split these files into smaller .csv files before loading them into your preferred processing programme, we recommend using CSV Splitter (Windows) or the Linux/MacOS split command.
We are very excited to see you next week!
The full information about the 7th OpenSky Workshop in Zurich, Switzerland (November 2019) can be found on the new dedicated website https://workshop.opensky-network.org
Is there any working intrusion detection system for ADS-B?
Written by Martin StrohmeierTl;dr: Not that we know of. At least not deployed anywhere outside a lab. Some proposals have been floated, however.
This post was inspired by the recent report by the US Congress Government Accountability Office on ADS-B, which was discussed in many articles around the web, such as this one, for example: GAO: Pentagon, FAA Lag In Addressing ADS-B Risks
Let’s have a look at the source, these are quotes from the GOA’s actual document, citing some of the work from contributors here at OpenSky: [1]
Yes, they do, at least partially, but there are massive differences between countries.
Let’s add some data to this popular question, shall we? This is from our recent paper at the 36th Digital Avionics Systems Conference [1]. We collected ADS-B and Mode A/C/S data from over 6000 aircraft operated by militaries all over the world (with a strong focus on Europe/the US) using the The OpenSky Network. The key plot to answer the question is the following:
