OpenSky - Behind the ScenesOpenSky · Tech
The OpenSky Network is a community-based receiver network which continuously collects air traffic surveillance data. Unlike other networks it keeps collected data forever and serves as a data provider for researchers all around the globe.
OpenSky’s data can be harnessed for very different purposes ranging from desaster analysis to meteological measurements. Because it is impossible to be aware of all use cases it is important to store all raw data in the hope that it will be useful at some point. Indeed, we have seen several times that this approach is worth the effort as new research questions are popping up quite frequently. Because of their diversity different aspects of the data need to be considered.
I have had the opportunity to help building the OpenSky Network (nearly) from scratch. It started in 2012 as a bunch of Python scripts and evolved into a complex distributed system. After several years of exprience as its main developer I would like to share some knowledge and give an introduction into OpenSky’s technology stack and the problems we faced within the last four years. I hope you enjoy reading and if you would like to share your thoughts, I am happy to receive your comments.
Coming Up Next
- General overview of the tech stack and data flow
- Part 2 - Batch Processing and Analysis
- Part 3 - Stream Processing
- Challenges of passive Mode S sniffing
- Building the OpenSky Alerting Service
- Drawing interrogator maps from sample data
- History of the OpenSky Network