Skip to content 
Have a pleasant journey with FlightStats and ArangoDB
The Challenge
Aviation data is fragmented and not uniformed to one standard. Flight status, weather, reference data, FAA Airport Delays and many data sources more affect the whole customer journey from destination A (e.g. home) to your final destination B (e.g. hotel). Data from all these sources have to be harmonized or transformed to get accurate analytical or predictive results.
We maintain and use reference data for e.g. airlines, airports or equipment. This data has been maintained across many tables in a relational database. We were running into some data related challenges which are summed up into the following:
- Store reference data with temporal information
- Make it easier to change schema as future needs arise
- Improve our authoring web UI
- Reduce load on our relational db
- Provide API access to the reference data
We needed a document db that allowed us to easily query the data by effective date. Storing and processing data from more than 30k airports and airlines is a challenge itself but for our needs, it is necessary to store each modification of those entities with its own effective date as a document as well (i.e. key = id + effective_date). All changes and their effective date have a significant impact on historical reports and thus on our predictive analytics.
For our partners and data sources we needed a solution which enables easy access to multiple environments and a versatile, performant API layer. Due to the growth of our company in terms of API requests, external data sources, and quantity of data, we faced the need for a scalable solution.
Furthermore, our team was and is growing fast. Thus we need a good documentation of the technology we use at FlightStats to get new team members up to speed – especially on advanced topics.
