There are couple of questions regd. the Impala and API:
1) The Tables on the Impala Shell provided from your end are those lively getting refreshed from the open sensors or there is a delay in pre-processing on your end once received the raw data from the sensor.
2) If the Live API is indirectly using Impala tables then those are also having any delay or it's live and fetches from different tables?
In short there's a delay for the Impala shell, stuff needs to be written. Should be less than an hour. If, for example, flight origins are requested via the API, they are fetched from the Impala tables you can access.
Thank you for the information that you have provided. I do have few more questions:
1) The state_vectors in the impala and states data in this URL - opensky-network.org/datasets/states/ both has only monday's data? (Every week dataset)?
2) In the architecture (Fig. 3) which shows Serving Layer need bit more clarification on this part, why there is a need for merging Live & Batch Datasets?