Inquiry on Time Delay - Impala

barathwajas
Topic Author
Offline
New Member

2 years 4 months ago #2011 by barathwajas

Inquiry on Time Delay - Impala was created by barathwajas

Hi,
There are couple of questions regd. the Impala and API:

1) The Tables on the Impala Shell provided from your end are those lively getting refreshed from the open sensors or there is a delay in pre-processing on your end once received the raw data from the sensor.
2) If the Live API is indirectly using Impala tables then those are also having any delay or it's live and fetches from different tables?

strohmeier
Offline
Administrator

2 years 4 months ago #2012 by strohmeier

Replied by strohmeier on topic Inquiry on Time Delay - Impala

Fig. 3 shows the (still current) parallel architecture: lenders.ch/publications/conferences/DASC15.pdf

In short there's a delay for the Impala shell, stuff needs to be written. Should be less than an hour. If, for example, flight origins are requested via the API, they are fetched from the Impala tables you can access.

barathwajas
Topic Author
Offline
New Member

2 years 4 months ago #2013 by barathwajas

Replied by barathwajas on topic Inquiry on Time Delay - Impala

Hello strohmeier ,
Thank you for the information that you have provided. I do have few more questions:

1) The state_vectors in the impala and states data in this URL - opensky-network.org/datasets/states/ both has only monday's data? (Every week dataset)?
2) In the architecture (Fig. 3) which shows Serving Layer need bit more clarification on this part, why there is a need for merging Live & Batch Datasets?

Thanks !!

strohmeier
Offline
Administrator

2 years 4 months ago #2014 by strohmeier

Replied by strohmeier on topic Inquiry on Time Delay - Impala

That link is only Mondays. The Impala shell is of course everything.

I don't think the details are relevant on the architecture and design decisions - it is histrocial and is what it always has been.