Inquiry on Time Delay - Impala

6 months 3 weeks ago #2011 by barathwajas
Hi,
There are couple of questions regd. the Impala and API:

1) The Tables on the Impala Shell provided from your end are those lively getting refreshed from the open sensors or there is a delay in pre-processing on your end once received the raw data from the sensor.
2) If the Live API is indirectly using Impala tables then those are also having any delay or it's live and fetches from different tables?

Please Log in or Create an account to join the conversation.

6 months 3 weeks ago #2012 by strohmeier
Fig. 3 shows the (still current) parallel architecture: lenders.ch/publications/conferences/DASC15.pdf

In short there's a delay for the Impala shell, stuff needs to be written. Should be less than an hour. If, for example, flight origins are requested via the API, they are fetched from the Impala tables you can access.

Please Log in or Create an account to join the conversation.

6 months 3 weeks ago #2013 by barathwajas
Hello strohmeier ,
Thank you for the information that you have provided. I do have few more questions:

1) The state_vectors in the impala and states data in this URL -  opensky-network.org/datasets/states/ both has only monday's data? (Every week dataset)?
2) In the architecture (Fig. 3) which shows Serving Layer need bit more clarification on this part, why there is a need for merging Live & Batch Datasets?

Thanks !!

Please Log in or Create an account to join the conversation.

6 months 3 weeks ago #2014 by strohmeier
That link is only Mondays. The Impala shell is of course everything.

I don't think the details are relevant on the architecture and design decisions - it is histrocial and is what it always has been. :)

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
This website uses cookies to offer you the best experience of our services. By using this website you agree to our privacy policy!