access Impala from Python?

Hi All. I would like to be able to look up an icao24 and get all StateVectors between two timestamps. The example on this page:

opensky-network.org/impala-guide

Shows how to do that using the Impala shell. I can run this query from the Impala shell and it works:

[hadoop-1:21000] > SELECT COUNT(*) FROM state_vectors_data4 WHERE icao24='a0d724' AND time>=1480760100 AND time<=1480764600 AND hour>=1480759200 AND hour<=1480762800;

Now I'd like to do the same thing from Python. I'm using impyla, which is here:

github.com/cloudera/impyla

And I'm trying this code, based on one of their examples:

from impala.dbapi import connect
conn = connect(host='data.opensky-network.org', port=2230,
user=user, password=password)
print(conn)
cur = conn.cursor()
print(cur)
cur.execute('SHOW TABLES')
res = cur.fetchall()

It gets to the first print statement, and prints:

<impala.hiveserver2.HiveServer2Connection object at 0x7f0b416da090>

So that's good so far. Then it hangs when running conn.cursor().

Does anyone have experience with this?

Alternatively, it looks like I can use the OpenSkyApi. I can run this example, and it works:

from opensky_api import OpenSkyApi
api = OpenSkyApi(username=user, password=password)
s = api.get_states(icao24='a65c95')
print(s)

But it doesn't look like that API allows me to get historical data from a time interval. Or does it?

Any help would be appreciated!

Allen

Hi Allen,

for security reasons Impala access is not supported through impyla or any other Impala client library for the moment. If you want to store results I recommend using the method described in Section 4 of the Impala Guide .

For the moment this is the only way to access the historical data set as a whole. The API supports retrieving historical data, but only for your own receivers. This is quite odd, I know, but we might open up the API for researchers in the future!

Hope that helps!

Markus

ONERA's Xavier Olive wrote a Python client for our Impala shell, just in case you are still interested:
github.com/xoolive/opensky_data