×

Notice

The forum is in read only mode.

Trino.history() - PYTHON CLASS

7 months 2 weeks ago - 7 months 2 weeks ago #2719 by Herson
FYI: Obtained the following issue:

DOWNLOAD: 0.00lines [00:00, ?lines/s]Exception "unhandled sqlalchemy.exc.OperationalError"
(trino.exceptions.TrinoExternalError) TrinoExternalError(type=EXTERNAL, name=HIVE_UNKNOWN_ERROR, message="Unable to execute HTTP request: Timeout waiting for connection from pool", query_id=20240411_172730_24771_zjmbu)
(Background on this error at: sqlalche.me/e/20/e3q8)
File: C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\trino\client.py, Line: 565

 
Python Function called at submitted at 6:27pm BST.  (11-April-24)

Trino.history().

SELECT state_vectors_data4.time, state_vectors_data4.icao24, state_vectors_data4.lat, state_vectors_data4.lon, state_vectors_data4.velocity, state_vectors_data4.heading, state_vectors_data4.vertrate, state_vectors_data4.callsign, state_vectors_data4.onground, state_vectors_data4.alert, state_vectors_data4.spi, state_vectors_data4.squawk, state_vectors_data4.baroaltitude, state_vectors_data4.geoaltitude, state_vectors_data4.lastposupdate, state_vectors_data4.lastcontact, state_vectors_data4.serials, state_vectors_data4.hour \nFROM state_vectors_data4 \nWHERE state_vectors_data4.lon >= ? AND state_vectors_data4.lon <= ? AND state_vectors_data4.lat >= ? AND state_vectors_data4.lat <= ? AND state_vectors_data4.time >= ? AND state_vectors_data4.time <= ? AND state_vectors_data4.hour >= ? AND state_vectors_data4.hour < ?",

Function Parameter values:

USING DOUBLE '0.201989',DOUBLE '0.247171',DOUBLE '50.95292',DOUBLE '50.98188',1710201600,1712880000,1710201600,1712880000

Richard.
7 months 2 weeks ago #2720 by Herson
If I reduce the number of days (stop-start parameters) to 10 days the trino.history() function worked i.e. it returned Pandas dataset of 33k rows.   This implies the issue experienced relates to the allocation of server resources.

I could re-write my code to submit multiple calls to trino.history(), 5 days at a time * 6 iterations which would then provide me with the 30 days info I require.   Would this be more efficient than increasing the server resources to support trino.history() that returns 200k rows in a single iteration? 

Thanks Richard
Powered by Kunena Forum