Missing historic data - 07/Sept/2022 onwards.

2 months 3 weeks ago #2090 by Herson
The Impala state_vectors_data4 table appears to have a problem  (Disk I/O error: Failed to open HDFS file hdfs) from the 7th Sept. 2022 onwards.

Proof:
 
select count(*), FROM_unixtime(hour, 'yyyy/MM/dd') as rth_date, 'WINCHESTER-WEST' as area_of_record
FROM state_vectors_data4 where lat between 51.06152 and 51.09048
and lon between -1.366071 and -1.320889
and hour between (1662447718 - (3600*24*10)) and (1662447718 + (3600*24*2))
group by rth_date order by rth_date desc;



Result:  Disk I/O error: Failed to open HDFS file hdfs://nameservice1/user/opensky/tables_v4/state_vectors/hour=1657105200/part-r-00169-858ba597-246d-4210-aa4d-91c6b44665d1.snappy.parquet
Error(2): No such file or directory
Root cause: RemoteException: File does not exist: /user/opensky/tables_v4/state_vectors/hour=1657105200/part-r-00169-858ba597-246d-4210-aa4d-91c6b44665d1.snappy.parquet

Thanks all.

Richard Herson - Founder of www.AircraftTrafficSurvey.com, a Not-For-Profit organisation serving our UK communities.
 

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago #2098 by strohmeier
Should work now, I hope.

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago #2100 by Herson
Confirmed - table now up to date.

Thanks.

Richard Herson - Founder of www.AircraftTrafficSurvey.com, a Not-For-Profit organisation serving our UK communities.

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
This website uses cookies to offer you the best experience of our services. By using this website you agree to our privacy policy!