Missing historic data - 07/Sept/2022 onwards.

The Impala state_vectors_data4 table appears to have a problem  (Disk I/O error: Failed to open HDFS file hdfs) from the 7th Sept. 2022 onwards.

Proof:
 
select count(*), FROM_unixtime(hour, 'yyyy/MM/dd') as rth_date, 'WINCHESTER-WEST' as area_of_record
FROM state_vectors_data4 where lat between 51.06152 and 51.09048
and lon between -1.366071 and -1.320889
and hour between (1662447718 - (3600*24*10)) and (1662447718 + (3600*24*2))
group by rth_date order by rth_date desc;



Result:  Disk I/O error: Failed to open HDFS file hdfs://nameservice1/user/opensky/tables_v4/state_vectors/hour=1657105200/part-r-00169-858ba597-246d-4210-aa4d-91c6b44665d1.snappy.parquet
Error(2): No such file or directory
Root cause: RemoteException: File does not exist: /user/opensky/tables_v4/state_vectors/hour=1657105200/part-r-00169-858ba597-246d-4210-aa4d-91c6b44665d1.snappy.parquet

Thanks all.

Richard Herson - Founder of www.AircraftTrafficSurvey.com, a Not-For-Profit organisation serving our UK communities.
 

Confirmed - table now up to date.

Thanks.

Richard Herson - Founder of www.AircraftTrafficSurvey.com, a Not-For-Profit organisation serving our UK communities.