Disk I/O error: Error reading from HDFS file

5 months 3 weeks ago #2601 by nmatton
Hello,

I frequently encounter a Disk I/O error, which appears to be associated with hfs files. Here's a sample of the output I receive. While I suspect it might be related to certain SQL queries (for instance, I seem to have issues using the "in" operator), I'm not entirely certain. Are there any specific limitations on the queries we can use to retrieve data?

Thank you for your assistance.

Best regards,
Nicolas

select * from state_vectors_data4 where callsign in ('BEL72R','BAW393B','BEL5KV') and time>=1685599200 and time<=1688195100 and hour>=1685599200 and hour<=1688194800 and lat>=50.6041 and lat<=51.1106 and lon>=3.9189 and lon<=4.9705 LIMIT 10;
WARNINGS: Disk I/O error: Error reading from HDFS file: hdfs://nameservice1/user/opensky/tables_v4/state_vectors/hour=1686146400/
Error(255): Unknown error 255
Root cause: BlockMissingException: Could not obtain block: BP-2086186090-192.168.6.170-1416410368441:blk_1716646581_875390380 file=/user/opensky/tables_v4/state_vectors/hour=1686146400/

 

Please Log in or Create an account to join the conversation.

4 months 3 weeks ago #2634 by lsperk
Hi, I am having a similar issue. Did you get it resolved?
ERROR: Disk I/O error: Failed to open HDFS file hdfs://nameservice1/user/opensky/tables_v4/state_vectors/hour=1694469600/
Error(2): No such file or directory
Root cause: RemoteException: File does not exist: /user/opensky/tables_v4/state_vectors/hour=1694469600/
at (:66)
at (:56)
at (:2157)
at (:2127)
at (:2040)
at (:583)
at (:94)
at (:377)
at $ClientNamenodeProtocol$()
at $Server$(:617)
at $(:1073)
at $Handler$(:2281)
at $Handler$(:2277)
at (Native Method)
at (:422)
at (:1920)
at $(:2275)

Here is the error I get:

Please Log in or Create an account to join the conversation.

1 month 2 days ago #2702 by strohmeier
One reason we've been working so hard on migrating things! This wasn't sustainable anymore, hence everything and everyone is being moved to Trino now. We'll send a message to all remaining Impala users soon but you can already switch. Read the updated opensky-network.org/data/impala, register and send us a message.

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
This website uses cookies to offer you the best experience of our services. By using this website you agree to our privacy policy!