Export data to a file after logging into the Impala shell

2 months 3 weeks ago #2099 by epitomeri
Hello,

After logging into the impala shell and while at the hadoop prompt [hadoop-2:21000] >, we are able to execute queries and see data on the terminal. Does anyone know how to pipe the output of the query to a file? Consider the following command that results in the display of the tables at the terminal. How do I modify this command so as to pipe the output to a text file? TIA.[hadoop-2:21000] > show tables;Satish

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago #2101 by strohmeier
Hi,

Please read the guide that comes with data access: opensky-network.org/data/impala

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago - 2 months 3 weeks ago #2102 by epitomeri
I already read the guide. What I see is only the "script" command that lets you record all the terminal activities. That is understood. All I am wondering about is, if we login to the shell through putty without using the script command and then execute a query from the impala shell (i.e., the hadoop prompt you see), is there a way to redirect that output to a file on a per command basis. This is my first time looking at impala and to some extent sql as well and hence the question. There must be some redirection operator like your > symbol that we use to redirect output from a linux terminal, I guess?

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago #2103 by strohmeier
From inside the shell (on a separate server), you won't be able to do this. The piping happens locally. At least, I am not aware of any method, and this has been up for many, many years.

The alternative is the use of a tool like traffic.

Please Log in or Create an account to join the conversation.

2 months 3 weeks ago #2104 by epitomeri
Thank you. We managed to use traffic and export some data to .csv. However, we are wondering if it is possible to reduce the sampling rate? It seems as if he historical data is being sampled at 1 second intervals. Is there a way to get it at some rate, say, 30 sec intervals?

Regards,
Satish

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
This website uses cookies to offer you the best experience of our services. By using this website you agree to our privacy policy!