topic Re: reading a txt file from a SSH server in Talend Studio

reading a txt file from a SSH server

Anonymous — Sat, 16 Nov 2024 13:50:06 GMT

Hi all. I have a problem.
In my scenario I must read a set of files (txt files) from a SSH server.
I'm able to connect to the server, but when I try to get files it gives me error.
Connecting to server I use tSCPConnection component. Then I use tSCPFileList component to specify path and type of file that must be read (txt files). Then I use tSCPGet component to "read" files, but it gives me this error:
java.io.IOException: Error during SCP transfer.
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:651)
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:612)
at isti.lettura_file_su_server_0_1.lettura_file_su_server.tSCPGet_1Process(lettura_file_su_server.java:540)
at isti.lettura_file_su_server_0_1.lettura_file_su_server.tSCPFileList_1Process(lettura_file_su_server.java:462)
at isti.lettura_file_su_server_0_1.lettura_file_su_server.tSCPConnection_1Process(lettura_file_su_server.java:270)
at isti.lettura_file_su_server_0_1.lettura_file_su_server.runJobInTOS(lettura_file_su_server.java:766)
at isti.lettura_file_su_server_0_1.lettura_file_su_server.main(lettura_file_su_server.java:682)
Caused by: java.io.IOException: Remote SCP error: scp: /disco_dcna1/isti/qsdatab/qsa/mydeck: No such file or directory
at ch.ethz.ssh2.SCPClient.receiveFiles(SCPClient.java:252)
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:647)
... 6 more
Where am I wrong? I must use another component?
My objective is to read files (*.txt) from server that must be used in elaborations.
Can anyone help me? Thanks.
I'm on talend open studio 3.1.0
Fabrizio

Re: reading a txt file from a SSH server

Anonymous — Wed, 12 Aug 2009 07:38:47 GMT

Anyone can help me, please??
Fabrizio

Re: reading a txt file from a SSH server

Anonymous — Wed, 12 Aug 2009 08:02:14 GMT

Hello Fabrizio
First, can you connect to SSH server successfully? Keep tSCPConnection and desactivate other componnt to check.
Best regards
shong

Re: reading a txt file from a SSH server

Anonymous — Wed, 12 Aug 2009 08:05:55 GMT

Yes I can connect successfully to SSH server, because it shows me a message box that I put when connection is ok.
Then after this I put a tSCPGet component in order to read file and i move my message box after this component as you can see in my figure.
Thanks. Fabrizio

Re: reading a txt file from a SSH server

Anonymous — Thu, 13 Aug 2009 08:01:36 GMT

Hello

Caused by: java.io.IOException: Remote SCP error: scp: /disco_dcna1/isti/qsdatab/qsa/mydeck: No such file or directory

I have tested and found that we should type in the full name of file at the moment, eg:
/root/shong/test.csv
It doesn't support wildcard character, eg:
/root/shong/*.csv
I think it should be added as a new feature. Would you like to report it on our bugtracker?
For now, you try to type in the full name of file or you can try the ftp component with SFTP support feature.
Best regards
shong

Re: reading a txt file from a SSH server

Anonymous — Sat, 15 Aug 2009 14:05:37 GMT

Hello

Caused by: java.io.IOException: Remote SCP error: scp: /disco_dcna1/isti/qsdatab/qsa/mydeck: No such file or directory

I have tested and found that we should type in the full name of file at the moment, eg:
/root/shong/test.csv
It doesn't support wildcard character, eg:
/root/shong/*.csv
I think it should be added as a new feature. Would you like to report it on our bugtracker?
For now, you try to type in the full name of file or you can try the ftp component with SFTP support feature.
Best regards
shong

But what I need is to read all txt files in a particular directory on the server.
I have done the same local on a machine with xls files and now i must do the same on the server.
I must read all txt files in the directory, because they are input for my procedures.
How can I achieve this???
Thanks,
Fabrizio

Re: reading a txt file from a SSH server

Anonymous — Sun, 16 Aug 2009 10:35:34 GMT

You need to correct your design (you must use an ITERATE link out of the tSCPFileList not "ComponentOk").
Connection---onOk--->FileList-----iterate----->FileGet-----(then attach any component you want to process the downloaded file).
In filelist you must specify a command to list the file you want, one per line (ie. "ls -1 /home/my/dir/*.txt" )
Then in fileget, you must specify a dynamic parameter for the file to be downloaded that represent the current iteration of FileList... press ctrl+space in the file list of FileGet to retrieve the published variable by FileList.
buona fortuna.