topic Re: Download exel file using http URL in Talend Studio

Download exel file using http URL

Anonymous — Wed, 26 Oct 2016 09:06:55 GMT

I need to download excel file from a http URL into my local system. Please help me out with possible ways to do so.

Re: Download exel file using http URL

Anonymous — Wed, 26 Oct 2016 09:45:59 GMT

Hi,
Here is a component TalendHelpCenter:tFileFetch which is used to retrieve a file via a defined protocol.
Best regards
Sabrina

Re: Download exel file using http URL

Anonymous — Wed, 26 Oct 2016 11:53:56 GMT

I'm getting below mentioned error while running the job -

Exception in component tFileFetch_1

java.lang.Exception: Method failed: HTTP/1.1 401 Unauthorized

at nikhil.download_file_0_1.Download_file.tFileFetch_1Process(Download_file.java:488)

at nikhil.download_file_0_1.Download_file.runJobInTOS(Download_file.java:812)

at nikhil.download_file_0_1.Download_file.main(Download_file.java:669)

Re: Download exel file using http URL

cterenzi — Wed, 26 Oct 2016 20:15:52 GMT

Does the web server from which you are downloading require a login? If so, you'll need to authenticate first before fetching the file. This can be complicated to set up.

In the one such job I've managed to get working, I had to point a tFileFetch at the website's login URL, select POST method, supply user_id and password as Parameters, save the resulting cookie, and specify the full path and filename for the cookie file (not just the directory, as the field name implies). Then I used additional tFileFetch components to retrieve files, making sure to check Read Cookie, and specifying the cookie file generated by the first component.

Working out the exact sequence required the use of Fiddler to read the underlying http calls for the POST parameters. ( http://www.telerik.com/fiddler)

Re: Download exel file using http URL

Anonymous — Wed, 26 Oct 2016 21:14:45 GMT

Hi,
After following the method inscribed in the web link, my CSV comes out in HTML format. I don't know the reason being. Please provide assistance.
Thanks,

Re: Download exel file using http URL

cterenzi — Thu, 27 Oct 2016 19:59:42 GMT

Is it the expected data with HTML tags around it, or is it a web page containing error messages?

If you're getting the expected data, it sounds like the web server is formatting it for screen viewing.

Re: Download exel file using http URL

Anonymous — Thu, 27 Oct 2016 20:29:20 GMT

This is what I get in the CSV file.

Re: Download exel file using http URL

Anonymous — Tue, 01 Nov 2016 10:25:39 GMT

I tried tfilefetch component for a file which doesn't needs authentication, but still its giving error while executing. Please find below the error I'm getting.

Exception in component tFileFetch_1
java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at nikhil.download_file_0_1.Download_file$1SocketFactory_tFileFetch_1.createSocket(Download_file.java:396)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
[statistics] disconnected
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at nikhil.download_file_0_1.Download_file.tFileFetch_1Process(Download_file.java:452)
at nikhil.download_file_0_1.Download_file.runJobInTOS(Download_file.java:812)
at nikhil.download_file_0_1.Download_file.main(Download_file.java:669)

I tried increasing timeout limit in advanced settings but error remains the same.

Please suggest something or a working example would be a great help.