topic Re: Download excel file from webpage in Talend Studio

Download excel file from webpage

_AnonymousUser — Tue, 04 Nov 2014 03:47:19 GMT

Hello,
I am trying to set up a job that will download an excel file from a webpage and store a copy of it locally. I have tried to use the tHTTPrequest component but it keeps giving me errors. I have the direct download url that I am trying in the request (if I enter it into the address bar the download popup is displayed). I tried to find examples that other people have tried, however I have had no luck.
Is this something HTTPrequest can do? Or maybe tFileFetch? If anyone has a solution, or can point me in the right direction of previous questions or examples that would be much appreciated.
Thanks in advance!

Re: Download excel file from webpage

Anonymous — Tue, 04 Nov 2014 06:32:51 GMT

Hi,
Could you please try to use https://help.talend.com/search/all?query=tFileFetch&content-lang=en to see if it is OK with you?
Best regards
Sabrina

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 03:30:32 GMT

Hi,
Could you please try to use to see if it is OK with you?
Best regards
Sabrina

Thank you for getting back so fast.
I have looked into that and tried a few attempts and I was still getting errors. Do you know of any examples that I could walk through to get familiar with tFetch?
Thanks

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 03:58:05 GMT

Hi,
What's the error you are facing? Any error info is printed on console?
Best regards
Sabrina

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 04:18:31 GMT

Hi,
What's the error you are facing? Any error info is printed on console?
Best regards
Sabrina

this is my error:
Exception in component tFileFetch_1
java.lang.IllegalArgumentException: host parameter is null
    at org.apache.commons.httpclient.HttpConnection.<init>(HttpConnection.java:205)
    at org.apache.commons.httpclient.HttpConnection.<init>(HttpConnection.java:154)
    at org.apache.commons.httpclient.SimpleHttpConnectionManager.getConnectionWithTimeout(SimpleHttpConnectionManager.java:149)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:152)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
    at mhtc_di.taxfoundation_cost_0_1.TaxFoundation_cost.tFileFetch_1Process(TaxFoundation_cost.java:331)
    at mhtc_di.taxfoundation_cost_0_1.TaxFoundation_cost.runJobInTOS(TaxFoundation_cost.java:641)
    at mhtc_di.taxfoundation_cost_0_1.TaxFoundation_cost.main(TaxFoundation_cost.java:500)

I have a URI that leads straight to the excel file to be downloaded. Destination "C:" and destination filename "output"

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 07:06:51 GMT

Hi,
Would you mind uploading your job setting screenshots into forum so that we can address your issue quickly.
Best regards
Sabrina

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 08:42:28 GMT

What is your URI in tFileFetch component?
Can you use that URL in internet browser and download the file?
Vaibhav

Re: Download excel file from webpage

Anonymous — Wed, 05 Nov 2014 22:06:37 GMT

Hi,
Would you mind uploading your job setting screenshots into forum so that we can address your issue quickly.
Best regards
Sabrina

Re: Download excel file from webpage

Anonymous — Wed, 12 Nov 2014 02:44:39 GMT

Hi,
Would you mind uploading your job setting screenshots into forum so that we can address your issue quickly.
Best regards
Sabrina

Is this what you are looking for?
The uri is the direct URL I wish to download the file from and I am just trying to put it on my computer for the time being before I try and parse the data contained within.

edit: screenshot did not load
TaxFoundation_cost_0.1.screenshot.screenshot
TaxFoundation_cost.zip.zip

I was able to download an excel file using HTTPrequest. However, when I attempt to open the file it says that the file is in an unsupported format or may be corrupted. Has anyone run into this issue before? If so, what was the solution?
Thanks,
Kevin

Re: Download excel file from webpage

Anonymous — Wed, 12 Nov 2014 03:48:44 GMT

What is your URI in tFileFetch component?
Can you use that URL in internet browser and download the file?
Vaibhav

Yeah its the same URL that brings me to the download popup from the website.

Re: Download excel file from webpage

Anonymous — Wed, 12 Nov 2014 05:10:40 GMT

If it is bringing a pop-up from the website to save the file, then I think it is difficult to get it using the tFileFetch component...
can you try using wget using command prompt? if it can download, then you can use tSystem component to download the file...
please check.
thanks
Vaibhav