[resolved] Retrieve Xml file from an Url

[Anonymous]

Hello,
I would like to save an xml file retrive from an Url.
In order to do that, I'm using a tFileFetch component but I have the following error :
Exception in component tFileFetch_1
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(Unknown Source)
at portefeuille_eg.test_0_1.test$1SocketFactory_tFileFetch_1.createSocket(test.java:382)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
I've checked in the advance setting "use authentication".
And if I call the url in internet explorer, a windows authentification windows open and after writing my ID, the xml appears well in internet explorer.
Someone have an idea please ?
Regards

[Anonymous]

Hi,
Did you check out "Die on error" option on tFileList Component to see if there is any error. Is there any response from server in console?
Best regards
Sabrina

[Anonymous]

Yes, these option is check and it return the same error.
connecting to socket on port 3547
30 août 2013 09:36:50 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: basic authentication scheme selected
connected
30 août 2013 09:36:50 org.apache.commons.httpclient.HttpMethodBase readResponseBody
INFO: Response content length is not known
30 août 2013 09:36:51 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: basic authentication scheme selected
30 août 2013 09:36:51 org.apache.commons.httpclient.HttpMethodDirector processAuthenticationResponse
GRAVE: Invalid challenge: Basic
org.apache.commons.httpclient.auth.MalformedChallengeException: Invalid challenge: Basic
at org.apache.commons.httpclient.auth.AuthChallengeParser.extractParams(AuthChallengeParser.java:97)
at org.apache.commons.httpclient.auth.RFC2617Scheme.processChallenge(RFC2617Scheme.java:93)
at org.apache.commons.httpclient.auth.BasicScheme.processChallenge(BasicScheme.java:111)
at org.apache.commons.httpclient.auth.AuthChallengeProcessor.processChallenge(AuthChallengeProcessor.java:161)
at org.apache.commons.httpclient.HttpMethodDirector.processWWWAuthChallenge(HttpMethodDirector.java:689)
at org.apache.commons.httpclient.HttpMethodDirector.processAuthenticationResponse(HttpMethodDirector.java:663)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:192)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at integration_conso_dwh.test_0_1.test.tFileFetch_1Process(test.java:668)
at integration_conso_dwh.test_0_1.test$1.run(test.java:1085)
Exception in component tFileFetch_1
java.lang.Exception: Method failed: HTTP/1.1 401 Unauthorized
at integration_conso_dwh.test_0_1.test.tFileFetch_1Process(test.java:671)
at integration_conso_dwh.test_0_1.test$1.run(test.java:1085)
disconnected
Job test terminé à 09:36 30/08/2013.

Regards,
Stef

[Anonymous]

Hi,
There is a response given by shong in https://community.talend.com/t5/Design-and-Development/problem-downloading-file-from-https/td-p/9123...
Would you mind checking it?
Best regards
Sabrina

[Anonymous]

Hello Sabrina,
I've seen with people in charge of this url, and they have noticed that the user/password is pass in the header http but also in the URL.
For them, there is a conflict and I should just pass the authentification by the header http. There is something in the tFileFecth which add automatically the authentifcation in the URL.
Do you know what ?

Regards,
Stephane

[Anonymous]

Hi,

I've seen with people in charge of this url, and they have noticed that the user/password is pass in the header http but also in the URL.
For them, there is a conflict and I should just pass the authentification by the header http. There is something in the tFileFecth which add automatically the authentifcation in the URL.

There are four protocols: http, https,ftp and smb. It seems that there is no such option adding automatically the authentifcation in the URL in the tFileFecth.
You can open a jira issue on talend bug tracker to see if there is a solution for that.
Best regards
Sabrina

[Anonymous]

Hello Sabrina,
Finally, peoples in charge of the url can only see my call when I'm using internet explorer.
The conflict of authentification is not done by the tFileFech ...
But something in the tfilefech generates a HTTP/1.1 401 Unauthorized exception.
Proxy parameters are good.
Maybe the format of my url pose problem to the tFileFech component ?
thsi is my url :
https://xxxxxxxx.com/v1/xml/GetData?&start=2013-07-16&end=2013-07-16&max-results=30&columns=lortal_s...
I've tried to call the url https://xxxxxxxx.com/v1/xml/GetData by passing parameters (start, end, max-results, columns, sort, site, start-index) with the post method but I have the same problem.
Regards,
Stephane

[Anonymous · Accepted solution]

Finally, I've solved my problem by :
- make an exception in the firewall for my url
- using a tHttpRequest with authentication
Regards,
Stephane