tFileFetch Ignores tSetProxy

[jasonatunr]

I have a job with a tFileFetch component to upload files in the job. Our traffic has to go through a proxy, but the proxy used is dependent on the context we are running.

For Example: production context = use proxy sample.com; development context = use <none>

Due to this need I cannot use the proxy settings on the tFileFetch component and instead executing a tSetProxy first then going to tFileFetch after the tSetProxy completes. The tFileFetch component does not pick up the proxy settings and instead tries to connect directly without any proxy settings. However, if I set tSetProxy to socks it does attempt to connect to that proxy, but our proxy server is not socks so the authentication message fails.

[Jesperrekuh]

I suggest using cUrl ... install it on your OS. Way easier to implement web integration...
I switched using talend web components (not for message queues) to cUrl, because of cookies, sessions, certifications, follow redirects, put post get , etc... way more flexible and route stdout to talend for proces information.
Good luck

[Anonymous]

tSetProxy sets the JVM default settings for the TCP stack and change the proxy settings. It is simply impossible for any component using the network to ignore that.

Are you sure the tSetProxy is in action before your tFileFetch is working?

[jasonatunr]

I agree, but here we are with it ignoring these settings. Below is how my job is designed. The scenario I'm testing is quite simple now and it simply does not care about the proxy being set.

Capture.PNG

[Anonymous]

The proxy settings are also dedicated to a particular protocol. Please take care your used protocol are part of the new settings.

[jasonatunr]

They are both set to http.

[jasonatunr]

I have tried several versions and I feel I should add some more diagnostics.

If I have a tSetProxy->tRestClient->tFileFetch the tRestClient goes to the proxy appropriately, but the tFileFetch still does not go to the proxy and instead attempts to connect directly.

I have also tried using java to explicitly assign the proxy to system.properties, but I get the same result as if I am using the tSetProxy. This leads me to conclude that the problem is definitely in the tFileFetch component.

[Jesperrekuh]

cUrl... =|;-))

[jasonatunr]

Security limitations of our deployment server limit our use of curl and cannot us it in this circumstance.

[Jesperrekuh]

Really Im sorry for you but this 'security' thing made me laugh. Must be frustrating for you.
If you are a little bit into programming and java: use Apache HttpClient : http://hc.apache.org/httpcomponents-client-4.5.x/httpclient/examples/org/apache/http/examples/client...

Look into the manual creating custom Routines in Talend. Pretty easy.