topic Re: Talend Open studio Component Installer ignores proxy in 7.2 in Installing and Upgrading

Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Sat, 16 Nov 2024 05:07:12 GMT

Hi,

Like many of us, I live behind corporate proxy. In 7.0.1 the component installer works with proxy as expected. However, recently I tried to upgrade to 7.2.1 and utterly failed. I have proxy set at JVM level, at the application properties and in Java console. The component installer ignores all of it and still attempts direct connection. I verified this with Wireshark.

Log entries for the failure.

!ENTRY org.talend.platform.logging 1 0 2019-07-30 10:54:17.371
!MESSAGE 2019-07-30 10:54:17,362 INFO org.talend.librariesmanager.librarydata.LibraryDataService - org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.postgresqlostgresqlom:42.2.5 from/to talend (https://talend-update.talend.com/nexus/content/repositories/libraries/😞 Connection refused: connect

!ENTRY org.talend.platform.logging 1 0 2019-07-30 10:54:18.482
!MESSAGE 2019-07-30 10:54:18,473 INFO org.talend.librariesmanager.librarydata.LibraryDataService - org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact net.sourceforge.jtds:jtdsom:1.3.1-patch-20190523 from/to talend (https://talend-update.talend.com/nexus/content/repositories/libraries/😞 Connection refused: connect

Maybe there is a fourth way to set proxy that the LibraryDataService does respect?

I noted, that the "Update componets" part seems to respect the proxy set in Java console, not the one in the env/preferences. However, nothing seems to apply to the single component installers.

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Wed, 31 Jul 2019 11:10:11 GMT

Hello,

We are able to get these two jar files from talend update site via browser. Does this issue only block these two jar files?

Best regards

Sabrina

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Wed, 31 Jul 2019 11:18:41 GMT

Yes, I can get them from browser too, if I manage to guess the URL as browser respects proxy setup. I didn't have time to do that. And my project needs in full a bit more than those 2... It's more than a little bit painful to guess full url of each and manually download. The proxy ignoring is problem not with those jars, but that down loader that does not seem to care about any proxy settings at all.

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Thu, 01 Aug 2019 03:40:14 GMT

Hello,

Not sure about how does your proxy setting look like. We will appreciate it a lot if you could give us some proxy setting screenshots on community. And please make sure there is no firewall or network issue.

Best regards

Sabrina

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Thu, 01 Aug 2019 08:30:28 GMT

Hi,

My network blocks all non-proxy internet access. this is normal in corp intranet. Proxy is supposed to be used. All other ways that Talend Open Studio is accessing internet, including the connectivity check on startup and the "feature downloader" respect proxy setup and connect fine, however, this particular part, the artifact loader of the in-ui downloader attempts a direct connection. I have checked this with packet captures. All other Talend traffic goes to proxy, this particular function however does not. As my proxy configuration works for all other internet activity in Talend, it must work.

I have set proxy in total 3 places, the JVM proxy options, the Preferences and in the Java Control Panel.

The first two duplicate each other as I can see in Talend startup logs that the UI prefrences are used to override the http.proxy* and https.proxy* parameters.

However, the component downloader/updater that pops up from the menu indicated

did not work and still attempted direct connections with that done. At that point, I configured the java control panel (picture below) with proxy instead of browser settings (there's apparently a known issue, that if the browser in turn is configured with PAC script that does not cascade to the java applications). This made the addional packages installer go to proxy, but fail to authenticate at proxy as it seems that proxy use method does not authenticate properly with windows integrated security for whatever reason. That I solved with my proxy managers and got to work.

With that out of the way only the in UI component downloader is left. With this there is nothing that can be done on proxy side as these artifact inventory info requests never go to proxy. Despite proxy being configured in any imaginable way and everything else using the proxy, it still does direct connection and fails...

In short, the artifact library access in org.talend.librariesmanager.librarydata.LibraryDataService class does HTTPS requests for POMs ignoring proxy. This I can not fix.

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Thu, 01 Aug 2019 08:45:52 GMT

Unless there is a 4th proxy configuration hole somewhere...

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Thu, 01 Aug 2019 09:00:39 GMT

I dug through the relevant code and saw that this is passed down to Eclipse-s Aether library, that seems to be the one ignoring the proxy...

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Thu, 01 Aug 2019 14:46:45 GMT

setting global _JAVA_OPTIONS up with proxy information had no effect on this issue either.

Re: Talend Open studio Component Installer ignores proxy in 7.2

schneich — Tue, 05 Nov 2019 11:57:37 GMT

Hi,

I am facing the very exact same issue.

I would very much appreciate, if Talend could provide a detailed manuel on "how to set up a proxy connection in corporate environment". Including authentication issues, such as "NTLM", ...

best regards,

Chris

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Wed, 06 Nov 2019 09:44:21 GMT

Hello @schnech1，

There is no such a detailed manual on "how to set up a proxy connection in corporate environment" as the corporate environment is complex and uncertain.

Are you getting credential problem when configuring proxy setting?

Best regards

Sabrina

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Wed, 06 Nov 2019 09:50:45 GMT

Hi,

It does not really matter how he configures it - the Eclipse base used in 7.2 has a bug in the proxy handling and no proxy configuration will take effect for certain library downloads... Unfortunately - you cant use 7.2 if you are behind a proxy. That was my conclusion. I'm still using 7.1 and hoping that the next version inherits from eclipse a fixed proxy handling.

Best,

Thade

Re: Talend Open studio Component Installer ignores proxy in 7.2

schneich — Wed, 06 Nov 2019 10:05:48 GMT

@xdshi wrote:

Hello @schnech1，

There is no such a detailed manual on "how to set up a proxy connection in corporate environment" as the corporate environment is complex and uncertain.

Are you getting credential problem when configuring proxy setting?

Best regards

Sabrina

Hi, unfortunately no. I only get log-entries such as this:

!ENTRY org.talend.platform.logging 1 0 2019-11-04 16:11:48.461 !MESSAGE 2019-11-04 16:11:48,461 INFO org.talend.librariesmanager.librarydata.LibraryDataService - org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.talend.libraries:jxl om:6.0.0 from/to talend ( https://talend-update.talend.com/nexus/content/repositories/libraries/😞 connect timed out

@thade Proxy-configuration has been a pain in every talend version. I took me weeks to get it working in version 7.0.1. I will try version 7.1 and test, if it is working there.

Re: Talend Open studio Component Installer ignores proxy in 7.2

schneich — Fri, 08 Nov 2019 08:53:31 GMT

Dear @thade and @xdshi ,

I just had a little session with our network specialist. A little summary.

We have a corporate environment with proxy (NTLM-Authentication), firewall and ssl-interception, scanning of internet traffic, ...

Version 7.0.1

To connect "Talend Data Integration" to the internet to be able to download additional components, we had to set an exception in the proxy. The proxy no longer asks for authentication, when trying to access the talend-update.talend.com URLs. With this rules, everything was fine.

Version 7.1

In addition to the above set of roules, we had to add "talend.com" to the list. I am not sure why, but now we are able to update Version 7.1 with additional components out of the talend repository.

Version 7.2

I can now confirm the statement of @thade : Talend Open Studio for Data Integration 7.2.1 does not use the proxy-configurations, but tries to access the internet directly. This is blocked by our network environment. (screenshot of our firewall)

Then there is something else: Talend sometimes asks me for the Community-Login. I added my credentials, but nothing seems to change. today, we have seen this error in the proxy-log. The proxy does not allow the connection to www.talendforge.org, because NTLM-Authentication does not work and because we did not add the url to the whitelist in the prosxy.

connection: service.name=Explicit HTTP client.address=xxx.xxx.xxx.xxx proxy.port=xx client.interface=0:0.1 routing-domain=default
  location-id=0 access_type=unknown
time: 2019-11-08 08:24:10 UTC
GET http://www.talendforge.org/exchange/webservices/contributedExtension.php?data={"contributedExtension":{"passwordHash":"xxx","username":"schnech1"}}
  DNS lookup was unrestricted
User-Agent: Java/1.8.0_232
user: unauthenticated
authentication status='need_credentials' authorization status='not_attempted'
EXCEPTION(authentication_failed): Authentication failed either because credentials were not provided or they could not be validated

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Tue, 26 Nov 2019 04:08:52 GMT

Hello @schnech1

Thanks for your summary. Does this issue only repro in V 7.2?

Best regards

Sabrina

Re: Talend Open studio Component Installer ignores proxy in 7.2

schneich — Wed, 27 Nov 2019 06:41:38 GMT

Hi @xdshi,

Hello @schnech1 

Thanks for your summary. Does this issue only repro in V 7.2?

Best regards
Sabrina

I assume you refere to my comment on the community login? No, this is related to Talend not being able to use proxy and NTLM-Authentication. This issue is present in all Talend versions 7.x. (As far as I know, the whole issue is based on the Eclipse version underneath Talend.)

Re: Talend Open studio Component Installer ignores proxy in 7.2

anna_t1 — Mon, 08 Feb 2021 20:38:56 GMT

What a headache? Has this been solved somehow? I'm still facing the same issues on 7.2 and I can't go change versions. I must submit to a linux server running this version, but when I try to build, it's always giving me error, because it can't fetch libraries through th proxy!

I've literally spent HOURS, trying to solve this.

This is ridiculous ... just ridiculous.

Talend should have provide a clean solution for this....

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Tue, 09 Feb 2021 08:48:33 GMT

Hello,

Sorry for any inconvenience.

What does your proxy settings of studio preferences look like?

Please make sure that configuring HTTP and HTTPS proxy at the same time, and user/password should be configured correctly both for HTTP and HTTPS.

So far, it seems that studio doesn't support ntlm authentication.

Is it possible to use BASIC authentication? or even don't use any authentication?

Best regards

Sabrina

Re: Talend Open studio Component Installer ignores proxy in 7.2

Anonymous — Tue, 09 Feb 2021 08:53:39 GMT

Hello,

Please add --talendDebug and -Dtalend.studio.proxy.printLogs=true when starting studio to try to catch proxy related logs and then attach all studio logs.

Best regards

Sabrina