topic Re: TOS DI Ubuntu - Performance issue on wide datasets in Installing and Upgrading

TOS DI Ubuntu - Performance issue on wide datasets

JEichhorn — Sat, 16 Nov 2024 00:34:46 GMT

Hi there,

our Data Analysts see performance issue when working with TOS DI, editing datasets having many columns.

We are using TOS DI 7.4 on Ubuntu 20.04 with JDK /zulu11.45.27-ca-jdk11.0.10-linux_x64/

The test case is

160 Columns in tFileInputDelimited

tMap Mapping into

tLogRow output.

In the tMap component, when selecting a specific row in the left schema pane, it takes up to 15sec until the column gets highlighted. In these 15sec the main java-process runs over 100% CPU-time. Each time selecting another element for mapping, it takes this much time. Generally, opening and defining any schema (from flatfile or DB and in all components which display the columns names) needs more time, the more columns are included, starting to become noticeable when using about 40 columns names)

I've tried disabling Theming, Animations, and Autmatic Refresh of Repository changes. None of these which did not solve the issue

The problem does not occur, using TOS DI on Windows 10.

The issue is getting way worse, when using TOS DI on our terminal server using x2go. Then the process get stuck up to 30sec.

Does anybody have an idea on how to solve this?

Re: TOS DI Ubuntu - Performance issue on wide datasets

Anonymous — Thu, 18 Feb 2021 10:13:32 GMT

Hello

The 7.4 milestone version is just for new features/bug fixes testing, I would suggest you to install the latest general availability (GA) version 7.3.1 on a compatible operating system, to check if you still have the performance issue.

Regards

Shong

Re: TOS DI Ubuntu - Performance issue on wide datasets

JEichhorn — Thu, 18 Feb 2021 13:57:51 GMT

Hi Shong,

thanks for looking into this.

Following your advice, I've setup Ubuntu 18.04 LTS and Zulu JDK.

Sadly I have to stat, that the findings mentioned, before still apply.

Do you have other suggestions?

Thanks a lot. Regards, Joerg

Re: TOS DI Ubuntu - Performance issue on wide datasets

Anonymous — Fri, 19 Feb 2021 03:11:47 GMT

@Jörg Eichhorn , I don't have a Ubuntu 18.04 environment for testing, can you please open an issue on our bugtracker with an example job?

Regards

Shong

Re: TOS DI Ubuntu - Performance issue on wide datasets

JEichhorn — Tue, 02 Mar 2021 13:45:54 GMT

Ahoi @Shicong Hong

I've managed to setup an account in bugtracker.

Could you please point me to the apropriate project, in which to report?

Thanks a lot. Regards, Joerg

Re: TOS DI Ubuntu - Performance issue on wide datasets

Anonymous — Thu, 04 Mar 2021 03:06:44 GMT

You can open the issue under Talend Unified Platform (TUP) project, thanks!

Regards

Shong

Re: TOS DI Ubuntu - Performance issue on wide datasets

JEichhorn — Thu, 04 Mar 2021 07:55:26 GMT

Thanks for your help, I've opened https://jira.talendforge.org/browse/TUP-30622