topic Talend 7.2.1: Import project heap space issue in Talend Studio

Talend 7.2.1: Import project heap space issue

Anonymous — Fri, 13 Dec 2019 10:33:23 GMT

Hello,

We encountered some issue with Talend Studio.

We are using :

Talend Open Studio for Data Integration
Version: 7.2.1

Here is the description of our issue:

- We have a project containing several jobs.
- after any modification to a job we export the project following this procedure:

-- File -> Export...
-- Folder General -> Archive file
-- a popup window open with several folders of our jobs and one folder with the main project
-- we only select the folder of the main project (so we check the checkbox of the project)
-- at the bottom of the popup, we check the box: Resolve and export linked resources
-- we click finish

- this generate a zip file with the whole project and we push this zip file on our SVN repository (this is the way we share our work with the other dev members)

Here is the issue:

- when we try to import this exported zip project, the import takes a really long time and throws an "heap space full" exception
- we noticed that the project zip file, once unzipped, is really massive : 350 MB... Among those MB, there is one file of 300 MB !
- the path of this file is .settings/relationship.index

Do you have a step-by-step process to export our project the right way ?
Maybe we are doing something wrong but we were unable to find any doc on Talend website

Thank you a lot for any help or advice

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Mon, 16 Dec 2019 08:48:23 GMT

Hello,

We are supposing that you have already checked this online documentation about:TalendHelpCenter:Importing projects .

Could you please upload your studio error log here? It is located in <Talend Studio installation path>/workspace/.metadata/.log

Best regards

Sabrina

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Mon, 16 Dec 2019 09:09:29 GMT

Hello,

Thank you for your quick reply.

We already followed the online documentation.

We noticed that the Export steps are a bit different in the doc and within our version of Talend Studio: for instance there is no checkbox with "Resolve and export linked resources" in your documentation

Please find attached our log -> you can find the issue with the key word "heap".

I found one of the error on the 2019-11-19 16:19:34

Best regards

.log

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Mon, 16 Dec 2019 14:10:56 GMT

Hello @gip-ph7 ,

Could you try to assign more ram to the .ini file Talend-Studio-linux-gtk-x86_64.ini (Linux), Talend-Studio-win-x86_64.ini(Windows) as a quick test?

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Mon, 16 Dec 2019 14:46:08 GMT

Hello,

We tried to add some RAM in the .ini file and indeed it solved our problem.

BUT we dont really like to allow so much RAM to Talend only.

What we dont understand is that, before version 7.2.1, our exports of projects were really easy:

We could export the project as a folder, everything was inside it (we had no need to take the .settings subfolder)

Now, since version 7.2.1, we have to export as a zip file and to take everything otherwise the project we show some errors after an import. But the size of those archives is increasing rapidly...

What we really need is a step by step process to export our projects so that:

- the size stays roughly the same

- we dont lose any mandatory file/data in the process and any other member of the team is able to reimport the zip file (or folder) to work on the project if needed

- we dont need to allow several GB of RAM to Talend

- the step-by-step process is valid with version 7.2.1

Thank you for your help

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Mon, 16 Dec 2019 15:04:29 GMT

Hi again @gip-ph7 ,

I really do not know how many jobs do you have in your project but 350 MB its a lot. Try to use the function "exports items" https://help.talend.com/reader/rhidJAe~uqB6FTNpM4g70w/ApzXLNcwsMUd74snojcNvQ

PD: Check "Export dependencies"

This will only export the necessary metadata to reproduce your project in other environment

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Tue, 17 Dec 2019 09:28:12 GMT

Hello,

We have tried your solution and it seems ok so far

I made an export of items and checked "Export dependencies"

One of my colleagues tried to import and work on the project: there are no issues for the moment.

The export folder is really small, like 7 MB !

I dont understand how it can be so different in size depending on the procedure you use...

We are making a few more tests before we make a final "Accept as solution"

Anyway thanks a lot for your help !!

Re: Talend 7.2.1: Import project heap space issue

Anonymous — Wed, 18 Dec 2019 07:10:49 GMT

Thank you again for your help.
We made some modifications on the project and used your export procedure between those modifications and all seems well !