Skip to main content
Announcements
A fresh, new look for the Data Integration & Quality forums and navigation! Read more about what's changed.
cancel
Showing results for 
Search instead for 
Did you mean: 
gt0731
Contributor III

Memory issue for Linkedin/salesforce sync integration job

 Attached  zip job is crashing due to too much processing at once and not enough RAM to do it and therefore needs simplifying and generally updating to make it better overall.

 

Any help on it would be much appreciated ?

Also, Advise on  what measure need to be taken here at memory level.

Labels (2)
11 Replies
Jesperrekuh
Specialist

which version are you on.
Please show a screenshot... we will point out what to adjust and how.
Memory and processing power is system related... so it works here and fails at your side ...

What did you already try to improve performance?!?
gt0731
Contributor III
Author

I already attached complete zipped folder about this job. Are you able to
extract the folder ?

I tried to implement
Apex soap api and at memory still struggling to figure out the process
Jesperrekuh
Specialist

Again, what version are you running? I need to be able to import your project and your Talend version could differ from mine. And what is the struggling part?

Love to help... but it's like asking... please solve world peace for me... and hand me over all of your shizzle ... and expect people here within the community just know what to do... and we have our own talend apocalypse too 0683p000009MA9p.png)

gt0731
Contributor III
Author

Sorry as i posted at my night time. I missed out prompt reply


I am using Talend Open Studio for Data Integration v6.3.1.2016.

I attached the screenshot of where the job crashes due to lack of memory. I am assuming your job will run all the way through as it isn't running the whole process where all the files on our servers and compared to pitchbook to produce the complete spreadsheet as at job tFileList_2.
I tried
I have already tried to increase the Ram argument under the Advanced setting to add the -Xmx2048M but this hasnt worked. I am not a Talend expert at all and have only just started using it and therefore my knowledge of it is very basic.
gt0731
Contributor III
Author

I also attached screenshot of job as well. files called linkedin_sync.jpg where in i highlighted the painful area
gt0731
Contributor III
Author

any help on it would be much appreciated ?
Anonymous
Not applicable

Hello,

Could you please let us know if this article helps?

https://community.talend.com/t5/Migration-Configuration-and/OutOfMemory-Exception/ta-p/21669

Best regards

Sabrina

gt0731
Contributor III
Author

@xdshi @Dijke

I will implement the your mentioned link as well. 

Here i am posting the exact error log for your refrence. 

This is the error log:

Starting job RMS_LinkedIn_Sync at 17:26 14/09/2018.

[statistics] connecting to socket on port 3825
[statistics] connected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
[statistics] disconnected
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[statistics] disconnected
at org.apache.xmlbeans.impl.store.Cur$CurLoadContext.attr(Cur.java:3044)
at org.apache.xmlbeans.impl.store.Cur$CurLoadContext.attr(Cur.java:3065)
at org.apache.xmlbeans.impl.store.Locale$SaxHandler.startElement(Locale.java:3263)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.reportStartTag(Piccolo.java:1082)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseAttributesNS(PiccoloLexer.java:1802)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseOpenTagNS(PiccoloLexer.java:1521)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseTagNS(PiccoloLexer.java:1362)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXMLNS(PiccoloLexer.java:1293)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXML(PiccoloLexer.java:1261)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.yylex(PiccoloLexer.java:4812)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yylex(Piccolo.java:1290)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yyparse(Piccolo.java:1400)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:714)
at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3479)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1277)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1264)
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:345)
at org.openxmlformats.schemas.spreadsheetml.x2006.main.WorksheetDocument$Factory.parse(Unknown Source)
at org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:194)
at org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:186)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:354)
at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:166)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:240)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:296)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileInputExcel_8Process(RMS_LinkedIn_Sync.java:20957)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileInputDelimited_2Process(RMS_LinkedIn_Sync.java:29394)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileList_2Process(RMS_LinkedIn_Sync.java:19152)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileDelete_8Process(RMS_LinkedIn_Sync.java:18439)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileDelete_7Process(RMS_LinkedIn_Sync.java:18263)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tFileDelete_4Process(RMS_LinkedIn_Sync.java:18087)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tRunJob_7Process(RMS_LinkedIn_Sync.java:61891)
at linkedinsync.rms_linkedin_sync_0_10.RMS_LinkedIn_Sync.tRunJob_6Process(RMS_LinkedIn_Sync.java:61661)
Job RMS_LinkedIn_Sync ended at 17:27 14/09/2018. [exit code=1]

Anonymous
Not applicable

Hello,

Have you already tried the solutions mentioned in article? Is it working for you? Don't hesitate to post your issue here.

Best regards

Sabrina