topic Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size? in Talend Studio

Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Sat, 16 Nov 2024 13:32:03 GMT

Hello,
i get the error "The code of method tOracleInput_3Process() is exceeding the 65535 bytes limit" when running my job.
The mentioned component is the start component of a subjob in the whole job. The name of the component is GP_DATEN_OBG.
How can i reduce the size of this subjob?
All i do is reading the data out of a table and processing it by calling 5 stored procedures and by doing 5 inserts.
The rest is for logging the progress into a logfile.
So i don't know how to shorten the code. There is nothing i can omit.
Many thanks for any help.
Regards Hartmut

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Thu, 25 Feb 2010 13:03:46 GMT

This doesn't solve the problem, but you are in company: https://community.talend.com/t5/Archive/resolved-If-else-if-expression-problem/td-p/172113
There are Java limits on method size, but of course this kind of hard limit are to be taken in account when choosing a code-generation approach: so only
Talend can do something on this matter (optimize / generate synthetic methods to split code in more chunks).
I see also this outside Talend in the past... this is the same limit that someone have experience when trying to autogenerate ie. JSP pages with more than 64k...
bye

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

jkrfs — Thu, 25 Feb 2010 15:30:17 GMT

I had to separate my jobs, there really is no way around it. 😕

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Mon, 01 Mar 2010 12:05:07 GMT

Hello,
thank you for your answers.
I also have splitted the work of the subjob into 3 parts and use now 3 subjobs.
It works in this way but i have one problem with the transaction control.
If subjob 1 works fine but while processing subjob 2 there comes an error, then my database is in an inconsistent state if the results of subjob 1 are not rolled back!
So the big question is, if there is any implicit COMMIT done when calling a stored procedure which inserts an entry into a table?
Does anybody know this?
In my whole job i create a database connection (tOracleConnection) and use this connection in every component accessing the database.
At the end of the whole job i do a COMMIT (tOracleCommit) if everything works fine.
Can i be sure that everything is rolled back in case of any error?
Many thanks in advance for any information.
Regards Hartmut

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Mon, 01 Mar 2010 13:10:13 GMT

Hello again;-)
to be sure that there is a ROLLBACK in case of any error i have added 3 tOracleRollback components.
I hope that in this way i can guarantee the transactional consistency of my database.
Regards Hartmut

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Mon, 01 Mar 2010 15:20:06 GMT

One option might be to use the bulk outputs rather than the regular outputs. You can use those to generate BCP files, then in your last subjob do a single BCP of all of the data.

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Tue, 02 Mar 2010 07:14:23 GMT

Hello Andrew,
thank you very much for this hint, i will check that.
At the moment i have a runtime of 40 minutes for processing 254.000 records.
Thats o.k. for me.
But if it will even go faster with bulk inserts then it would be great;-)
Regards Hartmut
I now have tested the Bulk Insert but the SQL*Loader hangs:
http://www.talendforge.org/forum/viewtopic.php?pid=37876#p37876

Re: Subjob exceeds the 65535 bytes limit -- how to reduce the size?

Anonymous — Tue, 02 Mar 2010 07:19:47 GMT

Hello,
i have new information about transaction control:
When running my job (consisting of a prejob and 4 following subjobs) there is no implicit COMMIT done.
That is there is really only one COMMIT at the end of the complete job.
That is when an error occurs while the job is running everything is rolled back.
That is exactly what i want.
Regards Hartmut