topic Re: the code of the method is exceeding the 65535 bytes limit in Talend Studio

the code of the method is exceeding the 65535 bytes limit

Anonymous — Sat, 16 Nov 2024 11:22:56 GMT

Hello,
I read few other articles regarding the error message above, looks like it is a limitation in Java. I wanted to get thoughts on how to split my job or if there are any alternatives for this problem.
I am working with a schema of 100 columns and we have lot of rules. So far we only added five rules and we get the above error.
I am attaching the job along for reference, and we are using paid version of talend big data.
Would using dynamic schemas help with this issue ? some other thread was mentioning that this issue is fixed in 5.6 is that true ?
Thanks for all the help in advance.

Re: the code of the method is exceeding the 65535 bytes limit

Anonymous — Fri, 19 Dec 2014 18:49:44 GMT

adding image again.

In case the image is not visible my job looks like below.
tmongodbInput---> filter1 _----> filter2 ---->..............--->filter3----> tmongodboutput

Re: the code of the method is exceeding the 65535 bytes limit

Anonymous — Mon, 22 Dec 2014 10:16:43 GMT

Hi,

Thank you for your post! We can't see the screenshot on our side. Could you attach it on the forum, please? That would be great.

For " https://community.talend.com/t5/Design-and-Development/the-code-of-the-method-is-exceeding-the-65535-bytes-limit/td-p/74493" error, it's probably caused by your large number columns which generated code that exceed 65535 bytes.
Please see:http://docs.oracle.com/javase/specs/#88659

Best regards
Sabrina

Re: the code of the method is exceeding the 65535 bytes limit

Anonymous — Mon, 22 Dec 2014 10:21:23 GMT

Hello,
Unfortunately, your job isn't visible, but I can make some guesses about how the job is structured. Please let me know if I'm on track!
The limitation comes from the Java class file specification and isn't specifically related to any Talend component -- although they are related, since the generated code must respect the limitation. It usually occurs with jobs with large schemas, and the workaround is to manually force the job into several distinct steps.
I tested with a tMongoDBInput with 250 String columns, followed by 12 tFilterRows, followed by a tMongoDBOutput and the generated method is larger than the 64kB limit in Java in Talend v5.4 and v5.6.
If you need all of the columns during your filters, one workaround for breaking the job into smaller steps is to write an intermediary file to disk (after N filters) and trigger an OnSubJobOK to resume processing at the N+1'th filter. If the dataset is small enough, you can use a tHashOutput/tHashInput pair to collect the results in memory (but this is only appropriate when the intermediary step is guaranteed to fit in memory, which depends on your filters, of course).
There's another workaround that focuses on reducing the number of columns that need to be passed around, but that depends on the input structure of your data (is it a JSON document at a single level, or with more structure?) and whether only a few columns are actually used in the filters. Is this the case with your job?
It looks like the tMongoDBInput doesn't support the Dynamic type, unfortunately.
If the workaround doesn't suit your use case, I encourage you to file a bug with the specific details of your requirements.
All my best, Ryan