Hello,
I am working on a migration job where we transfer Email bodies into Salesforce Task object. The Body goes into teh description field, as the this is a long text field, also supporting line breaks.
The individual messages can have several kb in size, so with a row commit level of 5000 the size limit of 10MiB is reached for some batches.
To my understanding, in this case, the batch is limited by the 10MiB, not by the 5000 rows. At least the uploaded request chunk is never exceeding this limit.
BTW the default value of the component is 10MB =10,485,760 bytes , whereas the real limit is 10MiB = 10,000,000 bytes, should be adapted as well in the future...
You will get an error message in some cases like:
InvalidBatch : Failed to parse CSV. Exceeded max file size: 10000293. Max file size in characters should be less than or equal to 10000000
But with correct batch size limit, some batches wont get processed, instead I get the following error message:
InvalidBatch : Failed to parse CSV. EOF reached before closing an opened quote
Inspecting the request, the body of the email is "cut", just before the row where the 10MiB limit is reached.
I think Talend does not respect multi-line content. This is something the component should handle in a smarter way.
I think this should be fixed within the component on Talend side?
Any suggestions is welcome, except adjusting row size until no loading errors occur.
Thanks, Alex
