Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello,
I have an issue with tsalesforceoutput in talend 6.4.1.
While using the tsalesforceoutput with a commit level set above 1, the first batch (successfully updated within the salesforce org) isn't logged into the output files.
As you can see, 199 records (the first ones to be updated) were lost in the execution, but a quick check in salesforce shows that those records were successfully updated.
Could you point me onto the right direction to resolve this issue, or at least tell me which version of talend can be used without this error ?
Thanks.
Hello, first, i'd like to apologize for the time i took to respond, i was away frome the office. Then i'd like to thank you all for your response, although it didn't resolve my issue, it did help me figure out other tests to do.
Debugging :
Input : 2521 records
Commit Level : 200
Cf : debugOver200.png, debugUnder200.png
With this, i tried to understand where exactly the component was bugged. I put a duplicate record into the first batch. Before the row 200, no records go beyond the SalesforceComponent. After, they are put inside the success files the way i expected it to (But starting at the record 200, the 199 first one are lost). The error (always the same error message, the first one encountered) just add up to the size of the batch (200 lines).
Execution without header and without error :
Input : 2521 records
Commit Level : 200
Cf : executionWithoutHeaderAndErrorFree.png
I tried different version without header, like @Florentina said. In this one, we see 2322 rows in the success file.
2521 - 2322 = 199 records lost.
Execution with header and Error in the first batch :
Input : 2521 records
Commit Level : 200
Cf : executionWithHeaderAndErrorFirstBatch.png
With the header on, and a duplicate record in the first batch, we get 200 lines in the rejected (expected : 2), and 2322 in the success.
200+2322=2522. The error message are always the same.
Execution without header and Error in the first batch :
Input : 2521 records
Commit Level : 200
Cf : executionWithoutHeaderAndErrorFirstBatch.png
Same execution, but without header. The result is exactly the same than before.
The log can be seen in logExecutionWithoutHeaderAndErrorFirstBatch.png
Execution without header and Errors in multiple batches :
Input : 2521 records
Commit Level : 200
Cf : executionWithoutHeaderAndErrorsInSameBatchMultipleTimes.png
In this test, i tried to put errors in multiple batches, so i duplicated records in different batches. That's where it gets interesting. For every batch containing flawed records, you get 200 errors messages printed out. No matter how many error a single batch contains, it will always print the first occurence, 200 times. Plus, errors isn't separated from success. In the exemple i provided, you can see 400 errors + 2322 success = 2722 records (?!?). But it is only with 2 different batches flawed. I tried with more errors in more batches, and went up to 1200 records in the errors, and still 2322 in the success. Why ? Absolutely no idea.
Execution with small dataset :
No header, No error :
Input : 179 records
Commit Level : 200
Cf : executionWithoutHeaderWithoutErrorDatasetSmallerThanBatch.png
I tried with smaller dataset to see if i could reproduce the results of @Florentina. No success, No errors, nothing.
No header, Error :
Input : 179 records
Commit Level : 200
Cf:executionWithoutHeaderWithErrorDatasetSmallerThanBatch.png,
executionWithoutHeaderWithErrorDatasetSmallerThanBatchCeaseOnError.png
Same thing than previous one. I tried a cease on error to see if the job saw the error, and it did. But without it, no success, no errors.
Your tests were really helpful, and your advices where pretty much the same thing i came up with on my own. The matter is that we need both the logs and the response time, but we can't use bulk and its asynchronous feature.
Let me be clearer. We use those talend jobs in workflows inside salesforce, meaning that we import data, then operate on it. We need a synchrone way to load them, because we need to be sure that all the records will be loaded before the operations on them start.
Thank you for your time
Nico
The output from tSalesforce components is not right, it hasn't been right for a long time...maybe never. Chaning the commit level to 1 to get proper logging is not a solution; This is a bug and honestly a pretty big bug if you are trying to reliably use tSalesforce components for anything serious...eg. it can't accurately / reliably report the outcome of an insert operation?...not good.
I've created a bug: https://jira.talendforge.org/browse/TDI-40145. Not sure what happens from here, but maybe if we get comments, or votes, or whatever to get the bug noticed might be a start to getting it fixed. It seems as though you all have put some indepth work into steps to repro, etc. Maybe you can add your notes to the existing bug?
Hello @jdope,
I already tried to submit a bug, so far no response. I created it in october, although i didn't really fed it with info, considering the fact that i didn't get any return. However what i did get is an assignee. I provided the URL to your bug, and the URL to this topic, hoping that maybe it will catch some attention. I also started watching your bug, and voted for it.
But for the adding my notes part, i'm not really sure how i am supposed to do it. The tests i did are quite space consuming, and i don't think it can fit inside a comment (which seems to be the only way for me to add something meaningful to your issue).
Any idea ?
Nico
@Moe ok great. Yeah as far as "adding notes" I think a link to this community post is probably good enough. Thanks for cross-referencing the bug I created with your bug. What was that bug #? I'd like to add that as a comment to the one I created.
@xdshi can you elaborate on the process for getting jira items evaluated / worked? Is there something from a community level that we can do (votes, comments, etc) to get the bug's priority elevated and worked?
Related bugs:
https://jira.talendforge.org/browse/TDI-40156
https://jira.talendforge.org/browse/TDI-40145
(these can be merged in JIRA if a JIRA admin JIRA is looking at this)
Anyone else who may be experiencing this issue please login to JIRA and vote up (right side) these JIRA issues!