I have a large lookup from salesforce in talend job...
i attached a picture for better visuals!
the order item object in salesforce contains millions of records
I want to delete all order items inside and order which is why i'm doing two lookups
first for the order ID that matches order number from sql, then using that order ID to match with all the order items within that order, finally delete.
THe order item object contains millions of records, i dont want to do reload at each row because it would crash and error out....
Once you identify the list of matching orders (after first join), you can store the data to a stage table.
Then you can release the matched order data in a controlled manner and you can even add the order list as parameter to order item table. This flow to iteration in a controlled manner will help you to delete the necessary records.
Warm Regards, Nikhil Thampi
Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved
