I'm having the following issue with DeltaLake Output (tDBOutput), when I select just the 'INSERT' or just the 'UPDATE' for 'Data Action', it is working great and processing inserts nice and quick (screenshot attached), processed 30k records in 70 seconds.
When I select 'UPDATE_OR_INSERT' or
'INSERT_OR_UPDATE'
, the processing speeds are awful.Any suggestions would be appreciated.
By using 'Insert or update' operation(No batch size settings in it's advances settings), for each row, it will do 2 db operations (one for lookup, another one is for db insert or update).
To speed the insert or update performance, it's recommended to use tMap do a lookup with the DB data and create 2 output rows : Update , Insert,
Then select the operation 'Update' for the update row, and select the operation 'Insert' for the insert row in the Db output components.
