Adding a transform to an existing task/table triggering a FULL load

[RichardP]

I added a transform to an existing task/table/column and it triggered a full load. Is there a way to prevent this?

[PGN]

Did you try Run/Advanced Run Options/Tables are already loaded. Start processing changes from: ?

Not sure if that works when changing/adding a transform, but it may be an option to try.  We've done this when migrating to new servers and restoring databases, so that we do not need to trigger a complete reload.

[Alan_Wang · Accepted solution]

Hi Richard,

A transformation shouldn't trigger a reload of all the tables, but tables affected by the transformation will be reloaded. This is by design as transformations are recreating the table structure. If you are changing the precision on the column, you can execute source table DDL changes which most endpoints would be able to handle without a table reload. 

[RichardP]

Thanks for the quick replies- it helps!

So just the operative table reloaded- not all tables in the task. I didn't try advanced run options, I want to resume where it left off. 

So I want to transform an existing column and not trigger a reload of table it belongs to. This is for an obfuscation project. For example I might be just adding NULL or 'XXX' to the transform. Ideally I would modify the task, resume processing, future records would be obfuscated, and I would manually run DML on the target to obfuscate extant records. Can I operate on this happy path?

[RichardP]

Support responded saying to stop the task, update the column with a transform, and use advanced tables loaded start from- and these steps should not trigger a reload.

[lyka]

Hi Richard,

We recommend testing this in your non-prod region to make sure that you dont encounter any issues in production. We want to avoid any metadata corruption or any unexpected problem. good luck!

If you encounter any issues, you can always open a case with support.

Thanks

Lyka