Files being deleted Automatically post run

[Anonymous]

Hi Team,

I am facing an issue where my Talend job is deleting the files automatically post run.

The data is transferred to staging tables and post that it is deleting the files from my source.

Attaching here with the screenshot for reference.

[Anonymous]

You need to give us more information than this I'm afraid. What files are you talking about? What components are using files? Can you show us the whole job because nothing you have shown will use a file.

[fdenis]

without a good view of each component we cannot help you.
can you give more detail of each components?

[Anonymous]

Hi,

The flow of my Talend job :

Connects to Snowflake --> Copies data from file Staging --- > Commit

EX: "Copy into abc.def.mytable
from @~/acc/Idds/loc
file_format = (
type= 'CSV'
FIELD_DELIMITER = ','
)
;"

Post this run.. Data is being inserted in my staging table. But the file present in that path is being deleted.

[fdenis · Accepted solution]

if you are using snowflake you may need to use something like set stage_copy_options = (purge = false)
regards,

[Anonymous]

Hi Denis,

Thank you so much fro your response.
But I do have similar jobs which run with the same logic and does not cause any issue.
Is there something with Talend which may cause this issue.?

[fdenis]

on, in your case Talend is just sending sql query to snowflake runtime.

[Anonymous]

Thank you so much .. It worked !!