topic Re: Files being deleted Automatically post run in Talend Studio

Files being deleted Automatically post run

Anonymous — Mon, 16 Jul 2018 15:03:15 GMT

Hi Team,

I am facing an issue where my Talend job is deleting the files automatically post run.

The data is transferred to staging tables and post that it is deleting the files from my source.

Attaching here with the screenshot for reference.

Re: Files being deleted Automatically post run

Anonymous — Mon, 16 Jul 2018 15:07:01 GMT

You need to give us more information than this I'm afraid. What files are you talking about? What components are using files? Can you show us the whole job because nothing you have shown will use a file.

Re: Files being deleted Automatically post run

fdenis — Mon, 16 Jul 2018 15:11:58 GMT

without a good view of each component we cannot help you.
can you give more detail of each components?

Re: Files being deleted Automatically post run

Anonymous — Mon, 16 Jul 2018 15:13:22 GMT

Hi,

The flow of my Talend job :

Connects to Snowflake --> Copies data from file Staging --- > Commit

EX: "Copy into abc.def.mytable
from @~/acc/Idds/loc
file_format = (
type= 'CSV'
FIELD_DELIMITER = ','
)
;"

Post this run.. Data is being inserted in my staging table. But the file present in that path is being deleted.

Re: Files being deleted Automatically post run

fdenis — Mon, 16 Jul 2018 15:33:20 GMT

if you are using snowflake you may need to use something like set stage_copy_options = (purge = false)
regards,

Re: Files being deleted Automatically post run

Anonymous — Mon, 16 Jul 2018 15:36:39 GMT

Hi Denis,

Thank you so much fro your response.
But I do have similar jobs which run with the same logic and does not cause any issue.
Is there something with Talend which may cause this issue.?

Re: Files being deleted Automatically post run

fdenis — Mon, 16 Jul 2018 15:40:22 GMT

on, in your case Talend is just sending sql query to snowflake runtime.

Re: Files being deleted Automatically post run

Anonymous — Thu, 19 Jul 2018 17:14:16 GMT

Thank you so much .. It worked !!