Move file using GET command in Talend

AbhishekBoga — Sat, 16 Nov 2024 00:09:02 GMT

Hi Team,

I have a scenario where I am putting all the files into snowflake stage area using PUT command and then loading all the files from snowflake stage area into Snowflake table using COPY INTO command. The COPY INTO command gives the output which says which files are LOADED, LOADED_FAILED & PARTIALLY_LOADED.

Now I want to read the output result of COPY INTO command and based on the above file status move the LOADED files into Archive folder and the rest into Reject folder. I came to know about the GET command in snowflake which move all the files from snowflake stage area into the table, but we can’t filter in GET command.

For below problem, my solution should be:

Move the files based on wildcard search into Archive / Reject folder
Process should be optimize since it will process 1000-2000 files at a time
File size is less (1-5 kbps)

Kindly let me know what are the alternatives to achieve the above problem

Re: Move file using GET command in Talend

Anonymous — Wed, 26 May 2021 09:58:43 GMT

Hi @not specified not specified,

You should be able to achieve these commands using the tDBRow component. Have you tried this? You can take example code from Snowflake's help and use it in the tDBRow component. One command at a time.

Can I ask that you only select labels that are directly related to your question? It is important that questions are categorised as correctly as possible.

Thanks.

topic Re: Move file using GET command in Talend in Talend Studio

Move file using GET command in Talend

Re: Move file using GET command in Talend