topic Re: Running Redshift multiple queries via Talend Open studio in Talend Studio

Running Redshift multiple queries via Talend Open studio

RS7 — Fri, 15 Nov 2024 23:46:05 GMT

Hi,

I'm new to Talend. I'm using Talend open studio.

I have table in redshift which stores queries(DDL, DML) in a column. I want to bring that query column in Talend and execute them one by one.

If there is any error i want to save in a table and also want to write success/failure status to Redshift table.

Thanks

Re: Running Redshift multiple queries via Talend Open studio

Anonymous — Tue, 07 Sep 2021 09:10:42 GMT

Hello,

You can use tFlowToIterate to iterate each row which stores queries(DDL, DML) and a variable in the query of tMysqlInput_2.

The job design should be: tMysqlInput_1--row1-->tFlowToIterate---->iterate-->tMysqlInput_2--->output

Best regards

Sabrina

Re: Running Redshift multiple queries via Talend Open studio

RS7 — Tue, 07 Sep 2021 23:57:45 GMT

Thanks Sabrina. I'm totally new in this area, could you please able to draw a rough design in studio and send me image.

Appreciate for your help

Re: Running Redshift multiple queries via Talend Open studio

Anonymous — Wed, 08 Sep 2021 07:12:53 GMT

a simple job design is as below.

tRedshiftInput--main(row1)--tFlowToIterate--tRedshiftRow

in the Query field:

(String)globalMap.get("row1.query")

Note: query is the column name defined on the schema of tRedshiftInput.

Please make some testings and see if you are able to read the query from table and execute it on tRedshiftRow one by one.

Regards

Shong

Re: Running Redshift multiple queries via Talend Open studio

RS7 — Tue, 12 Oct 2021 09:59:24 GMT

Thanks Shong.

My job is now running fine but when any error happen then tRedshiftRow is not going to OnComponentError side. For testing i'm inserting a row into a table that don't exist and expecting tRedshiftRow trigger onComponentError. I added image of my job below. Basically i want to update audit table. If query execute fine then i want that update audit table as success and if query fail then mark that particular query record as failed. Thanks