Create a generic pipeline that will read a configuration file and transfer data from source to destination table using the mapping. Map the source table/columns to destination table/columns using an ETL tool (Talend).

[uKumar1604167971]

 

Actually I am new person. I have very basic knowledge of talend.

Create a generic pipeline that will read a configuration file and transfer data from source to destination table using the mapping. Map the source table/columns to destination table/columns using an ETL tool (Talend)

requirement

1)

List all the database connections and see if it is already available.

a.

If yes: No action required

b.

If No: create the database (create a pop up to ask for credentials)

2)

Source Table:

a.

Validate source table exists. If not send an email of the error

b.

Validate source table columns exist: If columns missing then send error email

3)

Sink (Destination Table):

a.

Validate Destination table exists. If not Create the destination table and map the columns as per the configuration file.

4)

Run Pipeline:

a.

Once Step 1-3 is passed, trigger the pipeline as per the schedule or manually.

b.

Any failure should be logged and emailed

[Anonymous]

Hi @umesh Kumar​ this question seems to have been overlooked. I believe it is because you may have posted it on the wrong board. The Pipeline Designer board is for our Pipeline Designer product which wouldn't normally be used for a problem like this.

What you appear to be asking here is how to solve the complete requirement. Questions like these don't tend to be picked up very quickly since everyone who posts here has their own requirements to focus on. The best way to get a response here is to try to solve your requirement, then post specific questions about where you get stuck, including as much information as to what you have already done. Lots of small questions are responded to far quicker than posting an end to end requirement.

Maybe you can break down this requirement into smaller questions here and we can tackle each sub question at a time

Regards

Richard