Skip to main content
Announcements
SYSTEM MAINTENANCE: Thurs., Sept. 19, 1 AM ET, Platform will be unavailable for approx. 60 minutes.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Auto generate TalenD jobs

 Forgive my ignorance up front, I come from a background of using ETL/ELT tools such as Wherescape RED, SSIS and SAS DI Studio.

 

My question is around auto-generation of jobs.

 

SSIS is nothing more than BIML (Business Intelligence Markup Language) represented in a VisualStudio GUI and scheduled on something like Control-M or SQL Server Agent. The jobs do not need to be created by hand --- meaning dragging and dropping transformations and registered data sources and target databases. Instead the jobs (BIML) can be created by coding the BIML and automating the creation of jobs by extending the BIML with BIML Script (using VB .NET  or C#) .

Therefore a single BIML Script can generate hundred of jobs doing similar type work differentiated by parameters. Like loading source CSV files that are the same in structure but different by filename's suffix; example the date included in a filename like product_20180717.csv.

 

Anyway, I was wondering if the same level of automation can be achieved in TalenD? i.e. I write a script that generates TalenD jobs with the components I need.

 

 

Labels (3)
1 Reply
fdenis
Master
Master

Talend is a true ETL who can do a lot of thing around ETL like ftp rest managing file, connection oracle and ms sql………
on the enterprise version you can use dynamic schemas. When you have a lot of similar jobs just make one.
all you jobs (data jobs, orchestration jobs,....) can be called by an other one and use parameters so you can have one complex job called multiple time.

Last thing, it's not the best way but it's working.
Talend is a java generator. so if you still want to generate multiple similar jobs. you can copy and modify java code.

Regards,