Process Job on DB not Talend instance

[CFM90]

Hello Community,

I am new to Talend and have seen and worked through some online tutorials and consulted the Knowledge base (like https://help.talend.com/r/mjoDghHoMPI0yuyZ83a13Q/DIazoon9cl1kgT_36u~MDQ). But I have a fundamental understanding problem.

I want to use Talend mostly as an ETL tool conducting the following steps:

1. Extract data from a say Oracle/mysql/db2, save it onto an staging area on a mariadb/mysql.

2. Transform the data mainly on the staging area of the database itself and avoid pulling the data from the database to the talend server/instance.

3. Loading it to the final schema/DB in mysql

I am struggling with the second step. I am aware of the tETLMysqlInput/Map/output components but they cannot be connected to components for aggregation, ordering, filtering (some filtering can be done in the mapper of course). Am I doing something wrong, or is there a general setting so most components work on the DB instead of the talend instance?

I want to avoid pulling data to the talend server and working on it in Java most of the time, or at the end of a job.

I am used working with the SPSS Modeler which was quite capable of this workflow -- but he managed the orchestration of working on the DB or pulling it to the server in the background.

Thanks for sharing your insights.

Clemens

[David_Beaty]

Hi,

There are ELT components that allow you to perform the actions that you discuss. The Transformation is performed at the database end so the data does go on a round trip from staging DB -> Talend Server -> final DB

Thanks

David