Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello Community,
I am new to Talend and have seen and worked through some online tutorials and consulted the Knowledge base (like https://help.talend.com/r/mjoDghHoMPI0yuyZ83a13Q/DIazoon9cl1kgT_36u~MDQ). But I have a fundamental understanding problem.
I want to use Talend mostly as an ETL tool conducting the following steps:
Extract data from a say Oracle/mysql/db2, save it onto an staging area on a mariadb/mysql.
Transform the data mainly on the staging area of the database itself and avoid pulling the data from the database to the talend server/instance.
Loading it to the final schema/DB in mysql
I am struggling with the second step. I am aware of the tETLMysqlInput/Map/output components but they cannot be connected to components for aggregation, ordering, filtering (some filtering can be done in the mapper of course). Am I doing something wrong, or is there a general setting so most components work on the DB instead of the talend instance?
I want to avoid pulling data to the talend server and working on it in Java most of the time, or at the end of a job.
I am used working with the SPSS Modeler which was quite capable of this workflow -- but he managed the orchestration of working on the DB or pulling it to the server in the background.
Thanks for sharing your insights.
Clemens
Hi,
There are ELT components that allow you to perform the actions that you discuss. The Transformation is performed at the database end so the data does go on a round trip from staging DB -> Talend Server -> final DB
Thanks
David