Batch processing using MongoDb(source) and MySQL(output)

rohail_iqbal — Fri, 15 Nov 2024 21:09:03 GMT

All the records are extracted in a single run from mongoDb first and then get loads into MySQL

What I want to achieve :

I want to implement batch processing technique in my ETL job. First my ETL should check how much records are in the mongoDb then divide those records in batches and perform ETL execution.

The current structure of my job is like this

tMongoDbInput ---> tJavaRow --->tMap ---> tMySQLOutput --->tLogRow

What modifications I would require then

topic Batch processing using MongoDb(source) and MySQL(output) in Talend Studio

Batch processing using MongoDb(source) and MySQL(output)