Hi, I have a requirement wherein I have a table called userparent.This table has user and his immediate parent. This table I use to derive all the children , grandchildren , great grand children etc for every user using recursive sql in SQL Server.
Now I want to migrate this data to postgres.
So, should I do the recursion part however I am doing(using SQL) and migrate the data to postgres OR should I do the recursion using talend components , if Yes, any ideas on how to do it?
If you are doing in Talend, it means you are bringing a larger data set from source to Talend for processing. This means more IO from source, more fetch time, temp space allocation in Talend during recursion etc.
Ideally you should bring the most compact result set from source DB area after processing. It will make sure that the processing time of the later stages in Talend will be much less compared to fetching the entire source result set to Talend.
If the source is not having enough data processing capability (slightly different scenario) and is pumping data at a higher rate, then the ideal way to process them will be by using Talend Bigdata jobs compared to Talend Standard jobs.
