It's difficult to say without seeing your requirement, data structure, and volume; however, I'd say your starting point is to simply join your data using either tMap or tJoin.
Hi, requirement is something like below- Input file- file0.dat 15 lookup files- file1.dat,file2.dat....file15.dat lookup key is same for all lookups, say process_id now I am fetching certain columns from each lookup file. 1) If I add input file as main flow and rest all files as lookup on a single tMap, there is a big performance hit. 2) Another way I could think is, perform these lookups on multiple tMaps instead of single tMap. (not a very good option though) So I am looking around the other possible ways to perform this. Thanks in advance.
I still think it comes down to your data volume and also your expectation.
The simple answer is to join them in a single tMap or try tJoin.
If one or more look-up files is causing an unacceptable performance issue, then you can look for specific ways to improve this.
