Hello All, We have a requirement to read data from multiple json files stored at a particular location and transform all data and finally insert it to a postgre db. I am using Talend tFileInputJSON component to read json file, but i can read only 1 JSON file. Since i am fairly new to Talend, looking for experts opinion here on how to tackle this situation. Any support appriciated. Thanks, Manesh
Welcome to Talend! follow the instructions to get your job done. Your job design should be like this. tFileList--iterate--tFileinputJson---tMap--youroutput.
Add tfileList component, set the directory path where all files stored. Add file mask as per your file names default with "*.*". Connect tFileList to tFileInputJson with "iterate" flow. you will have " file name with path" in tFileList global variables. add this global variable to tFileInputJosn file path text box.
global variable which store the file name with path.
Thanks for your help Umesh.. Multiple file parsing Worked perfectly !
One question, can tMap hold data from all the json files before inserting into db or do i need to insert one by one ?
Thanks,
Manesh
Thanks Umesh and Holga for your inputs. I will try to follow it. Basically my JSON has few nested loops and i need to put some custom logic before inserting in database,so converting into csv may not help my purpose.
Below is my sample json file with very few fields,
{"prod_code":"2568945","default_sku_cd":"010101001","sku_list":,"color_details_list":,"size_details_list":,"length_details_list":,"price":"12.00",}
So here I want to achieve 2 things,
1st Job : read each file like above and Insert sku_list row by row into db, but before that i need to map fields like color_code in color_details_list array to get respective color name or if any field is null then replace by "-" before inserting record.
2nd Job : Insert level 1 fields (ex. prod_code,default_sku_cd,price etc) but before that loop through sku_list for few details (ex. sku_price,color_code,color_name) for sku_code matching with default_sku_cd and then add all these details as a single row.
Since i am fairly new to Talend, unable to decide what approach to take to find solution for above problems. Any directions, help is highly appreciated.