topic Re: Integrate data from many csv files in Talend Studio

Integrate data from many csv files

Anonymous — Sat, 16 Nov 2024 13:30:33 GMT

Hi!
I have Campaign A, Campaign B and Campaign C. Each campaign has a set of statistics exported in csv files.
Campaign A: A1,A2,A3 (A1,A2,A3 are names of csv files). I need to integrate data in these 3 files and store in a table X in Mysql DB. After create a job A on JasperETL to do this, table X will have 1 row which store integrated data about campaign A.
Campaign B also has B1,B2,B3 as names of csv files. I need to integrate data in these 3 files and store in 2nd row of table X.
The procedure to integrate campaign B is exactly the same as integrating campaign A.
I am thinking to create a new job B and copy all the components in job A to B. However, doing this, if I have 5 campaigns, I will have to create 5 similar jobs. If I have n campaigns, I will have to create n similar jobs.
How would you suggest me to do with this? Is there any way to automate the process?
Thanks a million!!
Mandy

Re: Integrate data from many csv files

Anonymous — Thu, 11 Mar 2010 13:07:48 GMT

Hi,
your schema from your Campaign A is the same of campaign B or not?
Maybe if it is the same schema you can try with contexts.

Re: Integrate data from many csv files

Anonymous — Tue, 16 Mar 2010 03:15:52 GMT

Hi,
My campaign A and campaign B have exactly the same schema.
Currently, I created a job for campaign A. I imported 3 csv files which store data about campaign A. I integrated these 3 files and export to a table X in my database.
Now I would like to integrate 3 files about campaign B and insert the data into table X. The process of integration is the same, the schema is the same.
Can you explain more how can I use context to solve my issue?
And...Is there anyway that TOS can pull data from csv files in a folder located locally in my pc and import to metadata?
Thanks for your time!
Mandy