topic Re: Management of merged data from different user for same file in Data Quality

Management of merged data from different user for same file

Anonymous — Sat, 16 Nov 2024 09:45:34 GMT

Hi there,

I'm using desktop version of Talend data preparation, in which I'm able to clean few rows fom csv file based on duplicate values. Now, it will be good if I explain my concern with Example :

Morning:

1. The client provide us csv sheet where there are 2000 rows.

2. I extract this file in dataset, and clean some data by applying some functions

e.g. Deleted 4 rows having city = 'ABC',

3. total rows = 1996

4. I save this file to preparation.

5. Either export this file or let it as it is there in preparation.

Evening:

1. The same client sent same csv file with some more additional rows, say 3000, i.e. 1000 rows added to previous 2000 rows.

2. In this file there are total 8 rows with city = 'ABC'

3. I again applied same function to clean data, delete rows with city = ABC, (4 rows I already deleted from previous sheet)

4. Total rows = 2992

5. Export

And the process goes in same manner, which is definitely not an efficient way.

How can we save and automate these rules again and again ?

Is there any solution (tool) provided by Talend for this scenario ?

Re: Management of merged data from different user for same file

Anonymous — Fri, 19 May 2017 12:51:59 GMT

Hi,

What you are looking for is the ability to reapply an existing preparation to another dataset with the same structure, right? If so, that is doable natively in Data Prep - see https://community.talend.com/t5/Data-Quality-and-Preparation/Multiple-Datasets-with-the-same-structure-using-the-same/m-p/21782#U21782

Let me know if that matches your needs.

Regards,

Gwendal

Re: Management of merged data from different user for same file

Anonymous — Fri, 19 May 2017 14:04:00 GMT

Yes, it does.
We want Talend Data Preparation to be use to access Microsoft Azure cloud. So please provide any suggestion or link.

Many thanks for your help