topic Re: Need to compare file data with db in Talend Studio

Need to compare file data with db

Anonymous — Fri, 30 Nov 2018 06:12:06 GMT

Hello,

I am recently having lots of trouble while comparing data stored in tHash (stored in hash is data from DB) with data from file.

The comparison is always unsuccessful. Please see snapshot "1. job" for job design and "2. tmap_comparison_snashot" for the tmap comparison between file data and db data.

1) In tmap I have used expression key for startdatetime and enddatetime between data from file and db because that is the join condition (see left side of the snapshot).

2) In tmap variabes part, there is an UDF "ConvertToFloat.floatFormat(row11.value)" used which gets upto 4 decimal places (refer var1 and var3). So, value from both File and DB are upto 4 decimal places.

3) In tmap variables part, I check if "Var.var1 == Var.var3", then based on this boolean condition I am considering result as accepted or rejects.

4) To see file data and db data, please check attached files - "db_details.txt" and "file_details.txt"

5) value - 96.4067 is present in both db and file, same with value - 105.6693 for same startdatetime and enddatetime.

6) Though the values up to decimal places are same for db and file, still the comparison is rejected for all records.

If some one can suggest me how to correct this or better way for comparison., it will be helpful.

Re: Need to compare file data with db

vapukov — Fri, 30 Nov 2018 08:27:04 GMT

try use equals instead of ==

result:

Re: Need to compare file data with db

Anonymous — Tue, 04 Dec 2018 13:29:05 GMT

@vapukov : Thank you for your thoughts. But changing from == to equals did not help.

Also below is the sample output I get. From DB, though there are no value which is 0.0, talend is assuming 0.0. I am not sure of the reason. There are 2 issues that I am currently facing:

1) I am unable to compare data in file with DB

2) I am unable to understand why 0.0 is being picked up for comparison.

index,startdatetime,enddatetime,subgroup,value,test_case,var1,var3
0,2018-11-30 12:00:00,2018-11-30 12:15:00,PF1_DH,116.6160,value in file 116.6160 is not same as in db 0.0,116.6160,0.0
1,2018-11-30 12:15:00,2018-11-30 12:30:00,PF1_DH,115.2323,value in file 115.2323 is not same as in db 0.0,115.2323,0.0
2,2018-11-30 12:30:00,2018-11-30 12:45:00,PF1_DH,115.2796,value in file 115.2796 is not same as in db 0.0,115.2796,0.0
....

Re: Need to compare file data with db

vapukov — Tue, 04 Dec 2018 19:01:25 GMT

Hi,

it hard to guess - what wrong?

good idea - attache as a picture (better then as attachments) all information, including all functions code

you need to add debug elements (like tLogRow) and debug all steps in your subjobs