Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi
Can some one help me on below query?
I have some 100 files, i want to load one file and compare it to other and save the output.
For example: first i want to load day 1 file compare it to day 2 file save the missing data in ouput.csv
next load day 2 file compare it to day3 file save the missing data in ouput.csv
like this it should keep on run in single load ...
Files are distinguished with date stamp.
below is the example which im trying out.
table:
load user_id,
status
from day1.csv
left join (table)
load user_id,
status,
'y' as flag
from day2.csv
output:
Load user_id
if(flag='y',status,'excluded') as status
resident table;
like above it should run automtically for all the days, for loop we have to use
Thanks,
Please adapt according your needs.
let i = '2020-03-18';
let j = '2020-03-19';
For x=1 to 2 //with 3 Files you have 2 loops
i= date(i,'YYYY-MM-DD')
j = date(j,'YYYY-MM-DD')
Map:
Mapping LOAD B,A
FROM
[all_transfer_event_users-$(i)*.xlsx]
(ooxml, embedded labels, table is Tabelle1);
tmp:
LOAD Distinct A, B
FROM
[all_transfer_event_users-$(j)*.xlsx]
(ooxml, embedded labels, table is Tabelle1) where ApplyMap('Map',B,0)=0;
store tmp into Differences_Table_$(j)_$(i).csv(txt);drop table tmp;
i = i+1;
j = j+1;
next x;
Hi Arlsan,
i want to compare the files and save the difference of those two files
like day 1 and day2 then day2 and day3 then day3 and day4 in a single run.
May i know how incremental load helps here?
can you pls provide some examples? for my understanding
Thanks,
If you could possibly share a sample of your data or code you've written , will be easier to understand .
thanks
updated the post please chekc
Have a look at the attached sample
hope this helps
Hi Frank,
Thank you so this works fine. file name is not day1 day2 i gave that for example
Actually the real file name is all_transfer_event_users-2020-03-18-19-26-41.csv like this with date stamp, so in this case how below loop will work? can you please help me on this?
j=$(j)+1;
i=$(i)+1;
next x;
Please adapt according your needs.
let i = '2020-03-18';
let j = '2020-03-19';
For x=1 to 2 //with 3 Files you have 2 loops
i= date(i,'YYYY-MM-DD')
j = date(j,'YYYY-MM-DD')
Map:
Mapping LOAD B,A
FROM
[all_transfer_event_users-$(i)*.xlsx]
(ooxml, embedded labels, table is Tabelle1);
tmp:
LOAD Distinct A, B
FROM
[all_transfer_event_users-$(j)*.xlsx]
(ooxml, embedded labels, table is Tabelle1) where ApplyMap('Map',B,0)=0;
store tmp into Differences_Table_$(j)_$(i).csv(txt);drop table tmp;
i = i+1;
j = j+1;
next x;