Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I would like to ask you how to write an 'Incremental Load" script which will load Hive table data filtered in the increments of seven days, such as:
where `date` = 18/1/2016, 25/1/2016, 1/2/2016.....17/9/2018, 24/9/2018, 1/10/2018, 8/10/2018.........onward
The script for the single date:
BN2:
LOAD Distinct
poi_region_name,
state,
efs_vc_host_name,
Count (Distinct nbn_avc_id) as Unique_AVC_Count,
tc4_11_avc_wo_ob_ds as TC4_download_speed,
tc4_11_avc_wo_ob_us as TC4_upload_speed,
tc1_11_avc_wo_ob_ds as TC1_download_speed,
primary_access_tech,
tc2_11_avc_wo_ob_ds as TC2_download_speed,
tc1_11_avc_wo_ob_us as TC1_upload_speed,
tc2_11_avc_wo_ob_us as TC2_upload_speed ,
Date(Date#("date", 'YYYYMMDD')) as Date
where left (nbn_avc_id,3)='AVC'
Group by poi_region_name,state,efs_vc_host_name,tc4_11_avc_wo_ob_ds,
tc4_11_avc_wo_ob_us,
tc1_11_avc_wo_ob_ds,
primary_access_tech,
tc2_11_avc_wo_ob_ds,
tc1_11_avc_wo_ob_us,
tc2_11_avc_wo_ob_us,
"date";
SQL SELECT
`poi_region_name`,
state,
`nbn_avc_id`,
`tc4_11_avc_wo_ob_ds`,
`tc4_11_avc_wo_ob_us`,
`tc1_11_avc_wo_ob_ds`,
`primary_access_tech`,
`tc2_11_avc_wo_ob_ds`,
`tc1_11_avc_wo_ob_us`,
`tc2_11_avc_wo_ob_us`,
`date`
FROM HIVE.prod.`baja`
where `date`='20180902';
Store BN2 into [lib://QVD/BN2.csv]
(txt,delimiter is ',');
;
I suggest not using the term incremental load as it has a specific usage within qlik.
for your usecase for dynamically creating the where clause. Why dont you create a variable in your script which will can be populated based on your logic and use in your where clause
e.g. where $(vTemp);