qlik sense binary load long reload time

[deep2021]

Hi,

Qlik sense binary load consumes a long time to reload. It has been observed that it is taking a long time for indexing some of the keys.
Can you please suggest how to reduce the reload time for the binary load?

Thanks

[MartW · Accepted solution]

So a binary load, loads all the data from 1 QS app to another. you can optimilze this to turn searchindex off. this always improves speed.

but it is also good to take a look at the app you load via the binary statement.

if the script from that app is badly written then the binary will also take a long time.

things you can do to improve are:

- reduce key values and autonumber them.

- turn off search index

- try and reduce unused fields in both apps. this will increase speed in general

[deep2021]

Hi,

Thanks for your valuable reply.

can you please suggest how to turn off the search index?

thanks

[MartW]

@deep2021 

set CreateSearchIndexOnReload=0;

[deep2021]

Hi,

After setting the above variable to 0, it is still a long load time . It has been more than an hour, binary load is not yet completed.

Below is a part of script,

set CreateSearchIndexOnReload=0;

drop Field CP_Key From CP_Agg;
drop Fields C.HR.accountId,
P.HR.accountId,
C.asHR.accountId,
P.asHR.accountId;

Autonumber %CP_Key,fullDocPath;

drop table HR_Base;

CP_Agg1:
NoConcatenate
load
[%CP_Key],
//CP_Key,
fullDocPath,
usageType,

sum([CP_Agg.p_#_of_records]) as [CP_Agg.p_#_of_records],
sum([CP_Agg.c_#_of_records]) as [CP_Agg.c_#_of_records] resident CP_Agg
group by usageType,[%CP_Key],fullDocPath;
//CP_Key;
//,usageType;
drop table CP_Agg;

rename table CP_Agg1 to CP_Agg;

CP_Agg_High_tmp:
//$(tmp_statement)
load fullDocPath
,#_of_records as #_of_records
//,sum(#_of_records) as #_of_records
from
[$(v_Lib_Connection_QVDFiles)/Extractor/$(v_Runtime_Version)$(v_Runtime_Partition)/UAggHigh_consumer.qvd](qvd);

CP_Agg_High_Base:
//$(tmp_statement)
load fullDocPath as fullDocPath
//,sum(#_of_records) as C_#_of_records
,#_of_records as C_#_of_records
resident CP_Agg_High_tmp;


drop table CP_Agg_High_tmp;


join(CP_Agg_High_Base)
//$(tmp_statement)
load fullDocPath,#_of_records as P_#_of_records
//load fullDocPath,sum(#_of_records) as P_#_of_records
from [$(v_Lib_Connection_QVDFiles)/Extractor/$(v_Runtime_Version)$(v_Runtime_Partition)/UAggHigh_producer.qvd](qvd)
//FROM [$(v_Lib_Connection_QVDFiles)/Extractor/$(v_Runtime_Version)$(v_Runtime_Partition)/$(v_UAggHigh_FilePattern).qvd](qvd)
//where usageType='Producer'
//group by fullDocPath;
;

CP_Agg_High_tmp:
noconcatenate
load fullDocPath
, sum(C_#_of_records) as CP_Agg_High.C_#_of_records
,sum(P_#_of_records) as CP_Agg_High.P_#_of_records
//, C_#_of_records as CP_Agg_High.C_#_of_records
//,P_#_of_records as CP_Agg_High.P_#_of_records
resident CP_Agg_High_Base
group by fullDocPath;
;

CP_Agg_High:
load * ,
if(CP_Agg_High.C_#_of_records=0
,''
,'C'
) &
if(CP_Agg_High.P_#_of_records=0
,''
,'P'
) as CP_Agg_High.CP_High_Flag
resident CP_Agg_High_tmp;
drop table CP_Agg_High_tmp;
drop table CP_Agg_High_Base;

Exit script;
//*;

Could you please suggest on this?