Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

sivaboggarapu20
New Contributor III

Create Nightly Load Script

Hi Team,

I have requirement like below

1.I need to Create a load script that can be scheduled in the QMC to run on a nightly basis (no visualizations required) that will look for new Billing Detail files ($AppImportPath) and load them if they are new (have not yest been loaded) via QVD files.


2.The script should only load new data and only update the QVD files that need to be updated.


3.If there are no new Billing files, the script does nothing.


Can any one give me solution for the above requirement


Thanks,

Siva

1 Solution

Accepted Solutions
omarbensalem
Esteemed Contributor

Re: Create Nightly Load Script

1) Let's assume you have a Qvd file : Table.qvd That contains all the previously Loaded Data.

2) Let's load the new data:

MyTable:

Load * from source;

Let vNoRows=NoOfRows('MyTable');


//Load from the QVD only the lines that exists in the New source

Qvd:

Load * from Qvd where exists(ID);

Let vExists=Peek(ID',$(vNoRows)+1,'MyTable');

Let vExistsLen=Len(trim(Peek(ID',$(vNoRows)+1,'MyTable')));

//Now, if vNoRows is filled => that means that the new source contains data that exists in the QVD;

//if it's null => the new data does not exist in the QVD: new data !

if $(vExists)='NULL' or  $(vExistsLen)=0 THEN

drop table MyTable;

//Import it another time and store it with the qvd (concatenate)

MyTable:

load * from source;

Qvd:

load * from Qvd where not exists(ID) ; //with update

//or load * from Qvd; insert without update


store MyTable into qvd path..


else // we don't do anything new, we just import the data in the qvd

drop table MyTable;

load * from QVD;

exit script

3 Replies
omarbensalem
Esteemed Contributor

Re: Create Nightly Load Script

1) Let's assume you have a Qvd file : Table.qvd That contains all the previously Loaded Data.

2) Let's load the new data:

MyTable:

Load * from source;

Let vNoRows=NoOfRows('MyTable');


//Load from the QVD only the lines that exists in the New source

Qvd:

Load * from Qvd where exists(ID);

Let vExists=Peek(ID',$(vNoRows)+1,'MyTable');

Let vExistsLen=Len(trim(Peek(ID',$(vNoRows)+1,'MyTable')));

//Now, if vNoRows is filled => that means that the new source contains data that exists in the QVD;

//if it's null => the new data does not exist in the QVD: new data !

if $(vExists)='NULL' or  $(vExistsLen)=0 THEN

drop table MyTable;

//Import it another time and store it with the qvd (concatenate)

MyTable:

load * from source;

Qvd:

load * from Qvd where not exists(ID) ; //with update

//or load * from Qvd; insert without update


store MyTable into qvd path..


else // we don't do anything new, we just import the data in the qvd

drop table MyTable;

load * from QVD;

exit script

sivaboggarapu20
New Contributor III

Re: Create Nightly Load Script

Hi Omar,

Can you share me your personal email id.

I would like to share my QVF file

Thanks,

Siva

sivaboggarapu20
New Contributor III

Re: Create Nightly Load Script

Hi Omar,

Here, I am attached my QVF file. Could you please look in it and let me know if any suggestions required?

Thanks,

Siva