Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
alexandernatale
Creator II
Creator II

Incremental stored. A practical case

Good morning everyone,

I was trying to create an incremental data load on qliksense saas. It would seem that the script works but I would like to be sure that I understand the mechanism correctly.

This is the script...

LIB CONNECT TO 'Earthquake INGV';
 
RestConnectorMasterTable:
SQL SELECT 
"#EventID",
"Time",
"Latitude",
"Longitude",
"Depth/Km",
"Author",
"Catalog",
"Contributor",
"ContributorID",
"MagType",
"Magnitude",
"MagAuthor",
"EventLocationName",
"EventType"
FROM CSV (header on, delimiter "|", quote """") "CSV_source";
 
[ingv]:
LOAD [#EventID] AS Id,
[Time],
[Latitude],
[Longitude],
[Depth/Km],
[Author],
[Catalog],
[Contributor],
[ContributorID],
[MagType],
[Magnitude],
[MagAuthor],
[EventLocationName],
[EventType]
RESIDENT RestConnectorMasterTable;
 
DROP TABLE RestConnectorMasterTable;

 

[ingv]:
LOAD * FROM [lib://Google_Drive - xxxxxxxxxxxxxxx@gmail.com/root/ingv.qvd] (qvd)
WHERE NOT EXISTS (Id);
store ingv into [lib://Google_Drive - xxxxxxxxxxxxxxx@gmail.com/root/ingv.qvd];


As far as I understand:
1. the rest connector loads a temporary table RestConnectorMasterTable to store the query result;
2. I generate a table on qliksense called ingv where I'm going to store the result of the query; the RestConnectorMasterTable table is deleted;
3. upload to ingv all the fields present in the .qvd file whose id does not already exist in the ingv table;
4. I store the "final" ingv table in the same .qvd file overwriting the results.

It's correct?

Labels (5)
1 Solution

Accepted Solutions
Benoit_C
Support
Support

Hello @alexandernatale,

 

Yes you perfectly understood the script.

 

This is a typical pattern for incremental data loads in Qlik Sense, where you load new data from a source, compare it with the existing data, and add only the new records. By doing this, you can reduce the time and resources required for data loads and ensure that your Qlik Sense application has the latest data available.

 

Regards,

Benoit

View solution in original post

1 Reply
Benoit_C
Support
Support

Hello @alexandernatale,

 

Yes you perfectly understood the script.

 

This is a typical pattern for incremental data loads in Qlik Sense, where you load new data from a source, compare it with the existing data, and add only the new records. By doing this, you can reduce the time and resources required for data loads and ensure that your Qlik Sense application has the latest data available.

 

Regards,

Benoit