Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
hic
Former Employee
Former Employee

A QlikView feature that is poorly known and brilliant in its simplicity is the Preceding Load.

 

If you don’t know what it is, then I strongly suggest that you read this blog post and find out. Because it will help you in your QlikView scripting.

 

So what is it?

 

It is a way for you to define successive transformations and filters so that you can load a table in one pass but still have several transformation steps. Basically it is a Load statement that loads from the Load/SELECT statement below.

 

Example: you have a database where your dates are stored as strings and you want to use the QlikView date functions to interpret the strings. But the QlikView date functions are not available in the SELECT statement. The solution is to put a Load statement in front of the SELECT statement: (Note the absence of “From” or “Resident”.)

 

Load Date#(OrderDate,’YYYYMMDD’) as OrderDate;
SQL SELECT OrderDate FROM … ;

 

What happens then is that the SELECT statement is evaluated first, and the result is piped into the Load statement that does the date interpretation. The fact that the SELECT statement is evaluated before the Load, is at first glance confusing, but it is not so strange. If you read a Preceding Load as

 

     Load From ( Select From ( DB_TABLE ) )

 

then it becomes clearer. Compare it with nested functions: How would you evaluate “Round( Exp( x ) )”. You would of course evaluate the Exp() function first and then the Round() function. That is, you evaluate it from right to left.

 

Input - Output.png

 

The reason is that the Exp() function is closest to the source data, and therefore should be evaluated first. It’s the same with the Preceding Load: The SELECT is closest to the source data and should therefore be evaluated first. In both cases, you can look at it as a transformation that has an input and an output and to do it correctly, you need to start with the part of the transformation closest to the input.

 

Any number of Loads can be “nested” this way. QlikView will start from the bottom and pipe record by record to the closest preceding Load, then to the next, etc. And it is almost always faster than running a second pass through the same table.

 

With preceding Load, you don’t need to have the same calculation in several places. For instance, instead of writing

 

Load  ... ,
   Age( FromDate + IterNo() – 1, BirthDate ) as Age,
   Date( FromDate + IterNo() – 1 ) as ReferenceDate
   Resident Policies
      While IterNo() <= ToDate - FromDate + 1 ;

 

where the same calculation is made for both Age and ReferenceDate, I would in real life define my ReferenceDate only once and then use it in the Age function in a Preceding Load:

 

Load  ..., ReferenceDate,
   Age( ReferenceDate, BirthDate ) as Age;
Load  *,
   Date( FromDate + IterNo() – 1 ) as ReferenceDate
   Resident Policies
      While IterNo() <= ToDate - FromDate + 1 ;

 

The Preceding Load has no disadvantages. Use it. You’ll love it.

 

HIC

96 Comments
Anonymous
Not applicable

Thanks Henric..That explains it well

0 Likes
421 Views
Anonymous
Not applicable

Thank you

0 Likes
421 Views
carlcimino
Creator II
Creator II

Maybe I missed this in the string but how does one aggregate/summarize/group by in a preceding load?  I understand the transformation of the data in the preceding load but can't seem to summarize after the transformation...  See script below.  Working from the bottom up the first preceding load where I perform my transformations works but when I add the second preceding load to summarize it fails and says Error: Invalid expression.

ODBC CONNECT TO BAR (UserId is , Password is );
LOSSES:
Load
CAT_LOSS,
CAT_DESC,
count(distinct(CLM_FILE_ID)) as CLAIM_COUNT,
sum(GROSS_REPT_L_ALAE) as GROSS_REPT_L_ALAE,
text(CAL_DATE) as CAL_DATE,
ACCT_D,
LEGACY_COMPANY,
LOB,
DATATYPE,
ACE_PROD_NUMBER,
PRODUCER_NUM_7_DIGIT,
SUB_PRODUCER_NUM,
STATE,
PERIL
;

LOAD
If(trim(len(CAT_ID))=0,'N','Y') as CAT_LOSS,
If(trim(len(CAT_ID))=0,'Non-Cat Loss',CAT_DSC) as CAT_DESC,
CLM_FILE_ID,
INCU_INDM_US_CURY_AMT+INCU_ALAE_US_CURY_AMT as GROSS_REPT_L_ALAE,
date(ACTG_MO&'/'&'01'&'/'&ACTG_YR) as CAL_DATE,
ACTG_YR_MO as ACCT_D,
MCC_DSC,
if(MCC_DSC like '*Fund*','Fund',if(match(MCC_DSC,'Boats','Yachts','Other Recreational Marine','Captives'),'ACE',if(MCC_DSC like '*ACE*','ACE','Chubb'))) as LEGACY_COMPANY,
if(match(MCC_DSC,'CHB Pr Lns Can Homeowners','CHB Pr Lns Homeowners','ACE PRS Firemans Fund Homeownr','ACE Private Rsk Svcs Homeowner'),'Home',
if(match(MCC_DSC,'CHB Pr Lns Can Personal Auto','CHB Pr Lns Pers Auto Phys Dmg','CHB Pr Lns Pers Auto Liability','ACE PRS Firemans Fund Auto', 'ACE Private Risk Services Auto'),'Auto',
if(match(MCC_DSC,'CHB Pr Lns Can Floater','CHB Pr Lns Floater','ACE PRS Firemans Fund Inland','ACE Private Risk Svcs Inland'),'Valuables',
if(match(MCC_DSC,'CHB Pr Lns Can Personal Excess','CHB Pr Lns Pers Excess','ACE PRS Firemans Fund Umbrella','ACE Private Risk Svcs Umbrella'), 'Excess',
if(match(MCC_DSC,'Other Recreational Marine','Boats','Yachts','CHB Pr Lns Can Yachts','CHB Pr Lns Yachts'),'Watercraft','Other'))))) as LOB,
if(MCC_DSC like 'ACE*','ACE-Fund BAR Losses',if(wildmatch(MCC_DSC,'Boats','Yachts','Other Recreational Marine','Captives'),'ACE-Fund BAR Losses','Chubb BAR Losses')) as DATATYPE,
PROD_ID as ACE_PROD_NUMBER,
LEGACY_MAIN_CODE as PRODUCER_NUM_7_DIGIT,
LEGACY_SUB_CODE as SUB_PRODUCER_NUM,
TAX_PLTDV_DSC as STATE,
PERL_DSC as PERIL

;

SQL SELECT
d.MCC_DSC,
a.ACTG_MO,
a.ACTG_YR,
a.ACTG_YR_MO,
b.PROD_ID,
a.INCU_INDM_US_CURY_AMT,
a.INCU_ALAE_US_CURY_AMT,
a.CLM_FILE_ID,
c.LEGACY_MAIN_CODE,
c.LEGACY_SUB_CODE,
a.CAT_ID,
a.CAT_DSC,
b.TAX_PLTDV_DSC,
b.PERL_DSC
FROM ATLAS_NA_DM_PROD..NON_STRTGC_CLM_SUM_AYM a join ATLAS_NA_DM_PROD..NON_STRTGC_CONTR_SUM_AMT b on
a.non_strtgc_contr_sum_amt_sk = b.non_strtgc_contr_sum_amt_sk
and b.cur_ind = 'Y'
inner join ATLAS_NA_DM_LNDG_PROD.ADMIN.PAS_PROD_XREF_LNDG c on
b.PROD_ID = c.PRODUCER_CODE
and c.MARKETING_OFFICE_CODE='PRS'
left join ATLAS_NA_DM_PROD..DIM_MCC d on
a.SRCE_RPTG_MCC_CD = d.MCC_CD
WHERE
(a.ACTG_YR = '2012' OR a.ACTG_YR = '2013' OR a.ACTG_YR = '2014' OR a.ACTG_YR = '2015' OR a.ACTG_YR = '2016' OR a.ACTG_YR = '2017')
AND
a.SRCE_RPTG_MCC_CD in ('0090800', '0090810', '0090820', '0090830', '0090899',
'0091000', '0091010', '0091020', '0091030', '0091090',
'0091100', '0091101', '0091110', '0091130', '0091140',
'0091190', '0121007', '0210010', '0210018', '0210020',
'3200318', '3200319', '3200320', '3200321', '3200323','0210040','0210050') 
AND (a.INS_CD = 'DIR' OR a.INS_CD = 'ASM');
 
store LOSSES INTO C:\QlikView\Scorecard\v7\LOSSEStest.qvd (qvd);

0 Likes
409 Views
chriscammers
Partner - Specialist
Partner - Specialist

You have to add a "group by" clause to the preceeding load statement  where you are using aggregate functions. Be sure to include all the column expressions that are not aggrgates in your group by list.

0 Likes
409 Views
carlcimino
Creator II
Creator II

So that part of the preceding load is similar to SQL.  Could I accomplish the transformation and grouping by in 1 preceding load?  Is there a benefit to keeping it separate?

0 Likes
409 Views
chriscammers
Partner - Specialist
Partner - Specialist

We have observed that preceding loads do tend to extend the load time as they behave like separate load statements running in sequence so clearly there is an argument for combining them into one. I find that if I have a series of calculated fields like what you have in your sample it is usually easier to follow if I do all the transformation operations and then the aggregation step.

The long and short of it is based on personal preference and performance, if thing are not performing well then you may need to try a different approach.

409 Views
anderseriksson
Partner - Specialist
Partner - Specialist

It is similar to any other load-statement in the script, only there is no source (FROM or RESIDENT) .
You can use WHERE and GROUP BY, probably also ORDER BY.

A problem with preceding loads is they are not as easy to read.
Have also noticed at times that preceding load seems to take more time to execute
then if using a temporary table and doing a RESIDENT load from that.

I would not attempt doing the transformation and grouping in the same LOAD statement.
Also I would separate loading from database and transformation of data.
Ever heard of ETL?
Your code you can't just run the transformation and grouping,
you always have to fetch the data from database also.
Separating those steps you can do the transformation over again without having to fetch new data from the database.

0 Likes
409 Views
Not applicable

great!

0 Likes
409 Views
markp201
Creator III
Creator III

More than once I see a preceding load which simply regurgitates the field names from the select. Typically its the result of using the wizard.  The only reason I use preceding is to gain access to functionality only available in Qlik (e.g. previous).  It's double-work when you have to change the select and remember to also change the preceding load.  It would seem to faster and easier to rename the fields in the sql rather then in the preceding load.  Is there a best method?

0 Likes
426 Views
hic
Former Employee
Former Employee

I wish there were a simple answer...

I usually do the same as you - I only use preceding load when needed, e.g. to make successive transformation or to use Qlik specific functionality. Renaming fields in the SELECT statement usually works excellently, so I do that quite often.

However, there are some odd ODBC drivers that are case-insensitive, and this causes problems. For example

   SELECT FIELD_A as NewFieldName FROM ...

will then create a field called NEWFIELDNAME - in upper case. This is extremely annoying, and the work-around is of course to rename the fields in a Preceding Load.

But if you then write

   Load FIELD_A as NewFieldName ;

   SELECT * FROM ...

you will get an inefficient SELECT statement: The star could mean hundreds of fields.

This means that you HAVE TO list all fields in both the Load and the SELECT, to get an efficient load sequence:

   Load FIELD_A as NewFieldName, ... ;

   SELECT FIELD_A, ... FROM ...

which seems like double work.

HIC

426 Views