Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
hic
Former Employee
Former Employee

Last week I wrote about how to create reference dates for exchange rates and warehouse balances: How to populate a sparsely populated field.

 

There is however also a second case where you want to create reference dates, but the data model is quite different. It is when you have a list of contracts with validity periods: Rental contracts, Insurances policies, Healthcare commitments, etc.

 

Each contract has a begin day and an end day. The analysts of an insurance company would probably want to ask the question: “How many valid insurance policies did we have on this specific day?” In other words, he wants to click on a reference date to see the count of policies that are associated with this date – even though this date doesn't exist in the source data.

 

The solution is to first load all policies in one table. Then load a second table that has one record per policy and date (Policies_x_Dates). This second table can in turn link to a master calendar.

 

Data Model.png

 

But the middle table does not exist in the source database. Instead, you must generate it using a While loop that loops over each record in the Policies table, i.e. each source record will be loaded several times. This way, you can load not only the “From” date and the “To” date, but also all dates in between:

 

Policies:
Load PolicyID, BirthDate, PolicyAmount, FromDate, ToDate, OtherPolicyAttribute
          From Policies;

 

Policies_x_Dates:

Load PolicyID,
          Age( FromDate + IterNo() – 1, BirthDate ) as Age,
          PolicyAmount / (ToDate - FromDate + 1) as DailyAmount,
          Date( FromDate + IterNo() – 1 ) as ReferenceDate
          Resident Policies
          While IterNo() <= ToDate - FromDate + 1 ;

 

Note that the Policies table has exactly one record per insurance policy, and the newly created Policies_x_Dates table has exactly one record per combination of policy and date. Note also that there are other fields that should be put in the Policies_x_Dates table, e.g., the age of the insured person, since this depends on the reference date. Further, it is possible to break up a cost or an income into daily amounts, which is useful when you want to show the correct amount distributed over the year.

 

Amount over time.png

 

The While loop is a very useful tool whenever you need to create additional tables in the data model. It is often a better option than the IntervalMatch.

 

Read more in Generating Missing Data In QlikView.

 

HIC

47 Comments
baltazarj
Contributor III
Contributor III

Now I understand.

Thank you, I will try my best to make it.

1,134 Views
patlechat237
Partner - Contributor
Partner - Contributor

Hello  all

i need your help for a similar problem i have a contract table with start and enddate and an invoice table  with invoice date using Hendric's approach to generate reference date for my contract table is working nice

but i have a problem with linking the invoice date from invoice Table with the master calendar since loading invoice dates into my linktable resindent to invoice Table is duplicating my data in invoice Table and sum of Amount from invoice table is giving back the same amount regarless of Month.

ps: sum for value from contract table is working perfect for day , month, year...

can someone help? thank you

760 Views
CJ_Bauder
Partner - Contributor III
Partner - Contributor III

@patlechat237  I am not sure I understand how you are linking the Linktable to the Invoice table - should not be particularly related to the Contracts table.

736 Views
patlechat237
Partner - Contributor
Partner - Contributor

Hello both tables share the same contract id

let me add a sample since you could better understand

thank you

id contract Pos start end Monthprice
1 1000001 10 01.06.2021 31.12.2023 206,95
1 1000001 20 01.02.2022 31.12.2022 33,66
1 1000001 30 01.09.2022 31.12.2023 72,12
2 1000006 10 01.01.2021 31.12.2023 229,46
2 1000006 20 01.11.2022 31.12.2023 5,90
3 1000009 10 01.08.2023 31.12.2025 1.287,07
3 1000009 10 01.08.2023 31.12.2024 1.188,98
3 1000009 10 01.08.2022 31.12.2023 1.097,01
3 1000009 10 01.12.2022 01.01.2024 40,74
3 1000009 20 01.01.2022 02.01.2024 101,04
3 1000009 30 01.02.2019 03.01.2021 5,40
3 1000009 40 01.05.2019 04.01.2022 10,80
3 1000009 50 01.01.2023 05.01.2024 9,18
id invoice no pos invoice date amount
3 451052 10 01.01.2021 934,00
2 447480 10 01.01.2021 219,58
3 446501 10 01.01.2021 40,74
1 451799 10 12.01.2021 179,35
2 447480 10 01.03.2021 219,58
1 453644 10 01.03.2021 179,35
2 455286 10 01.05.2021 219,58
1 457541 10 01.05.2021 179,35
2 460396 10 01.07.2021 219,58
1 460336 10 01.07.2021 179,35
3 460600 10 01.08.2021 934,00
2 460396 10 01.08.2021 219,58
1 462516 10 01.08.2021 179,35
3 446501 10 01.08.2021 40,74
2 465358 10 01.12.2021 219,58
1 468900 10 01.12.2021 179,35
2 470291 10 01.02.2022 202,16
1 473223 10 01.02.2022 188,14
3 476336 10 01.04.2022 1.013,00
2 476529 10 01.04.2022 219,58
1 476460 10 01.04.2022 188,14
3 468957 10 01.04.2022 40,74
2 476529 10 01.06.2022 219,58
1 480266 10 01.06.2022 188,14
2 481912 10 01.08.2022 219,58
1 484065 10 01.08.2022 188,14
3 487983 10 01.12.2022 1.013,00
2 487149 10 01.12.2022 219,58
1 490916 10 01.12.2022 188,14
3 491361 10 01.12.2022 40,74
3 494496 10 01.01.2023 1.097,00
2 492747 10 01.01.2023 229,46
1 492688 10 01.01.2023 206,95
3 491361 10 01.01.2023 40,74
3 494496 10 01.02.2023 1.097,00
2 492747 10 01.02.2023 229,46
1 496014 10 01.02.2023 206,95
698 Views
fnfr
Contributor
Contributor

I have a complex issue to deal with, I have two tables, where each record represent an event in a system. 

The first table has all events that represent a new entry for detailing information of a field (cod) used in the other table, these events have multiple fields that later on are used as criteria for calculation.

The second table has other events that use the field (cod) to associate the values of this table with the details that are in the other table, so far so good, but, the first table can have multiple entries for the same cod, and also there is a field for start date and end period of validity of the events in the first table.

What I need to do is, I need to search the first table based on some criteria:

1. events in the first table must have been included before the events was created in the second table, so here is the first datetime to compare between the tables (datetime the event was created).

2. period of the events in the second table must be between the start and end date of the event in the first table (period is a YYYY-MM format)

3. the end period in the first table can be null, so there are multiple scenarios where more than one event is applicable, in those cases I need to retrieve the latest one considering criteria 1

I have tried using IntervalMatch, but I endup with duplicates when joining the two tables, because of the events that get attached to more than one cod. 

 

 

615 Views
CJ_Bauder
Partner - Contributor III
Partner - Contributor III

@fnfr Hey! I realize this will be a pretty generic answer, but this is a tough issue that will take some time. This is going to require multiple steps of transformation and prioritization of that transformation.

As a start, you'll need to clean table one to either purge the duplicate rows or house the unique data per unique cod in a separate table.

You'll also need to replace Null() end dates with today() or a similar representation of current time. 

 

Lastly, both dates may need to be joined to calendars if you have heavy complications in comparison criteria. This is usually avoided via interval match but IM does not appear usable here.

 

Another comparison you can use is simple if()s on dates - though this is heavy and inefficient, it may also help to remove some confusion initially. 

573 Views
fnfr
Contributor
Contributor

@CJ_Bauder  I hear you, I have tried multiple ways of achieving this, I got close, some
cases I got 99% precision, but than there are others where I got 72%.

It is really tricky depending on how are the entries in the first table, I
guess I will just try another tool or programming language.

Thank you anyway for the response.

554 Views