Full Accumulation example

[ArturoMuñoz]

During the last days we have been working in a new visualization piece that will be live soon in The Telegraph website. We got some data with daily refugees arrivals to different countries, the goal was to visualize how many people have moved across Europe during the last few months in a map, but we also wanted to plot a line/area chart with the cumulative number of arrivals per country and per day so you can see trends and evolution over time. Our data set included daily data but it didn't contain cumulative numbers so we had to calculate them.

The calculation should be something like this:

If is the first row or country is different than previous row country, then load the field value, else load the field value plus same field previous row value.

Step 1 Data Load

Start by loading our data table with the daily arrival values by country.

//Load the main table
DataLoad:
LOAD
    "Date",
    Country,
    "Daily Estimated Arrivals"
FROM [lib://LibraryName/Data.qvd] (qvd);

The generated table will look like the picture below:

As you can notice the data is not sorted in a way that will let me successfully calculate the new field as I designed it because it will mix data from different countries, so my next move will be to sort the table by country and date.

Step 2 Sorting the table

//Sorting the table
SortLoad:
NoConcatenate Load
  "Date",
    Country,
    "Daily Estimated Arrivals"
Resident DataLoad
Order by Country, Date;

Drop table DataLoad; //Deletes the previous table no longer needed

Because I’m using a Resident Load statement to load and sort the values from the previous table, and both tables have identical field sets, I need to specify NoConcatenate before the load, otherwise both tables will be automatically concatenated. Using ‘Order By’ clause will sort the table by Country and then by Date. Finally, because the first table DataLoad will be no longer needed I’ll delete it using Drop Table.

My new table will be as in the picture below

Step 3 Adding the calculated new field

Now that we have the data table ready we can finally calculate the new field to the table containing the cumulative daily estimated arrivals. To do so we will use the peek and previous script functions. Both functions are similar but they have some key differences that you should learn, please don’t miss this blog post:  Peek vs Previous: when to use each

//Add cumulative data to the table
CummulativeLoad:
NoConcatenate Load
    "Date",
    Country,
    "Daily Estimated Arrivals",
    If(RowNo()=1 Or Country<>Previous(Country),
          "Daily Estimated Arrivals",
          "Daily Estimated Arrivals" +  Peek("Cumulative Daily Estimated Arrivals", RowNo()-2))
    as "Cumulative Daily Estimated Arrivals"
Resident SortLoad;

Drop table SortLoad; //Deletes the previous table no longer needed

Click on the image below to see how we created the new field to match with our calculation statement.

The final result will be a data table containing the fields to support both daily and cumulative charts.

Enjoy Qliking!

AMZ

(Note: I divided this example in 3 steps so hopefully it is clearer, but step 2 and 3 can be merged in one single step)

[stevedark]

Hi Arturo,

Thanks for sharing these insights.  I find Peek very useful for finding differences between values on two rows, but seldom use it for accumulations.

I did a blog post on accumulating by associating each day with every day prior to it in the data model:

http://www.quickintelligence.co.uk/qlikview-accumulate-values/‌

This same approach can be used for average bars and moving annual totals as well.

Hope it is of interest to others.

Steve

[ArturoMuñoz]

Thanks for sharing Steve, very interesting stuff.

In this particular example peek works great because it will let me target a field that has not been previously loaded in the table and  target a specific row.

For partial accumulations I'd also suggest this post The As-Of Table

[Anonymous]

Nice, Thanks for sharing.

[Gysbert_Wassenaar]

If you use RangeSum instead of + operator then you don't need to check for rowno()=1. You can also let peek use its default -1 offset to look up the previous value instead of calculating an absolute record number.

    If(Country<>Previous(Country), 

          "Daily Estimated Arrivals", 

          RangeSum("Daily Estimated Arrivals" ,Peek("Cumulative Daily Estimated Arrivals"))) 

    as "Cumulative Daily Estimated Arrivals" 

[ArturoMuñoz]

Thanks Gysbert! I like this one, it simplifies the expression removing the rowno()-2 and makes it more descriptive. Thanks for sharing.

[ecolomer]

Good work, very useful.

Thank's for sharing

Enrique Colomer

[fkeuroglian]

     Ass Always! Great work Arturo!

thanks for share

Fernando K.

[christian77]

Very nice! 

The thing is that QlikView is not a cube-based app. Totals are calculated in expressions. And that’s the
good thing. Let's say you have a new filter, besides of Country, Air Company, how would you do that on the script? Many more combination of totals, or accumulative calculations have to be done. Imagine that we have 10 filters. That's why the best thing to do, in my opinion, is calculate in design, rather than in the script.

Christian.

SalU2 Arturo.

[robert99]

Is it possible to drill down (say country --> City) using a script approach?

I guess just do the above twice?

I have done it as shown on this thread but is there a different way

NB I never used this approach because drilling down would not be possible it seemed to me

https://community.qlik.com/blogs/qlikviewdesignblog/2015/11/09/accumulations#comment-42554

[ArturoMuñoz]

Christian, design calculations, when available, will usually give you more flexibility and dynamism than Script based calculations that's right. However they will not always be available nor give you the result you were expecting, if you follow @stevedark's link in the first comment you will find that design full accumulation calculation not always works as you want it to do.