Skip to main content
Announcements
Document boards are being consolidated, this board no longer allows NEW documents READ MORE

The Fastest Dynamic Calendar Script (Ever)

cancel
Showing results for 
Search instead for 
Did you mean: 
tseebach
Luminary Alumni
Luminary Alumni

The Fastest Dynamic Calendar Script (Ever)

Last Update:

Jun 25, 2019 6:28:51 AM

Updated By:

tseebach

Created date:

May 22, 2014 7:42:46 AM

Attachments

Me and a colleague have put together a new Calendar, combining a super fast method for generating the needed calendar, while also removing a lot configuration options in the previous methods.

 

So if you're using a  Min/Max based approach like the one described here: Creating A Master Calendar

 

Then you're properly used to waiting for the Min/Max to find the largest values in your dataset. This can take minutes. What we have here takes less than a second. I've actually not been able to time this, because this calendar script takes less than a 1 seconds even on billions of rows on decent hardware. The only exception to this is if you use a date range starting before christ and spans a thousand years. If you have a date range, with a lot of holes in it (dates with no data), then you should use the last parameter:

Call Calendar('[Date (NK)]','MyCal','Pre.','','true');

The Attached qvw shows both methods (remember the qvs file).

I've not commented it, because most people will not grasp what is going on anyway, and don't need to

To try it out, include the following in your script, and add this to your code:

Call Calendar('DateField');

/* ************************************************************************************

itelligence Standard Qlik Calender Torben Seebach/Martin Didriksen Special Thanks to Rob Wunderlich and John Witherspoon

Configure Calender parameters

  Syntax:

       Calendar(DateField[,CalendarName][,Prefix[,Suffix[,FullCalendar]]])

  Example:

       Call Calendar('Date (NK)','MyCal','Pre.','','true'); // creates a the MyCal table and fields are prefixed with Pre. In the superfast variat

       Call Calendar('Date (NK)'); //creates a table called calendar based of "Date (NK)" field

  Mandatory:

       @_DateField Contains the name of the field to connect the calendar to

  Optional:

       @_CalendarName Contains the name of the calendar we create      
       @_CalendarPrefix A Prefix for all fields
       @_CalendarSuffix A Suffix for all fields      
       @_FullCalendar If this contains a value then it creates a calendar with all dates from Min(Date) to Max(Date), default is the slowest option

************************************************************************************ */

Sub Calendar (_DateField,_CalendarName,_CalendarPrefix,_CalendarSuffix,_FullCalendar)

  Let _StartTime = Now(); 
  Let _CalendarName = If(Len('$(_CalendarName)')=0,'Calendar','$(_CalendarName)');
  Let _CalendarPrefix = If(Len('$(_CalendarPrefix)')=0,'','$(_CalendarPrefix)');
  Let _CalendarSuffix = If(Len('$(_CalendarSuffix)')=0,'','$(_CalendarSuffix)');
  Let _FullCalendar = If(Len('$(_FullCalendar)')=0,1,0);
  Let _DateField = PurgeChar(_DateField,'"[]');

  "$(_CalendarName)":
   LOAD
        Distinct [$(_DateField)] as [$(_DateField)],
        Text(Date([$(_DateField)])) as [$(_CalendarPrefix)DateText$(_CalendarSuffix)],
        Year([$(_DateField)]) as [$(_CalendarPrefix)Year$(_CalendarSuffix)],
        Week([$(_DateField)])&'-'&Year([$(_DateField)]) as [$(_CalendarPrefix)WeekYear$(_CalendarSuffix)],
        Week([$(_DateField)]) as [$(_CalendarPrefix)Week$(_CalendarSuffix)],
        Month([$(_DateField)]) as [$(_CalendarPrefix)Month$(_CalendarSuffix)],
        Year([$(_DateField)])&'-'&Text(Date([$(_DateField)],'MM')) as [$(_CalendarPrefix)YearMonth$(_CalendarSuffix)],         'Q'&Ceil(Month([$(_DateField)])/3) as [$(_CalendarPrefix)Quarter$(_CalendarSuffix)],
        AutoNumber(MonthStart([$(_DateField)]),'_MonthSerial') as [$(_CalendarPrefix)MonthSerial$(_CalendarSuffix)],
        AutoNumber(QuarterStart([$(_DateField)]),'_QuarterSerial') as [$(_CalendarPrefix)QuarterSerial$(_CalendarSuffix)],

        AutoNumber(weekyear([$(_DateField)]) &'|'&week([$(_DateField)]),'_WeekSerial') as [$(_CalendarPrefix)WeekSerial$(_CalendarSuffix)] ;

  If _FullCalendar=1 Then
        LOAD Date(_DateStart+(Iterno()-1),'$(DateFormat)' ) as [$(_DateField)]
               While (_DateStart+(Iterno()-1)<=_DateStop);
        LOAD
               Floor(Min(Fieldvalue('$(_DateField)',RecNo()))) as _DateStart,
               Floor(Max(Fieldvalue('$(_DateField)',RecNo()))) as _DateStop
        AUTOGENERATE FieldValueCount('$(_DateField)');
Else
        LOAD Num(Fieldvalue('$(_DateField)',RecNo())) as [$(_DateField)]
        AUTOGENERATE FieldValueCount('$(_DateField)');
End If

  /* clean variables  */

Let _TotalTime = Round((Now()-_StartTime)*60*60*24,0.00000000001);
Let _StartTime = NULL;
Let _vDateStart = NULL;
Let _vDateStop = NULL;
Trace $(_CalendarName) created in: $(_TotalTime) seconds;
Let _TotalTime = NULL;

ENDSUB

Updated 2014-05-23:

  • Implement preceding load method suggested by Rob Wunderlich
  • Fixed Timer to show seconds
  • Corrected error in documentation
  • Added PurgeChar so [Date A],"Date B" also works.
  • Thanks to Stephen Redmond for pointing to a boolean error in line 22

Other than my brilliant Collegue Martin Didriksen, I also want to give John Witherspoon some credit for inspiration on the autogerate method used. I think he was the first who posted it online.

Tags (3)
Comments
stephencredmond
Luminary Alumni
Luminary Alumni
0 Likes
blaise
Partner - Specialist
Partner - Specialist

Yes, the offset parameter in weekstart is good but i would rather have a standard parameter that worked for all week*() functions

0 Likes
tseebach
Luminary Alumni
Luminary Alumni

Yes, I really would like the have better control of when 52/53 is generated.

0 Likes
stephencredmond
Luminary Alumni
Luminary Alumni

The easiest way to control what week / month / period / fy that a particular date falls into is to have the finance guys maintain it in an Excel spreadsheet.  Fully flexible and handles when they decide to change their accounting rules (as I have had with a client recently)!

Finance guys love maintaining spreadsheets 😉

Anonymous
Not applicable

Good process but not the fastest.

My QVD generation (java 1.8) generates creates any Master Calendar in less time and memory used.

0 Likes
tseebach
Luminary Alumni
Luminary Alumni

Antonio, can you elaborate on your comment? And how are you making sure that you're calendar has the same date ranges as the application?

0 Likes
blaise
Partner - Specialist
Partner - Specialist

Im also a bit curious as i don't think that generating a qvd file (even if it is created with java) a+ load this file in Qv with some sort of where clause is faster than an autogenerated one.@bestofwest

0 Likes
Anonymous
Not applicable

And If I need (hypothetically) I can generate 1000 calendars with same process

with different periods and without Threading.

A Real calendar must respects The Julian Calendar - http://en.wikipedia.org/wiki/Julian_calendar

Of course if You want pure speed use C++.

Remember that Qlik View is Visual C++ written so never a process inside Qlik View can

fasted that one compiled in pure C++  / Java.

0 Likes
blaise
Partner - Specialist
Partner - Specialist

Well we're not talking about generating a calendar the fastest way, we're talking about generating a calender to be used in Qv. Even if you generate a superfast calendar and save it in whatever file format it still needs to be loaded in Qv.

0 Likes
Anonymous
Not applicable

The title of discussion isn't Fastest ... (EVER) ?

And QV reads ALWAYS fastest from Optimize QVD as You know.

0 Likes
Contributors
Version history
Last update:
‎2019-06-25 06:28 AM
Updated by: