1 2 3 4 Previous Next

Qlik Design Blog

56 Posts authored by: Michael Tarallo

cloud blog 2.pngHey Guys! Thanks for taking a moment to read this blog and view the brief 60 second video on our Qlik Web Connectors. It is no surprise that we are surrounded by mountains of data. But, what good is it if you don't have easy access to it? I'm not talking about data that sits in traditional files and databases. I'm talking about data that sits in numerous cloud-based services, social media and web sites.Depending on your business, this data can contain valuable insights that help you make better decisions as wells as customer sentiments that help improve your products and services.


Our Qlik Web Connectors is the answer to your additional data needs, whether its from a cloud-based CRM, data warehouse or social media application. Watch the below video to learn how simple it is to fetch this data and bring it directly into your Qlik Sense or QlikView environments. And...Qlik Sense Cloud Business subscribers, guess what, you have access it to them too!


Qlik Sense in 60 - Qlik Web Connectors


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


Qlik Sense Cloud Business Subscribers:

As stated in the video, Qlik Sense Cloud Business subscribers can also get immediate access to a great selection of many of our web connectors, with more being added on a regular basis. With Qlik Sense Cloud Business, subscribers can quickly and easily fetch data from many web-based sources with a new integrated connector selection list and query manager. The addition of these features, eliminates the need to copy and paste any script as you would when using the standalone Qlik Web Connector service.


How-To Videos

For more information on a list of Web Connectors just released for Qlik Sense Cloud Business check out our recent announcement here: Getting More Connected in the Cloud


Additional Web Connector Resources:


Learn more from my colleague Adam Mayer who shows some brief examples using Twiter, Facebook and Google Analytics:



We want to hear from you, so please join the conversation by posting your questions and comments below.




Mike Tarallo


Can't see the video?


Download the .mp4 and watch from your computer or mobile device

Coming from a traditional business intelligence and ETL background where I’ve used both proprietary and open source solutions – things like OLAP, data modeling, SQL, 4GLs, semantic layers and even coding dashboards were commonplace for many years. So when I started with Qlik, I was not only thrilled, but quite biased towards the way I manged and created analytics . As I set out to build my first Qlik application I immediately had many questions:

  • Where do I build my semantic / metadata layer?
  • Where is my data modeler tool to define my table relationships?
  • How do I place parameters to pass where-conditions to my database?
  • How do I wire my visualizations to make my dashboards interactive?


As I set out on this journey to appreciate Qlik, I was surprised at what I found. In this multi-part blog series I will share with you some of my pleasant surprises that changed the way I create and work with analytics.

So let’s dive in further to my first Qlik pleasant surprise.


#1: Where's the "Semantic Layer"?!


NOTE: The subject matter in this topic can have many moving parts and is not meant to be a direct comparison to traditional BI or other data visualization tools. It is an expression of my experiences when first working with Qlik and meant to inform those who might have similar expectations when comparing Qlik to other BI tools.

A typical first step when using BI tools is that you usually create a metadata or semantic layer before you can begin creating your reports and dashboards. Depending on the tools being used, sometimes you even need to create more than one…one layer to support operational reporting and one to support interactive OLAP. (Often when a vendor's software stack combines a number of acquired tools, you may see this occur.) Creating abstraction layers such as these are usually performed by BI administrators whom are familiar with the data. Metadata / semantic layers provide a unified, consolidated view of data across the organization. They are important middlemen that connect back-end data to BI client tools and add a layer of governance and security. These centralized repositories store many attributes of your data including specific attributes used by the tools:

  • Connection information
  • Business friendly field names
  • Field and row-level security
  • Aggregations
  • Data modeling properties such as keys, and referential integrity
  • Calculated expressions

Once a semantic layer is prepared, the BI client tools can begin using the "data models", "business objects", "data dictionary" etc, to create reports and dashboards.

Now, when I started working with Qlik, I was unsure how this step was performed -OR- perhaps what I was doing seemed more transparent and less rigid than what I was used to. For starters I could quickly load data directly from many sources and go directly into my analysis. Qlik simply loads and indexes data into its apps during the app creation. These apps then operate within an associative indexing engine. (more on that in another article) - If I wanted to add multiple sources, I can visually profile the data and define relationships automatically - no complex data modeling or SQL required. I did not need to worry if I was "joining" correctly or even understand the relationships of the tables. I found this to be a huge time-saver and the process was fairly simple to navigate. This was indeed a pleasant surprise.

10-10-2017 8-20-16 AM.png

Once the data was loaded into the app, I immediately had access to my fields and could begin creating my analysis. In addition, I discovered that I can also set up reusable expressions, measures, dimensions and even predefined visualizations, through a feature known as Master Items. Master Items are defined in the Qlik Sense App rather than a centralized server. They allow business users to use a pre-defined library of assets so they can easily create and customize visualizations. (Note: Master Items are more beneficial to users of Qlik Sense Enterprise as opposed to those using Qlik Sense Desktop.)


This was a pleasant surprise as it was extremely easy to get started with Qlik and cut my data preparation time significantly, in turn giving me more time to create my analysis.So basically, I discovered that Qlik, defines its "semantic layer" within the app itself, as opposed to a separate repository?!? Hmmm.....

A healthy debate ensues

To some degree, I understand there are some opinions about this approach, and some may say "That's not a semantic layer". Now there are ways that you can export a Qlik data model and data index, created within the app, to disk (.qvd) and use them with other applications. This has been done widely and successfully by many of our customers. But...with the new Master Items approach introduced in Qlik Sense, we needed a way to centralized those assets for other applications as well, so our team of experts have created a Qlik Powertool, the Governed Metrics Service.

"The Governed Metrics Service (GMS) Powertool provides the ability to control and provision the use of governed Master Items. GMS loads externally defined metrics and applies them to one or more designated applications. GMS will quickly have you on the right track to governed self-service."

To learn more about GMS check it out here: Introduction - EA Powertools Governed Metrics Service

My journey with Qlik is no where near completed and I anticipated many more pleasant surprises. Next week I'll cover another pleasant Qlik surprise that helped me uncover things in my data I had not idea existed. Have a comment or question, a pleasant surprise you want to share? We want to hear from you. Use the comments section below to ask a question and join the discussion.


Michael Tarallo (@mtarallo) | Twitter

For starters, if you are a visual learner and new to Qlik, take a look at this brief video to get an idea of how to provision from multiple tables using Qlik Sense:


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube. (video and sample files)

For more videos that can help you get started with Qlik Sense:

Hey guys - thanks for joining me in this Tuesday's edition of the Qlik Design Blog. Today I have provided a video that walks you through the Qlik Sense Data Load Editor and at the same time highlights a simple use case example where Qlik Load Script can come in handy....loading data from an Excel file, while looping through its multiple sheets.


As you may know Qlik Sense has a powerful visual data preparation interface to help you profile and prepare data for analysis, but did you know that Qlik also provides advanced scripting capabilities to help you transform and augment data as well as perform many other tasks, including those you may found in popular programming languages?


Some common examples include:


  • Incremental loading
  • Adding row and field level security: Section Access
  • Creating .QVD files
  • ...and more


Watch this brief video to learn about the Data Load Editor and see it in action, read on to learn about what else Qlik Script can do.





NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.




You can learn more about what other great things Qlik Script can do in these awesome blogs by our Jennell McIntire jmc:



...and many more. Stay tuned to the Qlik Design Blog on Fridays, where Jennell and team blog about many interesting Qlik design topics.


Can't see the video? - You can download the .mp4 attached to this post.


Samples files used in the video also are available.


Visual Data Preparation References:





Michael Tarallo (@mtarallo) | Twitter

Hello Qlik Community! Qlik's Bruno Calver is back sharing his in-the-field experiences, building upon his User Experience white paper made available last year. This time he has authored a white paper on Data Literacy. Bruno has worked with many large global enterprises, helping them discover the value in their data and how to best represent it in order to drive results.  During this time he has come across many different ways of working with data. This article tries to consolidate the key take-aways within the context of data literacy. Be sure to download the attached PDF at the bottom of this post to learn more.



Literacy skills have always been top of the bill in the education system -- and for good reason. Equally, data literacy skills are climbing the agenda in today’s competitive business environment.


Organizations will soon rely less and less on pre-processed information and their gut instincts as a decision making paradigm. It is increasingly important for everyone to apply critical thinking skills to every problem and data set in order to achieve a competitive advantage and create truly innovative solutions.


However, for many of us it is not clear what data literacy means, let alone the skills and techniques that might help drive our own data literacy level. The attached article explores these concepts and provides 5 key areas to think about when analyzing your data - including practical examples:


  1. Trends & Context
  2. Internal & External Data
  3. Cohorts & Cell based analysis
  4. Averages, Aggregation & Distribution
  5. Bias & Non-Causal Correlations


Increasing data literacy skills can have a profound impact on organizations in the following ways:


  • Increasing user adoption and awareness of analytical tools and capabilities
  • Creating data driven cultures to enhance performance
  • Unlocking more value from your data investment


If this sounds interesting, then please read the article and see what new things you can discover about the language of data…!



Bruno is a Principal Solution Architect working in the UK with some of Qlik’s enterprise customers. His passion is working with business people to turn disparate and otherwise mundane data sets into insights and stories that can engage their audience, drive change and inspire new ways of thinking.

Hey guys, happy Tuesday! I'm pleased to be back in this edition of the Qlik Design Blog to introduce you to the September 2017 release of Qlik Sense....right off the heels of hurricane Irma I might add. Florida (my home state) and its surrounding neighbors have been hit hard by this storm and I would like to first express my deepest sympathy for those that were affected by Irma's wrath. We will recover, as we always have, and by working together and supporting one another, we will be back on our feet in no time...oh and hurricane Maria, you better watch yourself Missy!) Now on to the show!


As we have previously mentioned, Qlik will be releasing software 5 times a year. Which equates to a new release about every 10 weeks or so. (You can learn more about our continuous release cycle by checking out this blog here: Qlik Sense® will release 5 times a year!)


Rather than list the new feature highlights in this blog, I created a brief video to cover and demonstrate them. You can always check out the Qlik Sense release notes on the Qlik Help site as well as download the attach data sheet to learn more. You can also try these features right now by logging in to the Qlik Sense Cloud or by downloading a free version of Qlik Sense Desktop: Download Free Data Visualization Development Platform.




Now remember, each release will vary in capabilities and features, with some releases possibly having "more" in certain areas than others. It is important to note that the releases will not only cover Qlik Sense specifics, but other product areas such as Qlik NPrinting, Qlik GeoAnalytics, Qlik Sense Mobile, Qlik DataMarket, data connectors, API enhancements and new services and procedures. For example, with the September 2017 release we are also introducing a new Extension Certification Pilot program to allow extension authors to submit their extensions for review, to be certified and supported by the author for use within the Qlik platform. Stay tuned for more information on this pilot program to learn how you can submit your extension for certification.


NOTE - Qlik Sense Mobile: The app is available on the iOS VPP app store (Apple’s enterprise app store) this week.



This is a huge milestone for Qlik as it brings the power of our Associative Technology to the iPad, online AND offline. Customers and partners need to have signed up for Apple’s enterprise developer program to get access to the app. Once they are members of enterprise developer program, they can download the app from https://vpp.itunes.apple.com. Most medium and large business have this relationship with Apple and for those who do not, we are bringing the app to the iOS public app store in the near future, so hold tight.





Mike Tarallo


Additional Resources:

* To learn more about extensions - check out these videos:



* Did you miss what's new in our Qlik Sense June 2017 release? - Check it out here: Qlik Sense 2017 Releases - What's New


* More Videos


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


Can't see the video?


Download the .mp4 and watch from your computer or mobile device

Hey guys, I'm trying something new this time around, a talking-head video - or perhaps better known as video blogging.


In this video I answer community members' and twitter followers' questions. Recently I was asked to explain what Qlik Market was and who should use it. Watch this brief video to learn more.




If you have a specific question (that's right - I said specific question) about Qlik or Qlik Products and you want me to answer it, hit me up on Twitter https://twitter.com/mtarallo or tag me in a Qlik Community discussion. Now note I can't promise to answer every question, but hopefully our valued members and Qlikies can jump in too. If your specific question benefits others it may be featured in my next video blog. Enjoy!


Next on deck: How you can get your product ideas heard and considered.


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


Can't see the video?


Download the .mp4 and watch from your computer or mobile device.

einsteinshow.jpgIn this week's Qlik Design Blog I am pleased to share some tips from our own Mitul Vadgama of our Qlik Consulting Services team.  Mitul was recently working at a customer site when he noticed something that seemed a bit off when analyzing the customer's data. On the surface after loading the data everything appeared fine - but once they started performing selections, Mitul was able to identify the specific cause with the help of Qlik's Associative Difference, the elusive <NULL> value.


The Problem


The customer's data model had about 300+ fields with a number of these fields being of a nominal data type (also known as categorical data). Put simply, I like to describe nominal data as the attributes in the data that don't carry a numerical value. Examples of nominal data are as follows:





Mitul found that when business users were selecting values in a nominal field it gave the correct result, however when they performed advanced selections such as using the the Select excluded option available in a select list, the results did not match up against what was expected. Thanks to Qlik's Associative Difference they were able to quickly and easily identify that <NULL> values were being excluded when the customer performed a Select Excluded selection. Yes, I know that sounds weird, the Select excluded option excluded <nulls> from the exclusion selection. (read-on and watch the quick video so this makes more sense to you )


Click to enlarge image


The Fix


One way is to use the NullValue variable and NullAsValue statement set in the Data Load Editor settings.  (Qlik Help)



Set NullValue='<NULL>';

NullAsValue Field1, Field2;


In our example data we have <null> values in the StateProvince and OfficeStateProvince fields both in the Customers, and Employees tables. In the image above you can see that the <null> value is also not select-able in the filter pane.




Set NullValue='<Unknown>';
NullAsValue StateProvince, OfficeStateProvince;


After adding these settings to the Data Load Editor we were able to get the correct results as the <null> values were now being taken into consideration.


Click to enlarge image


Companion Video: Replacing Null Values in Multiple Fields


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


We hope this post has shown you another efficient way to handle <null> values in your Qlik application.


I'd like to thank Mitul for his valuable contribution to the Qlik Design Blog.






About Mitul: Mitul is a member of our Qlik Consulting Services team in the UK. He has worked on a variety of consulting projects using both Qlik Sense and QlikView with many enterprise customers. His passion is transforming data into valuable business insights, knowledge sharing, and enabling customers to get the most value out of Qlik products.





Can't see the video?


Download the .mp4 and watch from your computer or mobile device.

In the this video edition of the Qlik Design Blog, I show you how I use the Web File connector to combine retro-video game pricing data (Sega Master System) with my own local video game collection. The Web File connector is available in all Qlik Sense products, including the free version of Qlik Sense Cloud (Basic). It is commonly known that the Web File connector is fairly simple to use, however I wanted to demonstrate its use in a fun and meaningful way as well as provide additional information that will help you understand how and when it can be used and what to look out for. Enjoy!


8-7-2017 2-34-02 PM.png

Sega Master System Value Analysis - Mike's Collection - Qlik Sense App


The video is 16 minutes long. I first show you how to simply use the Web File connector with a internet based source via HTTP:// (Current Master System Video Game Prices - GameValueNow.com) and then build a few Qlik Sense KPIs. Then I expand on possible issues you COULD run into with Web File, as well as an example using FTP://, then I conclude with a simple demonstration analyzing some of my video game value data. Analysis result: Purchasing a number of the cartridge titles separately from the cases and manuals and the combining to re-sell may prove profitable. Enjoy!


Michael Tarallo (@mtarallo) | Twitter




Qlik Sense Cloud - Using Web File


Time Index:

• 0:00 - 8:30 - Intro and using the Web File connector

• 8:30 - 10:38 - Additional information to be aware of: https://youtu.be/HEcYYn1hrrk?t=8m30s

• 10:38 - 11:40 - Using FTP with inline credentials: https://youtu.be/HEcYYn1hrrk?t=10m41s

• 11:40 - 12:46 - REST connector information: https://youtu.be/HEcYYn1hrrk?t=11m40s

• 12:46 - End - Fun information / demo using finalized Video Game pricing analysis app: https://youtu.be/HEcYYn1hrrk?t=12m52s


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


Sample Qlik Sense app and data attached in case you wanted to play.
Web File Help document: Loading data from files ‒ Qlik Sense


Can't see the video?


Download the .mp4 and watch from your computer or mobile device.

What is Advanced Analytics?


Gartner defines Advanced Analytics as:


The autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations.


What resonates with me here are the key words "sophisticated techniques and tools" - these can mean statistical analysis environments, (perhaps R), or general purpose and machine learning program languages (Python / MATLAB) or even specific operations used for pattern search (regex), just to name a few. So what does this have to do with Qlik? Well, let's say you have an investment in R, Python, MATLAB or even something custom you created for your advanced analytics activities. You may want to reuse or apply algorithms from those systems to data available in your Qlik environment, taking advantage of Qlik's associative model as well - how would you do that? Previously, you might have to export the data from those systems and then import and associate it within Qlik. Well not any longer now that we have Qlik Advanced Analytics Integration.

What is Qlik Advanced Analytics Integration?

7-20-2017 5-40-37 PM.png



Simply put, Qlik Advanced Analytics Integration (AAI) enables direct server to server data exchange between Qlik Sense and 3rd party calculation / analysis engines via a connector or what we also call a plugin. So for example, a specific forecasting function that is part of an R library can now be called from within Qlik script and chart expressions and calculated on the fly. Passing the results back to the Qlik analysis engine and subsequent visualizations.

aa1.gifi.e. - Holt-Winters Forecast / K-means Clustering - running on R


With the release of Qlik Sense June 2017, Qlik specifically now supports the APIs that can provide connectivity to such engines. Allowing you to build virtually any connector to any system. The Qlik Sense engine is also now aware of specific expressions to directly call the 3rd party engine via these connectors. To kick things off we have provided two open source connector projects that enable connectivity to R and Python. (there are compiled binaries for R available here)


To learn more about Qlik Advanced Analytics Integration, join the conversation and obtain resources and community support please visit the Advanced Analytics Integration forum.


Take a look at this 60 second video that provides a quick overview of Qlik Advanced Analytics Integration:





NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.



Michael Tarallo (@mtarallo) | Twitter



Can't see the video?


Download the .mp4 and watch from your computer or mobile device.



Happy Tuesday everyone! - I have the pleasure of introducing our newest contributor to the Qlik Design Blog, Giuseppe Panella. I like to call him Gio! Gio has joined Qlik in 2015 after the acquisition of NPrinting by Qlik. He is now part of the Product Content and Media team as an interaction designer working on the production of videos for the Qlik Help channel managed on YouTube. Gio and I share the same love for video production at Qlik and will be teaming up to bring you the latest news, features, capabilities and innovation from time to time via the Qlik Design Blog and other channels. Make sure to follow / subscribe to get the latest updates on our posted content. Gio is now producing a video series called Tuesday Tips and Tricks - with each short clip describing a particular capability across the Qlik product line. Today's clip shows you how to use a Qlik Sense extension available from Qlik Branch that enables Qlik Sense to provide an On-Demand Report capability in combination with NPrinting. (This capability is native in QlikView.)  You can learn more about this feature here. http://branch.qlik.com/#!/project/58be6fc151be1c2744fb32a0 Future Tuesday Tips and Tricks will be promoted on the Qlik Design Blog from time to time and will also be made available in the Help Channel playlist. Enjoy!




Tuesday Tips and Tricks


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


Can't see the video? - You can download the attached .mp4 file to play on your computer or local device.

In this edition of the Qlik Design Blog, Denise is back discussing some additional connectivity updates for Qlik Sense Cloud Business and Qlik Sense Cloud Basic.


What's New


Qlik Sense Cloud Business users have been eagerly waiting for direct connectivity to an on-premise Microsoft SQL Server. We’ve seen rapid adoption of Qlik Sense Cloud Business’s connectors to cloud data sources – Salesforce, web connectors, etc. – over the past few months, but we know that just because you’ve moved your analytics to the cloud, you haven’t necessarily moved all of your data there. In this article you will find how to set up the connector and to learn about few other Qlik Sense Cloud updates.


MS SQL Server in Qlik Sense Cloud Business


The process of setting up connectivity between Qlik Sense Cloud Business and a MS SQL Server is similar to what you’ve done to set up other connectors. A prompt will ask you to enter the appropriate host information (server host name, port number, available database, and credentials) to establish the connection and allow access passed the firewall. Check out Mike Tarallo’s video below to see how he set up connectivity between his SQL server and the Qlik Sense Cloud Business environment (thanks Mike!). And, don’t forget, you can use the automated data refresh functionality to update your Qlik Sense apps from your MS SQL data!



Here are a few other updates and improvements that have also been introduced to the Qlik Sense Cloud offering:


Web File Connectivity


You might already have learned a trick to use the REST Connector in Qlik Sense Cloud Business to directly connect to web files, but we’ve now added a separate web file connector so that users have a much simpler and easier way to connect to an unsecured web file. Basically any file source that Qlik Sense can connect to (HTML tables, csv, xml, xls, etc) can also be made available over the web via HTTP and even FTP. Please refer to our online help to learn how to configure these types of connections. As an added bonus, this connector is now available in both Qlik Sense Cloud Business and Qlik Sense Cloud Basic. Qlik Sense Cloud Basic users now have another way to load data in addition to uploading files allowing them to freely experience how the associative model works to uncover insights across multiple data sources.


App Governance


When using Qlik Sense Cloud business you can have up to 50 users within a workspace. With up to 50 users viewing and editing multiple apps, group members often want transparency around when apps were accessed and by whom. We’ve added a “recent history” data stream to apps in the collaborative workspace so users have visibility into group activity, making governance and communication even easier to manage. Simply click on the “i” icon in any app to view the details:




Data Wizard Redesign


As you’ve been building out new apps, you probably noticed that the Data Wizard looks a little different lately. All of the same functionality exists – upload or attach files, access to Qlik DataMarket, etc. - but we’ve listed options differently for better usability. The biggest improvement is that all available connectors will be visible on the screen, which will be particularly helpful when we launch the next group of connectors on the way – check back soon!




NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


denise.pngDenise LaForgia is a Senior Product Marketing Manager focused on Qlik cloud solutions. On an occasional basis, Denise will share updates on our Qlik Sense Cloud solutions on the Qlik Design blog so please subscribe and stay tuned. You can also download the videos mentioned in these blog, see the .mp4 attachment on the bottom.

2laurie.pngThis week I am honored to introduce our newest guest blogger, Laurie Chan-Lam. Laurie is one of our talented architects in Qlik R&D who has personally worked on one of Qlik's unique and game changing capabilities, Smart Search. She recently was inspired by Mitul's blog post on Controlling Fields Shown in Smart Search - when encountering some questions regarding the possibility to create the Smart Search Index, BEFORE the actual search was initiated. (If you are not familiar with Qlik's Global and Visual Smart Search capabilities - you can review a few videos on the topic here)



Mitul's blog Controlling Fields Shown in Smart Search explains how you can configure what fields are searchable when creating the Search Index. So to recap, in order for the Smart Search capabilities to present the desired information to the user when using Smart Search, Qlik Sense needs to index it. With previous versions of Qlik Sense, the search index was created when first requested, therefore taking an additional amount of time before the results were displayed. Starting with Qlik Sense 3.1, you can now chose to create the search index at reload time as explained in the documentation by using the system variable CreateSearchIndexOnReload.


Follow these steps to learn more how to achieve this:



How can I turn on/off the creation of the search index at reload time?

In order to turn on the creation of the search index at reload time, several criteria need to be met:

  1. The Engine needs to be enabled to support the creation of search index at reload time. This can be done via a setting in the QMC. (Qlik Management Console)
    1. Go to menu items Engines
    2. Edit the node you want to work on
    3. Tick the Advanced menu
    4.   Tick the "Enable creation of search index on reload" box




Then in the app in the Data Load script:


The app needs to have the System Variable Set CreateSearchIndexOnReload=1; This statement exists by default in Qlik Sense. Note that the statement of Set CreateSearchIndexOnReload=0; will allow the user to delay the creation of the search index at search time.



Why should I create the search index at reload time?

Creating the search index at reload time makes the search index ready from the first use. Therefore there is no need to wait for the index to be created. The first use of Smart Search after the initial  reload will be as fast as the following ones.


Some exceptions:

Session apps

By definition, session apps are not persisted. Therefore, the corresponding search index shouldn't be persisted and Qlik Sense doesn't index at reload time for session apps.


Synchronized persistence

If the reload node is not the node on which the user is searching, then the creation of the search index on the reload node needs to be disabled. If you do not disable this, an index is created when you reload, which consumes time and disk space to no advantage

Thank You for reading.

Laurie Chan-Lam

Qlik Architect


Hey guys! - I'm very excited to introduce you to our latest release of Qlik Sense, June 2017. Yep - no version number or dots in this naming convention. With the announcement of our continuous release process discussed here, Qlik will begin releasing Qlik Sense every 10 weeks!  Since each release will be similar in scope, we are also replacing our number-based naming convention with releases identified by the month and year they were distributed.  This means that instead of Qlik Sense 4.0, our June release is simply called Qlik Sense June 2017.  (Most other Qlik products will also follow a similar cadence going forward as well.)



Currently customers and partners are invited to Join the Technical Preview and can get the latest software here. If you are not a customer or partner, our freemium product, Qlik Sense Desktop - will be available later in the month (UPDATE: available NOW here: Download Free Data Visualization Development Platform - don't forget you can always experience newer features before the software is available (such as the new data preparation features) without downloading any software by registering with Qlik Sense Cloud, thanks to our continuous release process.


There is so much goodness in this release that I don't want to create a giant text wall telling you about it. I created a brief summary video for your enjoyment as well as longer videos detailing and demonstrating the new features. You can view them at the video index below which include video files and samples. Please note that additional information on our Advanced Analytics integration is available in this community group Advanced Analytics Extensions and requires permission. Please send an email to mailto:insight@qlik.com to gain access. I am in the middle of working on a brief summary and "Hello World"-type video to demonstrate its concepts.


We want to hear from you, so please your comments and questions below!


Qlik Sense June 2017 - Highlights


The main video index for the Qlik Sense June 2017 Release and video .mp4 files and samples can be found here: Qlik Sense 2017 Releases - What's New


NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.



Michael Tarallo (@mtarallo) | Twitter

In this edition of the Qlik Design Blog, our Emerging Technology Evangelist, David Freriks is back discussing integration between Qlik and Kudu.



Navigating the analytics labyrinth with integration of Kudu, Impala, and Qlik


apachekudu_logo_0716_345px.pngUsing Hadoop for Big Data analytics is nothing new, but a new entity has entered the stale file format conversation with the backing of Cloudera – you might have heard of it, it’s called Kudu.


What is Kudu?


Let’s first take a step back and think about the dullest topic in the universe, file system storage formats. Flat files, AVRO, Parquet, ORC, etc. have been around for a while and all provide various advantages and strategies for data access optimizations in an HDFS construct. However, they all suffer from the same issue… static data that can only be appended to – unlike a real database.


So, enter Kudu – defined by Apache: “Kudu provides a combination of fast inserts/updates and efficient columnar scans to enable multiple real-time analytic workloads across a single storage layer.” Deconstructing that message – Kudu acts as a columnar database that allows real database operations that aren’t possible in HDFS file formats. It is now possible to interact with your Hadoop data where INSERTS, UPDATES, DELETES, ALTERS, etc. are now available as data operations. This means not just read/write capabilities for Hadoop , but also interactive operations without having to move to Hbase or other systems. IoT use cases, interactive applications, write-back, and traditional data warehousing are now possible without adding layer upon layer of additional technologies.


Now that we have a general understanding of what Kudu can do, how does this benefit Qlik? Kudu is fast, columnar, and designed for analytics – but with the ability to manipulate and transform the data to power new use cases.


Let’s start simple by showing how easy it is to move some data from an Impala table on Parquet into Kudu.


Starting in Hue we need to do some basic database-like work. To put data into a table, one needs to first create a table, so we’ll start there.


Kudu uses standard database syntax for the most part, but you’ll notice that Kudu is less specific and rigid about data types than your typical relational database – and that’s awesome. Not sure if your data is a varchar(20), or if it is smaller or larger?  No worries, with Kudu –  just declare it as a basic string.




Numerical data are basic as well, there a just few types to choose from based on the length of the number.   This makes creating columns and designing a schema very, very straightforward and easy to setup.  It also reduces data type problems when loading data.


Having a general understanding of table creation, we will go ahead and create a table we are going to copy from Parquet.  It’s worth noting there are some differences here versus creating a Parquet table in Hue.


•    First:  A Kudu table needs to have at least 1 primary key to be created.

•    Second:  A Kudu table needs a partition method to distribute those primary keys


Referencing the schema design guide, we are going to use a HASH partition and use the number 3 (since we have 3 worker nodes).


In summary, we have a bunch of strings, a few integers, and some floating decimals to represent prices and profit. We’ve identified our keys and specified our partitions – let’s roll!


The query runs for a second and viola – we have our new (albeit empty) table. Next, we need some data. We have an existing table that we would like to copy over into Kudu. We will run another query to move the data and make a little tweak on the keys to match our new table.


We had to cast our customer_sk and item_sk columns from string in Parquet to int in Kudu but that’s pretty easy to do as shown in the SQL here.


We run the INSERT query and now we have our data moved over into Kudu, and even better – that table is now immediately available to query using Impala!





Enter Qlik


With the data loaded into Kudu and exposed via Impala – we can now connect to it with Qlik and start building visualizations.


Using the latest Cloudera Impala drivers , we start the process of building a Qlik app by connecting to our new data set.




Opening Qlik Sense, we will create a new connection to our cluster and select our new table.


Once we have the table and columns selected – we can modify the load script created by the data manager to directly query Kudu (versus loading the data into memory) to take advantage of the speed and power of Impala on Kudu.(we do this using Direct Discovery - NOTE the Direct Query syntax) This change is accomplished with a slight alteration in the syntax to identify dimensions and measures.




We now have live queries running against Kudu data sets through Impala.




The great part about Kudu is that we’re just getting started with the possibilities of how we can leverage the technology with Qlik. Some things we’re cooking up for the not too distant future involve write-back with Kafka and Qlik Server Side Extension integration – so stayed tuned.


Please visit cloudera.qlik.com for more demos and to see the Kudu demo in action.



David Freriks (@dlfreriks) | Twitter
Emerging Technology Evangelist


First - a bit on Qonnections

Last week was my 6th Qlik Qonnections, our annual partner and user conference. As usual it was a tremendous event filled with learning, networking and of course "fun and games"...literally this time around for those whom attended.  What was in the past, a partner only event, has grown to include our valued customers for its 2nd year. Our customers, partners and analysts from all over the world came to one awesome place (Gaylord Palms in Kissimmee Florida - my home town!) to share, collaborate, communicate and witness all the great innovation each had to offer...of course including a few things from Qlik. In my humble opinion, each year Qonnections has increased in quality and content....and this one felt like the best one yet, credit goes to our amazing events team, our sponsors and of course our customers and partners! If you want to learn more about all the happenings at Qonnections 2017, I suggest you check our company blog for the daily recaps as well as Cindi Howson's latest blog on the Gartner Blog Network: Qlik Reveals More Roadmap and Vision.


The Qlik Analytics Platform Demo


While at Qonnections, along with my colleague Josh Good, we had the pleasure of presenting a session that in short, basically highlights everything available in the Qlik Analytics Platform. Qlik has so much growing goodness in one box, that it's becoming almost impossible to cover everything we can do for an organization's various needs in just one meeting. So this presentation was created to quickly show what is possible and is performed using 5 "Acts" that demonstrates our core product capabilities while connecting the full breadth of analytic use cases across a fictitious organization - using one coherent story-line. We originally used this as an internal enablement-type resource, but also realized the value it contains for our customers and partners and decided to publish it.


There are 2 videos in this blog, including links out to detailed specifics on each of the use cases. The first video (3 min) is basically a short summary of the 2nd video - introducing you the main concepts, but sacrifices the detailed demonstration. The 2nd video is a longer (23 min) step by step demo flow that dives in deeper into each analytic use case. I hope you find this information useful and please note I am checking on the public availability of the demonstration app used in the videos. Once approved I will post this as an attachment to this post. I am also looking into making all the resources available on our Partner Portal.




Michael Tarallo (@mtarallo) | Twitter




Qlik Analytics Platform Demo Highlights (short)


Qlik Analytics Platform Demo (long)



For more detailed information on the full range of Qlik Analytical use cases, please view this videos at the following links.



NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.


NOTE:  Can't see the video? Download the .mp4 to play on your machine or mobile device.



Filter Blog

By date:
By tag: