1 2 3 4 Previous Next

Qlik Design Blog

51 Posts authored by: Michael Tarallo

Hey guys, I'm trying something new this time around, a talking-head video - or perhaps better known as video blogging.

 

In this video I answer community members' and twitter followers' questions. Recently I was asked to explain what Qlik Market was and who should use it. Watch this brief video to learn more.

 

 

 

If you have a specific question (that's right - I said specific question) about Qlik or Qlik Products and you want me to answer it, hit me up on Twitter https://twitter.com/mtarallo or tag me in a Qlik Community discussion. Now note I can't promise to answer every question, but hopefully our valued members and Qlikies can jump in too. If your specific question benefits others it may be featured in my next video blog. Enjoy!

 

Next on deck: How you can get your product ideas heard and considered.

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

Can't see the video?

 

Download the .mp4 and watch from your computer or mobile device.

einsteinshow.jpgIn this week's Qlik Design Blog I am pleased to share some tips from our own Mitul Vadgama of our Qlik Consulting Services team.  Mitul was recently working at a customer site when he noticed something that seemed a bit off when analyzing the customer's data. On the surface after loading the data everything appeared fine - but once they started performing selections, Mitul was able to identify the specific cause with the help of Qlik's Associative Difference, the elusive <NULL> value.

 

The Problem

 

The customer's data model had about 300+ fields with a number of these fields being of a nominal data type (also known as categorical data). Put simply, I like to describe nominal data as the attributes in the data that don't carry a numerical value. Examples of nominal data are as follows:

 

 

nominal-scales.png

 

Mitul found that when business users were selecting values in a nominal field it gave the correct result, however when they performed advanced selections such as using the the Select excluded option available in a select list, the results did not match up against what was expected. Thanks to Qlik's Associative Difference they were able to quickly and easily identify that <NULL> values were being excluded when the customer performed a Select Excluded selection. Yes, I know that sounds weird, the Select excluded option excluded <nulls> from the exclusion selection. (read-on and watch the quick video so this makes more sense to you )

 

Click to enlarge image

 

The Fix

 

One way is to use the NullValue variable and NullAsValue statement set in the Data Load Editor settings.  (Qlik Help)

 

Syntax: 

Set NullValue='<NULL>';

NullAsValue Field1, Field2;

 

In our example data we have <null> values in the StateProvince and OfficeStateProvince fields both in the Customers, and Employees tables. In the image above you can see that the <null> value is also not select-able in the filter pane.

 

Example:

 

Set NullValue='<Unknown>';
NullAsValue StateProvince, OfficeStateProvince;





 

After adding these settings to the Data Load Editor we were able to get the correct results as the <null> values were now being taken into consideration.

 

Click to enlarge image

 

Companion Video: Replacing Null Values in Multiple Fields

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

We hope this post has shown you another efficient way to handle <null> values in your Qlik application.

 

I'd like to thank Mitul for his valuable contribution to the Qlik Design Blog.

 

mitul.png

 

 

 

About Mitul: Mitul is a member of our Qlik Consulting Services team in the UK. He has worked on a variety of consulting projects using both Qlik Sense and QlikView with many enterprise customers. His passion is transforming data into valuable business insights, knowledge sharing, and enabling customers to get the most value out of Qlik products.

 

 

 


 

Can't see the video?

 

Download the .mp4 and watch from your computer or mobile device.

In the this video edition of the Qlik Design Blog, I show you how I use the Web File connector to combine retro-video game pricing data (Sega Master System) with my own local video game collection. The Web File connector is available in all Qlik Sense products, including the free version of Qlik Sense Cloud (Basic). It is commonly known that the Web File connector is fairly simple to use, however I wanted to demonstrate its use in a fun and meaningful way as well as provide additional information that will help you understand how and when it can be used and what to look out for. Enjoy!

 

8-7-2017 2-34-02 PM.png

Sega Master System Value Analysis - Mike's Collection - Qlik Sense App

 

The video is 16 minutes long. I first show you how to simply use the Web File connector with a internet based source via HTTP:// (Current Master System Video Game Prices - GameValueNow.com) and then build a few Qlik Sense KPIs. Then I expand on possible issues you COULD run into with Web File, as well as an example using FTP://, then I conclude with a simple demonstration analyzing some of my video game value data. Analysis result: Purchasing a number of the cartridge titles separately from the cases and manuals and the combining to re-sell may prove profitable. Enjoy!

 

Regards,
Michael Tarallo (@mtarallo) | Twitter

 

 

 

Qlik Sense Cloud - Using Web File

 

Time Index:

• 0:00 - 8:30 - Intro and using the Web File connector

• 8:30 - 10:38 - Additional information to be aware of: https://youtu.be/HEcYYn1hrrk?t=8m30s

• 10:38 - 11:40 - Using FTP with inline credentials: https://youtu.be/HEcYYn1hrrk?t=10m41s

• 11:40 - 12:46 - REST connector information: https://youtu.be/HEcYYn1hrrk?t=11m40s

• 12:46 - End - Fun information / demo using finalized Video Game pricing analysis app: https://youtu.be/HEcYYn1hrrk?t=12m52s

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

Sample Qlik Sense app and data attached in case you wanted to play.
Web File Help document: Loading data from files ‒ Qlik Sense

 

Can't see the video?

 

Download the .mp4 and watch from your computer or mobile device.

What is Advanced Analytics?

 

Gartner defines Advanced Analytics as:

 

The autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations.

 

What resonates with me here are the key words "sophisticated techniques and tools" - these can mean statistical analysis environments, (perhaps R), or general purpose and machine learning program languages (Python / MATLAB) or even specific operations used for pattern search (regex), just to name a few. So what does this have to do with Qlik? Well, let's say you have an investment in R, Python, MATLAB or even something custom you created for your advanced analytics activities. You may want to reuse or apply algorithms from those systems to data available in your Qlik environment, taking advantage of Qlik's associative model as well - how would you do that? Previously, you might have to export the data from those systems and then import and associate it within Qlik. Well not any longer now that we have Qlik Advanced Analytics Integration.


What is Qlik Advanced Analytics Integration?


7-20-2017 5-40-37 PM.png

Architecture

 

Simply put, Qlik Advanced Analytics Integration (AAI) enables direct server to server data exchange between Qlik Sense and 3rd party calculation / analysis engines via a connector or what we also call a plugin. So for example, a specific forecasting function that is part of an R library can now be called from within Qlik script and chart expressions and calculated on the fly. Passing the results back to the Qlik analysis engine and subsequent visualizations.

aa1.gifi.e. - Holt-Winters Forecast / K-means Clustering - running on R

 

With the release of Qlik Sense June 2017, Qlik specifically now supports the APIs that can provide connectivity to such engines. Allowing you to build virtually any connector to any system. The Qlik Sense engine is also now aware of specific expressions to directly call the 3rd party engine via these connectors. To kick things off we have provided two open source connector projects that enable connectivity to R and Python. (there are compiled binaries for R available here)

 

To learn more about Qlik Advanced Analytics Integration, join the conversation and obtain resources and community support please visit the Advanced Analytics Integration forum.

 

Take a look at this 60 second video that provides a quick overview of Qlik Advanced Analytics Integration:

 

 

 

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

Regards,

Michael Tarallo (@mtarallo) | Twitter

Qlik

 

Can't see the video?

 

Download the .mp4 and watch from your computer or mobile device.

gio.png

 

Happy Tuesday everyone! - I have the pleasure of introducing our newest contributor to the Qlik Design Blog, Giuseppe Panella. I like to call him Gio! Gio has joined Qlik in 2015 after the acquisition of NPrinting by Qlik. He is now part of the Product Content and Media team as an interaction designer working on the production of videos for the Qlik Help channel managed on YouTube. Gio and I share the same love for video production at Qlik and will be teaming up to bring you the latest news, features, capabilities and innovation from time to time via the Qlik Design Blog and other channels. Make sure to follow / subscribe to get the latest updates on our posted content. Gio is now producing a video series called Tuesday Tips and Tricks - with each short clip describing a particular capability across the Qlik product line. Today's clip shows you how to use a Qlik Sense extension available from Qlik Branch that enables Qlik Sense to provide an On-Demand Report capability in combination with NPrinting. (This capability is native in QlikView.)  You can learn more about this feature here. http://branch.qlik.com/#!/project/58be6fc151be1c2744fb32a0 Future Tuesday Tips and Tricks will be promoted on the Qlik Design Blog from time to time and will also be made available in the Help Channel playlist. Enjoy!

 

 

 

Tuesday Tips and Tricks

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

Can't see the video? - You can download the attached .mp4 file to play on your computer or local device.

In this edition of the Qlik Design Blog, Denise is back discussing some additional connectivity updates for Qlik Sense Cloud Business and Qlik Sense Cloud Basic.

 

What's New

 

Qlik Sense Cloud Business users have been eagerly waiting for direct connectivity to an on-premise Microsoft SQL Server. We’ve seen rapid adoption of Qlik Sense Cloud Business’s connectors to cloud data sources – Salesforce, web connectors, etc. – over the past few months, but we know that just because you’ve moved your analytics to the cloud, you haven’t necessarily moved all of your data there. In this article you will find how to set up the connector and to learn about few other Qlik Sense Cloud updates.

 

MS SQL Server in Qlik Sense Cloud Business

 

The process of setting up connectivity between Qlik Sense Cloud Business and a MS SQL Server is similar to what you’ve done to set up other connectors. A prompt will ask you to enter the appropriate host information (server host name, port number, available database, and credentials) to establish the connection and allow access passed the firewall. Check out Mike Tarallo’s video below to see how he set up connectivity between his SQL server and the Qlik Sense Cloud Business environment (thanks Mike!). And, don’t forget, you can use the automated data refresh functionality to update your Qlik Sense apps from your MS SQL data!

 

 

Here are a few other updates and improvements that have also been introduced to the Qlik Sense Cloud offering:

 

Web File Connectivity

 

You might already have learned a trick to use the REST Connector in Qlik Sense Cloud Business to directly connect to web files, but we’ve now added a separate web file connector so that users have a much simpler and easier way to connect to an unsecured web file. Basically any file source that Qlik Sense can connect to (HTML tables, csv, xml, xls, etc) can also be made available over the web via HTTP and even FTP. Please refer to our online help to learn how to configure these types of connections. As an added bonus, this connector is now available in both Qlik Sense Cloud Business and Qlik Sense Cloud Basic. Qlik Sense Cloud Basic users now have another way to load data in addition to uploading files allowing them to freely experience how the associative model works to uncover insights across multiple data sources.

 

App Governance

 

When using Qlik Sense Cloud business you can have up to 50 users within a workspace. With up to 50 users viewing and editing multiple apps, group members often want transparency around when apps were accessed and by whom. We’ve added a “recent history” data stream to apps in the collaborative workspace so users have visibility into group activity, making governance and communication even easier to manage. Simply click on the “i” icon in any app to view the details:

 

app_gov.png

 

Data Wizard Redesign

 

As you’ve been building out new apps, you probably noticed that the Data Wizard looks a little different lately. All of the same functionality exists – upload or attach files, access to Qlik DataMarket, etc. - but we’ve listed options differently for better usability. The biggest improvement is that all available connectors will be visible on the screen, which will be particularly helpful when we launch the next group of connectors on the way – check back soon!

 

data_wiz.png

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

denise.pngDenise LaForgia is a Senior Product Marketing Manager focused on Qlik cloud solutions. On an occasional basis, Denise will share updates on our Qlik Sense Cloud solutions on the Qlik Design blog so please subscribe and stay tuned. You can also download the videos mentioned in these blog, see the .mp4 attachment on the bottom.

2laurie.pngThis week I am honored to introduce our newest guest blogger, Laurie Chan-Lam. Laurie is one of our talented architects in Qlik R&D who has personally worked on one of Qlik's unique and game changing capabilities, Smart Search. She recently was inspired by Mitul's blog post on Controlling Fields Shown in Smart Search - when encountering some questions regarding the possibility to create the Smart Search Index, BEFORE the actual search was initiated. (If you are not familiar with Qlik's Global and Visual Smart Search capabilities - you can review a few videos on the topic here)

 

 

Mitul's blog Controlling Fields Shown in Smart Search explains how you can configure what fields are searchable when creating the Search Index. So to recap, in order for the Smart Search capabilities to present the desired information to the user when using Smart Search, Qlik Sense needs to index it. With previous versions of Qlik Sense, the search index was created when first requested, therefore taking an additional amount of time before the results were displayed. Starting with Qlik Sense 3.1, you can now chose to create the search index at reload time as explained in the documentation by using the system variable CreateSearchIndexOnReload.

 

Follow these steps to learn more how to achieve this:

 

 

How can I turn on/off the creation of the search index at reload time?

In order to turn on the creation of the search index at reload time, several criteria need to be met:

  1. The Engine needs to be enabled to support the creation of search index at reload time. This can be done via a setting in the QMC. (Qlik Management Console)
    1. Go to menu items Engines
    2. Edit the node you want to work on
    3. Tick the Advanced menu
    4.   Tick the "Enable creation of search index on reload" box

 

ww.png

 

Then in the app in the Data Load script:

 

The app needs to have the System Variable Set CreateSearchIndexOnReload=1; This statement exists by default in Qlik Sense. Note that the statement of Set CreateSearchIndexOnReload=0; will allow the user to delay the creation of the search index at search time.


xx.png

 

Why should I create the search index at reload time?

Creating the search index at reload time makes the search index ready from the first use. Therefore there is no need to wait for the index to be created. The first use of Smart Search after the initial  reload will be as fast as the following ones.

 

Some exceptions:


Session apps

By definition, session apps are not persisted. Therefore, the corresponding search index shouldn't be persisted and Qlik Sense doesn't index at reload time for session apps.

 

Synchronized persistence

If the reload node is not the node on which the user is searching, then the creation of the search index on the reload node needs to be disabled. If you do not disable this, an index is created when you reload, which consumes time and disk space to no advantage


Thank You for reading.


Laurie Chan-Lam

Qlik Architect

excited.png

Hey guys! - I'm very excited to introduce you to our latest release of Qlik Sense, June 2017. Yep - no version number or dots in this naming convention. With the announcement of our continuous release process discussed here, Qlik will begin releasing Qlik Sense every 10 weeks!  Since each release will be similar in scope, we are also replacing our number-based naming convention with releases identified by the month and year they were distributed.  This means that instead of Qlik Sense 4.0, our June release is simply called Qlik Sense June 2017.  (Most other Qlik products will also follow a similar cadence going forward as well.)

 

 

Currently customers and partners are invited to Join the Technical Preview and can get the latest software here. If you are not a customer or partner, our freemium product, Qlik Sense Desktop - will be available later in the month (UPDATE: available NOW here: Download Free Data Visualization Development Platform - don't forget you can always experience newer features before the software is available (such as the new data preparation features) without downloading any software by registering with Qlik Sense Cloud, thanks to our continuous release process.

 

There is so much goodness in this release that I don't want to create a giant text wall telling you about it. I created a brief summary video for your enjoyment as well as longer videos detailing and demonstrating the new features. You can view them at the video index below which include video files and samples. Please note that additional information on our Advanced Analytics integration is available in this community group Advanced Analytics Extensions and requires permission. Please send an email to mailto:insight@qlik.com to gain access. I am in the middle of working on a brief summary and "Hello World"-type video to demonstrate its concepts.

 

We want to hear from you, so please your comments and questions below!

 

Qlik Sense June 2017 - Highlights

 

The main video index for the Qlik Sense June 2017 Release and video .mp4 files and samples can be found here: Qlik Sense June 2017 - What's New

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

Regards,

Michael Tarallo (@mtarallo) | Twitter
Qlik

In this edition of the Qlik Design Blog, our Emerging Technology Evangelist, David Freriks is back discussing integration between Qlik and Kudu.

 

 

Navigating the analytics labyrinth with integration of Kudu, Impala, and Qlik

 


apachekudu_logo_0716_345px.pngUsing Hadoop for Big Data analytics is nothing new, but a new entity has entered the stale file format conversation with the backing of Cloudera – you might have heard of it, it’s called Kudu.

 

What is Kudu?

 

Let’s first take a step back and think about the dullest topic in the universe, file system storage formats. Flat files, AVRO, Parquet, ORC, etc. have been around for a while and all provide various advantages and strategies for data access optimizations in an HDFS construct. However, they all suffer from the same issue… static data that can only be appended to – unlike a real database.

 

So, enter Kudu – defined by Apache: “Kudu provides a combination of fast inserts/updates and efficient columnar scans to enable multiple real-time analytic workloads across a single storage layer.” Deconstructing that message – Kudu acts as a columnar database that allows real database operations that aren’t possible in HDFS file formats. It is now possible to interact with your Hadoop data where INSERTS, UPDATES, DELETES, ALTERS, etc. are now available as data operations. This means not just read/write capabilities for Hadoop , but also interactive operations without having to move to Hbase or other systems. IoT use cases, interactive applications, write-back, and traditional data warehousing are now possible without adding layer upon layer of additional technologies.

 

Now that we have a general understanding of what Kudu can do, how does this benefit Qlik? Kudu is fast, columnar, and designed for analytics – but with the ability to manipulate and transform the data to power new use cases.

 

Let’s start simple by showing how easy it is to move some data from an Impala table on Parquet into Kudu.

 

Starting in Hue we need to do some basic database-like work. To put data into a table, one needs to first create a table, so we’ll start there.

 

Kudu uses standard database syntax for the most part, but you’ll notice that Kudu is less specific and rigid about data types than your typical relational database – and that’s awesome. Not sure if your data is a varchar(20), or if it is smaller or larger?  No worries, with Kudu –  just declare it as a basic string.

 

1.png

 

Numerical data are basic as well, there a just few types to choose from based on the length of the number.   This makes creating columns and designing a schema very, very straightforward and easy to setup.  It also reduces data type problems when loading data.

2a.png

Having a general understanding of table creation, we will go ahead and create a table we are going to copy from Parquet.  It’s worth noting there are some differences here versus creating a Parquet table in Hue.

 

•    First:  A Kudu table needs to have at least 1 primary key to be created.

•    Second:  A Kudu table needs a partition method to distribute those primary keys

 

Referencing the schema design guide, we are going to use a HASH partition and use the number 3 (since we have 3 worker nodes).

 

In summary, we have a bunch of strings, a few integers, and some floating decimals to represent prices and profit. We’ve identified our keys and specified our partitions – let’s roll!

 

The query runs for a second and viola – we have our new (albeit empty) table. Next, we need some data. We have an existing table that we would like to copy over into Kudu. We will run another query to move the data and make a little tweak on the keys to match our new table.

 

We had to cast our customer_sk and item_sk columns from string in Parquet to int in Kudu but that’s pretty easy to do as shown in the SQL here.

 

We run the INSERT query and now we have our data moved over into Kudu, and even better – that table is now immediately available to query using Impala!

 

3a.png

 

 

Enter Qlik

 

With the data loaded into Kudu and exposed via Impala – we can now connect to it with Qlik and start building visualizations.

 

Using the latest Cloudera Impala drivers , we start the process of building a Qlik app by connecting to our new data set.

 

4a.png

 

Opening Qlik Sense, we will create a new connection to our cluster and select our new table.

 

Once we have the table and columns selected – we can modify the load script created by the data manager to directly query Kudu (versus loading the data into memory) to take advantage of the speed and power of Impala on Kudu.(we do this using Direct Discovery - NOTE the Direct Query syntax) This change is accomplished with a slight alteration in the syntax to identify dimensions and measures.

 

5a.png

 

We now have live queries running against Kudu data sets through Impala.

 

6a.png

 

The great part about Kudu is that we’re just getting started with the possibilities of how we can leverage the technology with Qlik. Some things we’re cooking up for the not too distant future involve write-back with Kafka and Qlik Server Side Extension integration – so stayed tuned.

 

Please visit cloudera.qlik.com for more demos and to see the Kudu demo in action.

 

Regards,

David Freriks (@dlfreriks) | Twitter
Emerging Technology Evangelist

C_86UVPU0AAZQLi.jpg

First - a bit on Qonnections


Last week was my 6th Qlik Qonnections, our annual partner and user conference. As usual it was a tremendous event filled with learning, networking and of course "fun and games"...literally this time around for those whom attended.  What was in the past, a partner only event, has grown to include our valued customers for its 2nd year. Our customers, partners and analysts from all over the world came to one awesome place (Gaylord Palms in Kissimmee Florida - my home town!) to share, collaborate, communicate and witness all the great innovation each had to offer...of course including a few things from Qlik. In my humble opinion, each year Qonnections has increased in quality and content....and this one felt like the best one yet, credit goes to our amazing events team, our sponsors and of course our customers and partners! If you want to learn more about all the happenings at Qonnections 2017, I suggest you check our company blog for the daily recaps as well as Cindi Howson's latest blog on the Gartner Blog Network: Qlik Reveals More Roadmap and Vision.

 

The Qlik Analytics Platform Demo

 

While at Qonnections, along with my colleague Josh Good, we had the pleasure of presenting a session that in short, basically highlights everything available in the Qlik Analytics Platform. Qlik has so much growing goodness in one box, that it's becoming almost impossible to cover everything we can do for an organization's various needs in just one meeting. So this presentation was created to quickly show what is possible and is performed using 5 "Acts" that demonstrates our core product capabilities while connecting the full breadth of analytic use cases across a fictitious organization - using one coherent story-line. We originally used this as an internal enablement-type resource, but also realized the value it contains for our customers and partners and decided to publish it.

 

There are 2 videos in this blog, including links out to detailed specifics on each of the use cases. The first video (3 min) is basically a short summary of the 2nd video - introducing you the main concepts, but sacrifices the detailed demonstration. The 2nd video is a longer (23 min) step by step demo flow that dives in deeper into each analytic use case. I hope you find this information useful and please note I am checking on the public availability of the demonstration app used in the videos. Once approved I will post this as an attachment to this post. I am also looking into making all the resources available on our Partner Portal.

 

Enjoy!

 

Michael Tarallo (@mtarallo) | Twitter
Qlik

 

 

 

Qlik Analytics Platform Demo Highlights (short)

 

Qlik Analytics Platform Demo (long)

 

 

For more detailed information on the full range of Qlik Analytical use cases, please view this videos at the following links.

 

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

NOTE:  Can't see the video? Download the .mp4 to play on your machine or mobile device.

 

down-arrow-png-17.png

The following is a recording from our latest Tips and Tricks Webinar.

 

 

We want to hear from you. Please leave your comments and questions in the comments section below.

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

Happy Tuesday everyone!. You guys are going to love this one. In this edition of the Qlik Design Blog, our Emerging Technology Evangelist, David Freriks is back discussing integration between Qlik and a powerful big data unstructured search platform called Solr. Not only does David discuss an out-of-the-box approach to this integration, he takes it to the next level and touts the power of the Qlik Platform APIs.

 

Solr

 

In case you haven’t seen it – there is a super powerful unstructured search platform used within the big data ecosystem called Solr, built on the Apache Lucene search engine library. What’s great about Solr is that it can index just about anything, text, xml, JSON, PDF, Word, Excel, including almost any kind of text based data. That means you can drop just about anything into Solr and make it searchable using the power of Lucene core which powers the Solr platform.

 

So, where does Qlik fit in you may ask? Well, let’s observe what a Solr query output looks like:

 

1.png

Standard Solr query output

 

Hmmm, not very user friendly, not to mention it was somewhat slow to execute. Here is a little bit about what we’re looking at:

 

  • This data is the collective set of Enron emails from its infamous collapse in early 2000’s.
  • We’ve loaded this data set into our Cloudera cluster and indexed it using Solr.
  • Once this data was loaded and indexed we tested with a series of queries.
  • A full query on someone with a lot of references such as Ken Lay can run upwards of 15 minutes to bring back every email that contains a reference to him.

 

Imagine 10’s or 100’s of users each waiting 10-15 minutes for a single question to be answered, it clearly dilutes the effectiveness of the engine as a business tool.

 

Enter Qlik

 

Qlik has a tremendously powerful REST connector that is perfectly suited for connecting to sources such as Solr. (A great resource created by Mike Tarallo on the Qlik REST connector can be found here: Working with the Qlik REST Connector, Pagination and Multiple JSON Schemas - check it out to understand the basics of how it works and how the response data is assembled within Qlik)

 

What follows is how we are using the Qlik REST Connector to connect to Solr.

 

Qlik In-Memory Analytics with Solr

 

Now that we are armed with the Qlik REST Connector, and the appropriate Solr REST API connection parameters, we can pull the entire Enron email data set into the Qlik engine via Solr. (Refer to the Apache Solr Documentation to learn more,)

 

2.pngQlik REST Connector configuration

 

By pulling the entire data set, and loading it into Qlik, we now ensure that all users have sub-second access to all the data down to the most granular level, and thanks to our associative search technology – all the data has been indexed and correlated in-memory.  We can gain further insights by incorporating stock market data. Combining Enron’s stock performance with their emails tells an interesting story of rising email volume along with collapsing stock prices and elevating trade volumes.

 

5.png

Power of Qlik Data Visualization - Enron stock performance correlated with email volume


Using a mix of visualization techniques, we can see a pretty interesting collection of data, including the famous “deleted emails” gap on the bottom right chart.

 

Performing some additional analysis, we can drill in on the height of the crash that also correlates with the spike in email volume, followed by a rapid drop in volume.

 

7.pngDrop in trade volume

 

Making a few more selections we can dive down into a specific name, or comment to filter down the result sets further.

 

6.png Detail and specifics - name, email address

 

This associative search allows us to dive down into the details of the “TO” elements of the data set and see the metrics affiliated with those names.  We can also jump over to the final sheet of the Qlik Sense app and look at the individual emails body content filtered by our prior selections made in the application.

 

8.png

 

QIX API Powered Solr Search

 

The above approach of using Qlik in-memory to front end the Solr search engine is just one of the many ways Qlik can access unstructured data in big data systems. Let’s consider another application also using Qlik with Solr – this time with just the Qlik API’s. As a quick refresher, the Qlik engine (called QIX) is a fully API enabled engine with tremendous extensibility that allows Qlik to plug into any web based technology (like Solr). Using the awesome QlikSocial framework from the esteemed Johannes Sunden he adapted the webapp to connect to Solr on demand and build a full webapp from scratch. This is a great example of what we call Custom Analytics.

 

We start with a search box… And our name(s) of interest:

 

9.png

 

Now unlike the formatted Qlik Sense app, when a user hits the “search” bar – everything will happen dynamically on the fly using the API’s.

 

10.png

 

Qlik will dynamically generate a REST connection to Solr, create and load the requesting data into memory, and then build a web app around the data using bootstrap.js and angular.

 

11.png

 

The webapp is still using the Qlik engine, so selections and the search engine are still available – but all the charts and graphics are html and d3js charts – not Qlik. We’re just powering the app and the data interactivity with the QIX engine!

 

Summary

 

Solr is an extremely powerful unstructured search engine that can benefit from the speed and structure of Qlik analytics. It can provide a focusing lens on the core Solr search technology. That data can be consumed in a number of formats including a completely structured Qlik Sense app, or as an API powered web application without any Qlik UI components.

 

For more information, visit our demo site at cloudera.qlik.com

 

Enjoy!

 

Regards,

David Freriks (@dlfreriks) | Twitter
Emerging Technology Evangelist

Wow that's some title huh? Ooooh - "Qlik Sense Cloud Business and the Web Connectors" - sounds like the title for a fantasy adventure novel. Seriously, Denise LaForgia and I are back with a Qlik Sense Cloud Business update including some new videos to briefly introduce you to some really cool and new capabilities available in Qlik Sense Cloud Business - our new Web Connectors starting with access to data for Google Analytics, Twitter and Facebook. Take it away Denise!

 

Thanks Mike!

 

As promised in my blog last month, I’m back with more exciting updates about new features in Qlik Sense Cloud Business. Following our launch of REST Connectivity, I’m excited to announce that Facebook, Twitter and Google Analytics data sources are now also available in Qlik Sense Cloud Business under our new Web Connectors package.

 

For business users in particular, these connectors provide an easy way to bring together and analyze multiple data sources and data sets that are critical to sales, marketing, and other business initiatives. While some tools for social and sentiment analysis might allow you to analyze data from those sources individually, the power of Qlik Sense lets you associate this data about social and digital activity with other information about your customers, sales, marketing campaigns, customer service, and more.

 

Here’s an overview of the type of data each connector can return:

 

Google Analytics

The data returned includes many of the fields you’d see in the Google Analytics dashboard, such as page views, top landing pages, most visited pages, etc. You can retrieve data on any Google Analytics-enabled website.

  • Accounts: returns the accounts that the user has access to
  • DataFromQueryURI; enter a query URI on this table
  • DataFromTemplateQuery: returns a report from one of the available prebuilt queries
  • WebProfiles: returns the profiles that the user has access to
  • WebProperties: returns the web properties the user has access to

 

Watch Mike's brief video to get a general idea of how it works:

 

Community page and video download

 

 

Twitter


The content returned includes all tweets that include a hashtag or search term, and you can use Twitter query operators to pull data for more specific, detailed searches.

  • Mentions: returns up to 800 tweets for a Twitter screen name
  • Search: returns tweets based on a search term and other parameters
  • SearchAdvanced: returns tweets based on search term and other parameters and returns more columns than simple search
  • UserSearch: returns information about an account based on a named user search or topic search

 

In this video Mike shows how simple it is to get after Twitter data:

 

Community page and video download

 

Facebook Fan Pages and Groups

 

The data retrieved includes textual content (posts and comments) as well as counts of likes and shares.

 

  • Feed: returns the feed of posts (including status updates) and links published by the selected page, or by other users on the page.
  • Page: returns a single page
  • User Info: returns a single user - note - user identify / vanity id is used as the parameter

 

In the final video we close the loop on the last of the connectors by simply getting access to Facebook data:

 

Community page and video download

 

How to Get Started

 

While in your Qlik Sense Cloud Business workspace, you can set up your connections within your app by going into the data load editor and selecting the Create New Connection button. You’ll have to authenticate each connection using credentials from an account – your personal account, or one belonging to your business, group or organization. Once the connection is established, you can begin retrieving data.

 

5-2-2017 6-57-50 AM.png

 

4-28-2017 10-42-32 AM.png

 

 

We’re rolling out additional connectors in the next few weeks, so stay tuned for additional information!

 

Learn more


Of course check out the videos and for more detailed information and instructions, visit these resources:


Enjoy your day!

 

Denise LaForgia

Senior Product Marketing Manager

Qlik

 

Resources:

 

Picture1.png

 

Good Tuesday Qlik Community! Today I have a fun and entertaining way to showcase the power of the Qlik Sense platform, its APIs and the art of the possible. But first, let me give you some background. In March, Qlik participated in the Gartner BI Bake Off (direct highlights here) - and for the 'innovation' portion of the presentation, my colleague Josh Good demonstrated a neat concept, showing what is possible with the Qlik platform and it's robust set of APIs - a Qlik Sense Bot - developed by my colleague Juan Gerado Cabeza. Since then we are getting numerous requests to see and learn more about what was shown on that day, so I have create this brief video to share some of its insights.

 

4-18-2017 7-38-28 AM.pngWhat is a Bot you may ask?

 

You already know some popular bots – Siri, Cortana, Alexa and of course Google. These guys are programs designed to perform tasks, such as setting an alarm, telling you the weather, searching online or even ordering a pizza. So why not have a bot tell you who your top sales people are, alert you when profit margin reaches 20%  – or - even have it send you charts and reports directly to your device?  The possibilities are endless. Juan as done just that, creating an analytical assistant - that in actuality turns Qlik data visualization into conversational analytics.

 

Now understand the Qlik Sense Bot - IS NOT a product provided by Qlik, but rather a neat concept that demonstrates the art of the possible when using the Qlik platform and its APIs. Now - neatness, coolness and Qlik platform superiority aside, what I like the most about this innovation is that it provides many real world applications. Bots can be used by anyone, anywhere, on any device and at anytime – scheduling automated tasks and providing access to information when and where you need it – reducing complexities, and increasing availability of insight by simply having a conversation. So in my opinion I do see this as a viable product offering, but you never know. Watch this brief video to learn more about this awesome concept. If you want even more information about this superior innovation, please contact us at qlik.com, or you can reach out to us on Twitter or on the Qlik Community.

 

Have a great day!

 

Regards,

Michael Tarallo (@mtarallo) | Twitter

 

Pushing the Boundaries of Analytics  - Qlik Sense Bot

 

We want to hear from you. Please leave your comments and questions in the comments section below.

 

NOTE: To increase resolution or size of the video, select the YouTube logo at the bottom right of the player. You will be brought directly to YouTube where you can increase the resolution and size of the player window. Look for the 'settings' gears icon in the lower right of the player once at YouTube.

 

NOTE:  Can't see the video? Download the .mp4 to play on your machine or mobile device.

 

down-arrow-png-17.png

denise.png
Happy Tuesday everyone! Thanks for joining me in this week’s Qlik Community Design Blog. Today I have the pleasure of introducing our newest guest blogger, Denise LaForgia. Denise is a colleague of mine in the Product Marketing group and is a Senior Product Marketing Manager focused on our cloud solutions. In this week's edition she will be covering our new REST connectivity recently made available to Qlik Sense Cloud Business subscribers. On an occasional basis, Denise will share updates on our Qlik Sense Cloud solutions. Take it away Denise.

 

Hi Everyone,

 

Welcome to this first installment of what I would like to refer as our Qlik Sense Cloud Update blog. I plan on bringing you all the news about the latest updates in Qlik Sense Cloud as well as some tips and tricks to help you get the most out of your Qlik Sense Cloud subscription. Occasionally, I might even ask Mike to embed a supplemental video to go along with the topic as we have done in this article. Please note that I will also provide continuous updates in the Qlik Sense Cloud Community Section along with the occasional appearance here. We have a lot of exciting features rolling-out, so stay tuned!

 

This week we’re excited to announce the launch of REST Connectivity in Qlik Sense Cloud Business. We know Qlik Sense Cloud Business users are eager for additional data connectivity options in order to automatically import and associate data sets from multiple sources. REST connectivity provides flexibility to a wide range of connectivity options with many of the applications you may be using in your business or project group or team.

 

So what is REST?

 

REST stands for Representational State Transfer, a modern and lightweight, secure communications protocol used to transfer data over the web. The Qlik Sense Cloud Business REST connector is designed to load data into a Qlik Sense app from a service that supports REST. It can return data in many formats such as JSON, XML, or CSV. Most web-based applications, social media channels, cloud-based CRM systems and even Google Analytics are REST-enabled, which means you can now build a connection between Qlik Sense Cloud Business and those data sources.

 

How does it work?

 

The Qlik Sense Cloud Business REST Connector can be considered a 'generic' connector, meaning it gives you the flexibility to configure a connection with any REST-enabled source you’d like to pull data from. Depending on which application you want to connect to, you can navigate to its developer area and configure that application’s settings to open up a REST connection. Visit this area in our help section to read examples on how to do that for LinkedIn, Twitter, Facebook, and Google Analytics. (included in video) Once you have the query parameters, head to the data manager or data load editor in Qlik Sense Cloud Business to complete the connection.

 

rest1.png

 

 

You can also use the REST Connector to load data files directly from public web files, such as DropBox, by simply entering the file’s URL in the REST Connector configurator.  The Qlik Sense Cloud Business REST Connector loads the data into your app and automatically parses the information into appropriate table and field structures so that it’s easily used with your application’s data model. And, you can use the scheduled refresh feature in Qlik Sense Cloud Business to ensure your data files from the REST Connector are always up to date.

 

Ready to learn more - webinars, videos:

 

Watch Mike's video below or go to the Set Up Select Sources for REST Connectivity page for more information about how to connect to different data sources – including Facebook, Twitter, and Google Analytics - using REST. Mike will also be presenting a Tips and Trick's webinar on REST Connectivity with a LIVE Q&A on May 10th at 1PMEST - you can learn more about it and register HERE.

 

Regards,

 

Denise LaForgia

Senior Product Marketing Manager

Qlik

 

 

 

 

Introduction to the Qlik Sense Cloud Business REST Connector and JSON Schemas

 

Can't see the video? Download the .mp4 to play on your machine or mobile device.

 

Additional Qlik Sense Cloud Connectivity Resources

 

How-To Guides:

 

How-To Videos:

Filter Blog

By date:
By tag: