With QlikView, power users can set up associative analysis sandboxes. (We wrote about analysis sandboxes in earlier posts here and here.) QlikView analysis sandboxes can incorporate trusted data sources like a data warehouse as well as external data sources like industry trend or stock market data. All this data is merged in one in-memory location, either on the power user's local machine or a QlikView Server. The power user can immediately begin interrogating the data for answers and insights.

QlikTech technical advisor Elif Tutuk has helped guide our customers in their use of QlikView for analysis sandboxes and recommends the following approach. Pull data from the source systems into a QlikView data file (QVD).* The QVD is a highly-compressed replica of source data. It includes just the data you would want to include in an application-typically not 100% of the data in the source system(s). Production applications as well as the sandbox environment can point to this shared storage area. Then you can go to town creating list boxes, charts, and tables. A chart could be very complex-it could consume a good portion of RAM to visually represent years of data by any combination of dimensions in hopes that the user may "catch something" (identify a trend or outlier in the data).

The QlikView approach to analysis sandboxes has several advantages over traditional approaches:

  • Power users can experiment and explore without harming the original data. They can use the same QlikView data file that's used by production QlikView applications, and also include additional data sources or calculation logic. They can massage the data when they create charts and graphs. Because power users are on a different server from other end users-even while all are accessing the same underlying data source-end user performance is unaffected by the power user's activities.
  • QlikView delivers a high-performance user experience. All users benefit from this approach: power users as well as more casual business users. With QlikView associative analysis sandboxes, there's no waiting for IT to stage the data. QlikView compresses the data at roughly 10:1 and delivers fast load performance. Right away, power users can start analyzing and visualizing the data and identifying patterns, trends, and outliers-and answering urgent, ad hoc business questions.
  • The associative experience facilitates the natural flow of insight discovery. Query-based analysis sandboxes make use of the intersections of data sets. In contrast, with QlikView every search term in the analysis is part of an entire network of data. Relationships among the data are clearly visible. QlikView shows the user not just which data is associated-but also which data is unrelated. QlikView looks through a network of associations for the data that's connected to the user's current selections. (See related blog posts here and here.)

The associative experience and QlikView's visualization capabilities facilitate the natural flow of insight discovery and support a "build to think" approach to BI. With QlikView, its does not take longer to build an analysis solution than it does to think one. (For more on the "build to think" approach to BI see related blog posts here and here.)

* A QVD is a file that contains a table of data exported from QlikView. It is a native QlikView format. The file format is highly compressed and is optimized for speed when reading data from a QlikView script. Reading data from a QVD file is typically 10-100X faster than reading from other data sources.

Information management pros can go about setting up analysis sandboxes in several ways. QlikTech technical advisor Elif Tutuk has put a lot of thought into this and I've got some of her perspectives to share with you. (Analysis sandboxes are environments where power users can experiment with data, explore analysis, and conduct ad hoc questioning in support of answering urgent, ad hoc business questions. We wrote about the concept in an earlier post here.)

Pros and Cons of Traditional Analysis Sandbox Approaches

 

Pros

Cons

Data warehouse-centric sandboxes

In organizations where a data warehouse exists, sometimes the warehouse is partitioned to give power users an area where they can run granular queries. With this approach, it's fairly easy for the database administrator to observe the users' behavior in the warehouse.

As power users run detailed queries in their search for answers, they negatively affect performance for other users running production reports. The solution: spend money on data warehouse processing power to improve performance for the user community.

Replicated sandboxes

The organization purchases another database server, or a data warehouse appliance, and replicates the entire data warehouse. Power users can play in this sandbox without affecting performance for other users.

Replicas are expensive to build. Organizations must purchase, install, and maintain a separate database platform and replicate the data that resides in the data warehouse. And-the sandbox becomes a silo not part of the enterprise BI solution.

Managed client sandbox

The analysis sandbox runs on the power users' desktop or laptop. They run queries against the data warehouse and dump the data into Excel to massage and combine it with additional data sources.

This approach is limited by the memory and processing capacity of the user's machine. The data is not compressed and the user can only pull part of the needed data. As a result, they cannot get a 360 degree view on the business question they are trying to analyze. This approach also leads to an issue of data silos living in Excel spreadsheets and not falling under data governance policies.

 

 

 

While traditional approaches to analysis sandboxes solve some problems, they create others. If the power user is exploring all the relevant data, it's done at a high cost or creates silos or negatively affects system performance for other end users. If the power user is experimenting with a subset of the data to protect end users from performance problems or to keep the costs down, then the path to insight may be slow and difficult.

Power users need a tool to make their analysis ideas tangible faster-especially when they are under pressure to answer urgent, ad hoc business questions. The faster they can make their analysis ideas tangible, the sooner they can evaluate the ideas, refine them, and zero in on the best way to solve a problem or exploit an opportunity.

We have a third post about analysis sandboxes in the works: a discussion about analysis sandboxes the QlikView way, along with some high-level, good-practice recommendations on how to deploy. Stay tuned!

What do kids do when they play in a sandbox? They use mini loaders to fill toy dump trucks with sand. They make mud cakes and build castles. All the while, they are experimenting and learning. What happens when you put heavy rocks in the loader's bucket? How tall can you make your castle before it topples over?

The same is true for BI professionals using analysis sandboxes.

I recently spoke with QlikTech technical advisor Elif Tutuk about analysis sandboxes and have some of her insights to share with you. Elif is a consultant who spends her days with QlikView customers, helping them get the most they can out of their QlikView deployments.

An analysis sandbox is an environment in which power BI users-data-savvy users with strong query and database skills and a solid understanding of the business and business processes-experiment with data, explore analysis, and conduct ad hoc questioning. They can also create prototype BI applications without negatively affecting performance of back-end data sources or the production BI environment. They use these environments to explore enterprise data, combine it with local and external data, and then massage and package the resulting data sets.

Analysis sandboxes can be used to answer urgent, ad hoc questions

Depending on the degree to which the decision makers are tech-savvy, they may even be able to use the sandbox to answer questions themselves. Consider these scenarios:

  • Revenue variance analysis. A general manager receives a monthly revenue statement and sees that revenue is less than expected. She assigns a business analyst to go away and figure out the factors contributing to the variance. Because the question is not a routine business question, the analyst uses an ad hoc information source and does his or her best to find the root cause of the revenue variance with limited time and information available.
  • Quick-reaction decision-making. In a competitive, dynamic business environment, it's not uncommon for an executive to have a 4-hour window in which to make a decision. He may need help from a business analyst to support a quick decision with evidences in the form of data. Without an analysis sandbox, the business analyst may have a hard time finding answers-especially if finding answers requires melding data from multiple sources, and required crunching through a huge volume of data quickly.
  • Customer buying trend analysis. Let's say a management team is talking about why some customers stop buying a particular product after a few months. Perhaps it's not possible to answer this question using higher-level, readily-available data. So the power user has to go into demographic data and purchasing data, and maybe analyze the competition's promotions during a particular period of time. The power user would go to an analysis sandbox to do some exploratory analysis in search of answers.

What these scenarios all have in common is a burning need for fast business answers, in an ad hoc manner. An analysis sandbox is important because the longer it takes to find the root cause of a problem, or the answer to a burning business question, the greater the cost to the organization.

We've identified several ways organizations can set up analysis sandboxes. We'll explore these, and discuss a good practice for using QlikView for analysis sandboxes, in upcoming posts. Stay tuned!

It's the Users, Stupid!

Posted by Jeffrey Boehm Nov 19, 2010

A few things crossed my desk recently that made me pause and question whether the BI sell-side community gets it yet. From blog posts to marketing materials to analyst reports, everyone seems to agree that "traditional BI" isn't meeting the need. But lost in much of this hyperbole is the most important thing that needs to change. BI needs to be about the users. For too long business intelligence has been viewed as a data problem, an IT problem, a corporate governance problem. Yes, there are real issues to tackle there. But the lack of focus on the user is what has kept BI back, and this is where the true next generation solutions will excel.

In a recent post on TDWI the author correctly points out the limits the data-centricity of the current generation of BI tools and why that needs to change. But the author then goes on to say that the next generation should be "network-centric." There are indeed changes in computing infrastructure that will shift how vendors and companies deploy all flavors of enterprise applications. But end users don't care how or where a BI application is deployed - they care about whether it meets their needs in terms of flexibility, performance, usability, etc.

A recent product launch invitation from a "traditional BI" vendor highlighted IT as a primary audience for the event. Were business users encouraged to attend? Not really - unless you are a "finance manager" or a self-proclaimed "business analyst." Average business people - the people out on the front lines making decisions every day - are not targeted because this vendor, and most other traditional BI vendors, still view IT and power users as their target constituency. This is further encouraged by the continual stream of BI grading reports that focus on deep infrastructure stacks and IT requirements, and not user issues around flexibility, time to decision, usability, cost of ownership, etc.

I'm not suggesting that infrastructure issues aren't important, nor should we ignore IT and power users. Not at all. Instead I'm suggesting that the next generation of BI needs to be user-centric. They need to be based around "build to think" rather than "think to build." End users get this and they are already voting for user-centric solutions with their feet and budgets. Progressive IT leaders are also moving in this direction, applying the adage of providing the tools and enabling users to fish, rather than trying to fish for them. Vendors, analysts and pundits needs to recognize this shift and get on board - or risk becoming obsolete and irrelevant.

This shift is happening in other markets: CRM (Salesforce.com), video conferencing (Skype), and mobile devices (Apple iPhone). Those tools meet the back-end requirements, but are built with the user as a top priority. And now it is happening in BI. Are you leading this charge, or stuck in "traditional BI" thinking?

I'm proud to announce the availability of a new 8-minute QlikView video tour. For those of you new to QlikView, this video provides an overview of what makes QlikView different. For old hands, it highlights some of what's new in QlikView 10. This series of short, interconnected video segments covers seven topics: consolidation, search, visualization, enterprise readiness, scalability, extension and collaboration, and the ability to deploy anywhere. Click to watch the video here.

According to Peter Simonsen, our Sr. Director Global Web Marketing, we created this video to give people a good visual introduction to what QlikView is all about. Sure, you could get to a point of understanding by reading content on our web site or exploring QlikView demo apps. But we wanted to show people what QlikView can do for them in a quick and engaging way.

The project was a labor of love-a collaboration among the corporate marketing, product marketing, and product management teams at QlikTech, in conjunction with an outside creative agency.

  • For early brainstorming, core team members met in an immersive environment. At the beginning of the project, team members worked together in a 3D immersive environment called Teleplace. There, we collaborated on an initial version of the script, storyboarded the video segments, explored our collection of QlikView demo apps to decide which ones would be best to show, and tried to come up with imagery to communicate some of the more abstract concepts in the video. We were all able to share our computer screens simultaneously in the 3D space. We all created sticky notes and uploaded images to a brainstorming wall, and any of us could rearrange the images and edit the sticky notes as we worked.
  • We finalized the script and flow using good old fashioned email and phone. Between the early brainstorming and the formal project work, we had lots of discussions about the script and the flow of the video segments. During this time, we were preparing for the launch of QlikView 10, and lots of efforts were under way to make sure our messages were all aligned. The core team exchanged lots of emails and participated in WebEx meetings as we evolved the script and finalized the video flow. At key points in the process, project owner Peter Simonsen reached out to get buy-in from a larger group of internal stakeholders.
  • Once our agency began working on the videos, we used their Basecamp site. Once the script was finalized and the agency began creating the videos, we used their Basecamp project management workspace. There, we communicated with the team and managed collaboratively-created assets, like scripts, screen captures, and video drafts. We were able to use email to add comments to discussion threads; all communications were logged in the appropriate Basecamp folder.

Creating the QlikView product tour video was fun and exciting, and we hope you enjoy the fruits of our labor. (And if you're looking for additional info about QlikView 10, click here for the QlikView 10 landing page.)

For more than a year, a serial shooter was on the loose in the Swedish city of Malmö. He shot at people while they stood at bus stops, sat in their cars, and moved about indoors. One person has died and several others have been wounded in the attacks. (For more information, see the November 7, 2010 Associated Press article, "Police arrest man in Swedish immigrant shootings.")

What does this have to do with QlikView, other than the fact that Malmö borders the town of Lund, where QlikView was founded? A lot. Using QlikView, the police were able to identify the suspect by analyzing 10 years of crime report data, and have made an arrest.

This morning I spoke with Malmö police analyst Berth Simonsson about the role QlikView played in this case. Simonsson said, "QlikView has been a labor-saving tool for the police. Police analysts ask questions and QlikView delivers answers instantly. Instead of going through the reports manually, we have been able to go through lots of information quickly and found the link that otherwise would have been hard to detect."

The Malmö police department is a long-time customer of QlikView, but up until now had not used QlikView to analyze crime reports, for legal reasons. Once the department received permission to use QlikView in this way, analysts loaded 10 years' worth of crime reports (2 million reports, comprising 2 billion rows of data) into an existing QlikView application. It took three hours to load the data and configure interactive reports, and then they could immediately begin investigating the data.

  • Nine months of work in one minute. Simonsson said that without QlikView, analysts would have had to read every crime report by hand to search for clues that might lead them to a suspect. This would have been very difficult. He estimated that it would have taken three people three months to read through just one year of reports-never mind 10 years' worth-to find answers that QlikView made visible in just one minute. So every time the department uses QlikView to analyze historical crime reports in this way, it saves nine months of effort.
  • QlikView is a powerful investigative tool. If police analysts have an idea about a case, they can use QlikView to test their idea. They can search for any city, time of day, reported behavior, or other details. The QlikView associative experience enables them to explore trends in the data. If they find something of interest (e.g., red car or red truck), they can click a button to view the entire crime report. In QlikView, the crime report comes up immediately. With another system the department uses, analysts have to wait a half minute for a report to load.

Because QlikView is so easy to manage and use, the department plans to expand its use into new areas. Even with about 100 QlikView applications in place, Simonsson is one of just two people who support QlikView for the Skåne (southern Sweden) regional police department. Simonsson said, "The two of us are supporting 3,500 people in the police force, including about 50 analysts. We have been supporting it for about three years now. It's very easy, very straightforward." And now that the police analysts have experienced the power of QlikView for solving crimes, they are planning to use it more broadly. For example, the department checks about 10,000 people every year in Skåne; what they want to do is analyze who has been checked and when, similar to the way they can now analyze crime reports.

There's no question-the Malmö police department's ability to conduct analysis more quickly and comprehensively will go a long way toward helping to protect the public. We're proud to play a role in it!

Seeing Is Believing

Posted by Jeffrey Boehm Nov 15, 2010

I joined QlikTech about a month ago and had heard a lot about the company, the product, the culture, and the customer base prior to signing on. After nearly twenty years marketing software solutions in business intelligence, data warehousing and search, I was certainly intrigued by what I kept hearing as the "QlikView way." I've now had the opportunity to experience this first hand and am very excited by what I'm seeing:


The user community is passionate. From the thriving QlikCommunity to the excitement I saw among the users at Q-Days in Atlanta or the Boston Tech Days, I have never seen end users so passionate about enterprise software. QlikTech is focused on bringing BI to end users - and from what I've seen so far, the end users are reacting more like Apple users than traditional enterprise software users.

 

QlikView is truly reaching new audiences. I had heard that QlikView was more business-user centric than any other BI product, but I've been amazed to hear where it is being used. Mike Thompson from Children's Hospital of Atlanta told the Q-Days audience that an emergency room physician built a QlikView app to analyze patient visit data. I'm not aware of too many other enterprise platforms where physicians are "developing" new applications!

 

QlikView is forging a tighter relationship between IT and Line of Business. While QlikView is all about empowering line of business (LOB) users to more rapidly perform their own analyses, QlikView is also driving a better, more strategic dialog with IT. About half the audiences at the recent events were IT, and each customer presenter talked about how LOB and IT are working together: IT ensuring the right data is getting into the application in a secure, reliable way, and LOB deriving tremendous business value through new insights into their business areas.

 

Business insights in days, not years. Even before joining QlikView I downloaded the software and found it incredibly easy to create my first QlikView application. In less than a week I was more proficient in QlikView than I was after months of ramping up at previous companies. And I heard the same from customers: Tim Moore from Colonial Life relayed how he built his first QlikView app over a lunch break in the QlikView training. Colonial Life has now deployed QlikView apps to thousands of employees, agents and brokers.

 

I'm only a month in to my QlikTech journey, but I'm already hooked. If you are hooked also, tell me why. In my global product marketing role, I plan on engaging with our vibrant community and would love to hear about your experiences, what you think makes QlikView stand out, and how we can do a better job sharing that message with the world.

I sat down and talked with Håkan Wolgé, the main architect behind QlikView, while I was in Sweden a few weeks ago. I had two main questions for him about his take on QlikView and the associative experience.

Erica Driver: At QlikTech we use the word "associative" to describe one of QlikView's differentiators. What does this word mean to you?

Håkan Wolgé: QlikView is all about the user experience. Not just the user interface . . . the user experience. With QlikView, the user experience is associative. We bring all the data the user might want to analyze in an application into memory and connect it together using key fields. This allows the user to wander through the data, seeing associations and connections they wouldn't be able to get in any other way.

We keep the data in memory in relational form. With QlikView there are no pre-calculated cubes. We do create cubes on the fly based on current selections the user makes. But cubes are not exposed to the user other than through QlikView objects like charts, graphs, and list boxes. If you want to get technical, I thought analyst Curt Monash did a good job in his article "The Underlying Technology of QlikView." What it really comes down to, though, is that the way we've architected QlikView makes it simple to use, develop, and sell.

Erica Driver: Is QlikView columnar or record-based-and does it matter?

Håkan Wolgé: QlikView is somewhere in between columnar and record-based. We're neither, and we're both. Our records are indices into "symbol tables" (what many people call data dictionaries). We get high performance in part because when a user creates a chart, QlikView only has to look through the data dictionary once. Also, data in QlikView is highly compressed. We find that we generally get a 10:1 compression ratio. This means if you bring a gigabyte of data into memory, we can compress it to 100 megabytes.

Does this matter? Well, it mostly matters that the technology we use delivers a simple, high-performance user experience. By this I mean users need to be able to ask and answer business questions on their own, without help from IT. Their business analysis system must be responsive. They can't be forced to wait a half hour for a query to run-or two weeks for IT to create a new query. A simple, high-performance, associative user experience is where QlikView really shines.

For more in-depth info on QlikView's architecture and the associative experience, see the QlikView Technology White Papers, "The Associative Experience," and "QlikView Architectural Overview."

Erica Driver

Fighting Fire with Data

Posted by Erica Driver Nov 8, 2010

Thinking on your feet is a good thing when battling a burning building, but not so much when you're a fire department battalion chief trying to decide how to allocate scarce resources. A new QlikView in Action white paper is available, titled "A New Approach to Putting Out Fires: Data-Driven Decisions." This case study, written by CITO Research and sponsored by QlikView, describes the value a municipal fire department in the U.S. was able to achieve with QlikView.

  • The agency's challenge: visibility into the numbers. City agency budgets are under pressure during a time when governments are forced to cut services to try to balance the numbers. But to balance the numbers, you have to know what the numbers are. The systems supervisor at this fire department said, "A lot of our management decisions were made flying blind." Prior to implementing QlikView, the agency didn't have key performance indicators because it didn't have a way to access and make sense of its data.
  • QlikView was brought in to unlock the agency's data. The fire department needed a way to unlock data from its ERP systems so it could begin tackling some tough challenges. The systems supervisor said, "We were putting in data with a bulldozer and pulling it out with tweezers." The bulldozer was traditional BI software and the tweezers were reports and spreadsheets. The systems supervisor read about QlikView in a magazine, and that was the beginning of a shift toward data-driven decision-making.
  • Voila! Reduced incomplete incident reports, faster emergency response time, and more. Since implementing QlikView, the number of incomplete incident reports has fallen from more than 500 annually to fewer than 50-an improvement with national implications, given that these reports trickle up into nationwide statistics. Response times to emergency calls dropped dramatically once this data became visible. Compliance with continuing education courses has dramatically risen. And the fire department chief now understands the total landscape of the agency's finances.

At this municipal agency, tech-savvy business managers were able to take their expertise and apply it to their data, with nobody in between. Less technically-oriented leaders in other departments were able to use apps created by QlikView content developers to better prepare for "combat operations" (firefighting) and derive insights that resulted in improvements to critical city services. To learn more about this QlikView customer story, download the white paper here.

QlikView 10, which was released in mid-October, introduces a whole new level of openness. One of the highlights of this release is a new extensions capability. With extensions, content developers can create custom visualizations and user interface components for use within QlikView. Extensions can also be used to bring existing mapping tools, Gantt charts, tag clouds, infographic charts-or any other visualization-into QlikView. Once integrated into QlikView applications, custom and third-party visualizations can take advantage of QlikView's core capabilities.

This video clip shows stream chart and Gantt chart extensions in QlikView. The stream chart shows group product sales over time. The Gantt chart, typically used for project management, is a bar chart that shows a project schedule with the start and end dates of various project tasks.

Content developers create QlikView extensions using common web-based technologies such as Adobe Flash, HTML, Java, JavaScript, and Microsoft Silverlight. QlikView Workbench includes a template to help jumpstart the process. Once they create and package extensions, content developers can install and use them within any QlikView application. Extensions can be created once and reused multiple times.

Extensions are just one of the new capabilities that make QlikView 10 more open and extensible. For content developers we also now offer the new QlikView Data Exchange (QVX) format for mapping data through custom connectors directly into a QlikView-ready format. For IT pros, QlikView 10 provides a configurable directory service provider for integration with enterprise directories and user databases, as well as a set of new application programming interfaces (APIs) that facilitate the flow of information and command between QlikView and its environment. For more information check out the datasheet "What's New in QlikView 10?"

Filter Blog

By date:
By tag: