Standing on the platform waiting for the #7 to approach, I inadvertently began a conversation with a craggy gentleman about the controversial meaning of life.image1abc.png A gruff sigh got my attention as the man looked down at his wrist. The train was about 5 minutes late. “What am I doing here?” his lips muttered as he looked up from his watch. I thought for a moment. Does he mean here as in on the platform waiting for the train? Does he mean here in that very spot where he stood sandwiched between the column and the subway performer? Does he mean here in this city? It was difficult to infer the true meaning of his question, so feeling sociable and inquisitive I initiated a conversation. I glanced over with a smile and implied “Rough day?” In hindsight I wish I kept my mouth shut and simply moved over.

This situation prompted me to think about the importance of semantics. Semantics is the study of meaning and interpretation within linguistic expression. How many meanings could the word here have? In this situation at the train station, here required knowledge of additional context. The gentleman provided semantic clues during our conversation that helped me accurately discern between relevant and irrelevant content to understand his meaning.

The conversation moved from the platform to the train -- standing room only, of course. He ranted about politics, religion, healthcare, education -- any topic that perturbed him. They all seemed to meld together into one big ball of anger and discontent. The pace dragged on and my eyes started to glaze. I was lucky if I got a word in at all. There was a pause in the conversation. “Son, do you know why you are here?” he asked. Being a wise guy and looking for anyway to get out of this intolerable conversation, I cocked my head to one side and pointed to the floor beneath my feet and replied, “You mean here, or do you mean here on the train or do you mean here talking to you?” He said “Let’s not argue semantics, you know what I mean?”

"No sir, I’m afraid I don’t."

Semantics Are Critically Important for Business Discovery

So, what does this have to do with Business Discovery? Semantics are the key to turning a deluge of raw data into a relevant source of actionable information. When applied correctly, semantics cannot only define relevant meanings of data, but it can be used to apply a consistent and reusable approach to managing BI applications and its ever expanding sources of data.

When we discuss linguistic semantics we interpret the further meaning of words and phrases by using existing knowledge and our surroundings. Sights, sounds, and symbols can all have a semantic meaning. However, without semantic clues it is left up to an individual’s own interpretation and knowledge, which can be flawed.

The Semantic Layer

BI software providers use the descriptive form of the word “semantic” to describe components of their software. This comes in the form of what is usually known as a semantic layer or framework -- an architecture that describes and provides a consistent meaning of reusable business application attributes, objects, and data. “A layer of meaning,” as I like to call it. This layer tends to be strictly defined by BI administrators who should have familiarity with the business problems and strong collaboration with peers, rather than using loose interpretations.

Metadata

In BI, a semantic layer bridges the gap between the way  users work with data and how BI applications process it. This is achieved by the use of metadata. Metadata is an ambiguous term that can be defined in this content as data about data. (how ironic)  It is traditionally designed in a top-down fashion. Created in a prescribed manner, it provides a consistent foundation for reuse in development, analysis, and even data governance. Properly defined metadata can reduce information inaccuracies and misguided decisions by decreasing the number of disparately defined objects, metrics, and dimensions used throughout an organization. "Business users get data, IT gets governance." Join me in my next article, “Metadata the QlikView Way,” where I expand on these topics and discuss the value and benefits of defining metadata the QlikView way.

It was recently mentioned to me by a customer that there are some mobile BI products on the market that offer “complete functionality'” when disconnected. And this got me thinking... what does complete functionality really mean?

online v offline comm.jpg

In the world of reporting-based tools, data is usually pre-aggregated in some fashion. The result is that a user can access a finite set of high level data points, essentially answering the questions the report writer thought of in advance. Reports may include some degree of drill-down capability or interactivity, offering the illusion that the user is exploring data in their own way. But unless the report writer thought of the exact questions a user wanted to answer, the experience is limiting in terms of the business value it can provide.

So what does this infer for mobile BI? On mobile devices, reports are typically generated by the server and pushed down to a device, where they can be opened and viewed. Well, it's certainly not unreasonable to think that this approach works when offline. But I would offer that reporting-based architectures act like an entirely and always offline solution, meaning that even when a server connection is available the tool will still provide the same limited user experience. The only difference is that when online a user can update or download new reports.

“Complete functionality” for QlikView means something entirely different. QlikView offers the ability for users to explore data their own unique ways, following their own paths to discovery and better insight. However, in order to provide this combination of flexibility and power, QlikView needs to processes large data sets on the fly. This requires compute power that today's mobile devices simply can't offer. So that's why QlikView on Mobile runs “online” -- it relies on the QlikView Server to do the heavy lifting.

So when evaluating mobile BI solutions, make sure you are delivering business value. Don’t assume that because users are mobile they will be satisfied with a limited experience when a connection is available.  The need for true Business Discovery is critical in mobile environments where questions are variable – when people need to test hypotheses on the road, or have an observation on-location that sparks an insight.

We do, of course, understand that some users don't always have a connection, and that there is a valid need for some level of access when disconnected. We think there could be a “best of both worlds” solution.  Stay tuned…

Last month Google announced the public launch of Google BigQuery, a service to bring Big Data analytics to all businesses via the cloud. Google BigQuery enables developers and business users to quickly and easily gain business insights from massive amounts of data without any hardware or software investments. With Google BigQuery, users can run ad hoc, SQL-like queries against datasets that contain billions of rows.

Capture.PNG

QlikView Google BigQuery demo application (http://Qlikview.com/bigquery)

Last week we created a QlikView demo app showing an example of QlikView integrated with Google BigQuery. The demo app contains birth record data for all babies born in the U.S. between 1975 and 2004. Users can interact with huge amounts of data in a simple, visual way, asking questions like, “What is the average age of mothers now vs. in 1975?” The app churns through millions of rows of birth record data stored in BigQuery in mere seconds.

The demo app provides seamless integration with BigQuery using two QlikView capabilities:

  • A custom connector. QlikView developers can use the QlikView BigQuery connector to load BigQuery data into QlikView’s in-memory data model so business users can remix and reassemble it in new views and create new visualizations on the fly. Users can make selections in the data and see what data is associated, and what data is not.
  • An extension object. For massive sets that are too big to fit in memory, even when compressed, QlikView developers can create an extension object to directly query the BigQuery database. Business users can interact with the BigQuery data by making selections in list boxes to get just the relevant cut of the data they need in a user-friendly chart or graph, without creating a single line of SQL code.

People at work are constantly being challenged to efficiently access, filter, and analyze massive amounts of data. This demo shows how QlikView, integrated with Google BigQuery, can provide Business Discovery for business users who want to find insights in very large data sets. With QlikView’s unique associative experience, business users can navigate and interact with the BigQuery data any way they want to, asking ad hoc questions as they come to mind.

The QlikView integration with Google BigQuery enables non-technical and non SQL-savvy users to interact with billions of rows of data in seconds. They can navigate through the massive amounts of data to find what’s relevant to them, and to get answers to their specific business questions without requiring specialized skills.  

Sometimes it’s good to step out of one’s normal activities just to see what ideas get sparked or unexpected connections you make. Last week I went to the Text Analytics Summit in Boston for just this reason. I took loads of notes – my tag cloud is below.

Tag Cloud from Text Analytics Summit Boston.png

As you might imagine, the term Big Data came up a lot at this conference. My favorite definition came from Meta Brown of LinguaSys: “If it’s hard for you to handle, it’s Big.” More formally, Big Data refers to the enormous volume, velocity, and variety of data that exists and has the potential to be turned into business value. Big Data can be structured or unstructured. It can be created by people, calculated by systems, or generated by machines. According to IDC, the volume of digital content in the world will grow to 2.7 billion terabytes in 2012, up 48% from 2011 — and it’s rocketing toward 8 billion terabytes by 2015. (See the IDC report, “IDC Predictions 2012: Competing for 2020,” December 2011.)

In a couple of specific examples from the Text Analytics Summit:

  • NASA is using aviation data to improve flight safety. In the U.S. alone there are roughly 9 million flight departures a year, each of which generates data on hundreds of parameters every second during a flight. Data is generated by the aircraft, satellites, and other systems. In addition, each flight has unstructured data associated with it, such as safety reports and write-ups from pilots and co-pilots.*  Some data resides with the airlines; other data resides in government systems. NASA (the National Aeronautics and Space Administration) is using analytics to dig through all this data to uncover insights that can identify and prevent potential runway incursions and other accidents.
  • eBay is using social media data to get closer to customers. As of June 2012, eCommerce giant eBay had indexed more than 40 million blogs and forums (60 billion posts – that’s 10,000 a second!), which amounts to 65 terabytes of data. Why? The company has a social data intelligence program in place to help decision makers better understand the company’s audiences, influencers, and competitive position, and to deliver superior customer service. A global social analytics team works with multiple groups across the company to find and share insights from all this data.** 

A theme that emerged from the Text Analytics Summit is what I think of as “little data” (though it’s really only little in comparison to Big Data). In use cases like NASA’s aviation safety research program, where every single piece of data is important for the analysis at hand, it’s not useful to take just a subset of the data because something critically important may be missed.

But in uses cases like eBay’s (using customer insights drawn from social media to drive revenue and improve the customer experience), it can be highly effective to use sampling and other techniques to grab a subset of all the data and perform the analytics on that. It’s similar in the legal sector; during a lawsuit, a company’s legal team wants to use and present to the opposing side only documents that are relevant to a particular case and issue.

It may seem like a no-brainer, but it came across loud and clear during presentations at the Text Analytics Summit: an organization’s ability to optimize the process of deriving value from Big Data in a cost-effective way depends on the use case, business drivers, and characteristics of the data.

 

---------------------------------

* According to the Bureau of Transportation Statistics Research and Innovative Technology Administration, in 2012 (ending on the last day of February) there were 9,098,000 departures, compared to 9,125,000 in the same period 2011, a change of -0.3%. For more information see http://www.transtats.bts.gov/.

**On June 13, 2012, eBay’s social commerce analyst Palm Norchoovech shared these insights in a presentation titled, “Global Social Analytics @eBay” at the Text Analytics Summit in Boston, Massachusetts. You can find more info here: http://bit.ly/GSnH03.


The first couple of years in QlikTech’s history, the company was called QuikTech and the product was called QuikView. It was a game with words: The product name insinuated that you could view things quickly and at the same time the letters Q-U-I-K were an abbreviation for what we believed in: Quality, Understanding, Information and Knowledge.

 

Banner QuikTech.JPG

 

We believed that a business could improve its processes and product quality by empowering employees and encouraging them to engage in lifelong learning. And we meant all people – we saw everyone as a decision maker. To get information from data was an important part of creating the understanding, the knowledge and the quality. We were inspired by the management trends of the time, especially by employee empowerment as described in the book "Moments of Truth" (Swedish: "Riv pyramiderna") by Jan Carlzon, president and CEO of Scandinavian Airlines.

 

Thus, the abbreviation was an early attempt to make a values statement and it was there long before the genesis of the product. What the abbreviation stood for was really the ideological base when founding the company.

 

Another, less glamorous reason for the strange spelling was the old DOS file name restriction. File names could not have more than 8+3 characters. There was just no room for a “C”. Hence quikview.exe.

 

And why did we change from Quik to Qlik? Well, we tried to register “QuikView” as a trademark, but our application was rejected. There were already too many software products with the prefix Quick, Qwick, Qvick, or Quik. But we still wanted to protect the product name! However, at this time, we had started to realize that a defining characteristic of the product was that you clicked and viewed. The fact that it was quick was not the first thing that came to mind. So the step to rename the product to “Qlik” + “View” was not very big – in fact, it was even an improvement of our values statement: We just replaced the words “Understanding” and “Information” with “Learning” and “Interaction”.

 

Banner QlikTech.JPG

 

Today our mission statement is “Simplifying Decisions for Everyone, Everywhere.” The words we use to describe our mission have changed slightly: From the general “Knowledge” and “Quality” to the more specific “Decisions” – which is the main step in transforming knowledge into quality. As I see it, the current mission statement is more to the point than our original values statement. Further, it includes the idea that all people are included, which is something we took for granted but failed to express in our initial values statement. In all aspects, the current mission statement is a very good description of what we stood for already 18 years ago and what we still stand for today.

 

We are still true to our initial values. We just express them differently.

 

For more on the QlikTech history, see The QlikTech Company History.

 

HIC

Last week Jeff Ma give a speech at our Business Discovery World Tour event in Dallas, Texas. In the 90s, Jeff was a member of the MIT Blackjack Team. He was the inspiration for the main character in the book Bringing Down the House and the movie 21. In 2010, Ma wrote a book called The House Advantage: Playing the Odds to Win Big in Business. In it, he takes concepts from blackjack into the world of business.

House Advantage Jeff Ma.JPG

In his presentation and the Q&A afterward, Ma talked about some important business lessons he learned from his time as a blackjack pro:

    • Emotions play second fiddle to mathematics. Every decision Ma made at the blackjack table was based on math. “You can’t let emotions sway your decisions,” he said. Even after he lost $50,000 in just a few minutes at a table one day, he knew he had made the right decisions and he went on to recover all the money — and more — later in the weekend. During Q&A at the end of Ma’s presentation I piped up and said that I get it that playing blackjack is a numbers game, without much room for emotion. “But,” I asked, “Is business really like that?” I was glad when he said, “No, of course not.” Because the bigger the business decision, the more important it is to work with, rather than ignore, the emotions involved. Why? Because making the decision is just the beginning. Executing on a decision is when the real work begins. Successful execution requires participation and buy-in from multiple people, and buy-in comes from not just hard data but the relationships and communication among participants.
    • Indecision is still a decision. Ma talked about omission bias — the tendency to think that harmful actions are worse than harm that comes from inaction. He said that four times the number of mistakes are made due to inactivity vs. mistakes due to taking action. He gave a non-blackjack example from his own life. His mother had a stroke and the medical professionals laid out the odds, based on statistics, of recovery with surgery vs. without. She had a 22% chance of living beyond 60 days if they did not operate. So Ma’s family took action and his mom had the surgery, with a positive outcome.
    • When you know you're right, stick to your guns. The lesson: make the difficult decisions. One time, in an important game, Ma split tens. Generally, splitting tens in a blackjack game is not a good idea. But in this particular instance, he thought it through and knew it was the right thing to do. As he stood around the table, he was subject to groupthink. He announced his move and got stares and groans all around. He acknowledged that it was hard to do something that would buck the trend. “But,” he said, “if they all knew what I knew and had the same technology I had, they would make the same decision I needed to make.” Ma made a great business intelligence analogy here: “In an age of Big Data and access to new insights, you will come up with recommendations people won’t want to hear.”
    • The right decision doesn’t always have the desired outcome. For me, this was the most profound insight Ma shared. I usually think of a decision being the right one based on how it turned out. He says no, that sometimes even when you make a data-driven decision that should turn out well, the outcome isn’t what you wanted. This is where the unpredictability of being human, the importance of timing, and the complexity of economics and business come into play. Because something doesn’t turn out the way you hoped doesn’t mean you made the wrong decision.

Want to learn more? Jeff Ma has posted a recording of his presentation up on his web site.

The Dark Matter of Data

Posted by Erica Driver Jun 5, 2012

“I think there’s a sense that many of us have that the great age of exploration on earth is over. That for the next generation they’re going to have to go to outer space or the deepest oceans to find something significant to explore. But is that really the case?” — Dr. Nathan Wolfe

I recently watched a TED video by Dr. Nathan Wolfe, founder and CEO of the Global Viral Forecasting Initiative. I was attracted by the title, “What's Left to Explore?” In this video, Wolfe explains that about 20% of the genetic information in the human nose (which can be obtained through a nasal swab) doesn’t match anything that we’ve ever seen before:  no plant, animal, fungus, virus, or bacteria. This is also the case for 40% to 50% of the genetic information in the human gut.

Wolfe refers to these unknowns as “biological dark matter.” According to Wikipedia today, dark matter is “an unknown type of matter hypothesized to account for a large part of the total mass in the universe.” Biological dark matter is genetic matter that can’t be typed or matched with anything we’ve seen before. 

In biological dark matter, Wolfe sees an exciting possibility: identifying an entirely new class of life (like the concept of a virus, identified by Dutch scientist Martinus Beijerinck within the last century) that may enable us to identify the cause of a cancer or the source of an outbreak, or create a new tool in molecular biology.

The same holds true for business data. One might be tempted to think, “We’ve got data. Lots of it. We’ve got Big Data. What’s left to explore?” Sure, you’ve got data. But it’s in myriad different systems and in an endless number of formats. What might you discover if you brought seemingly unrelated data from these multiple source systems into one QlikView app, where the associations (and lack thereof) in the data would become visible, perhaps for the very first time?

A QlikView app automatically maintains all the associations in the data and calculates aggregations on the fly. The result? Through an associative experience, users can now explore the dark matter of data. They can click or tap away, or lasso sections of charts or regions of maps and at all times they can see what data is associated with their selections and what data is not.

Users make a selection in one chart and all other charts and graphs in the entire app update to reflect that selection — with no hard coding. Users think of a second question, and a third one, and make more selections, and all the charts and graphs in the app update again instantly. It’s through this associative experience that users can easily see, in a visual way, any outliers in the data or any unexpected relationships. (For more info see the QlikView White Paper, What Makes QlikView Unique.)

Dr. Nathan Wolfe offers a great lesson for would-be explorers and it applies to explorers of data as well as biology and genetics: “Don’t assume that what we currently think is out there is the full story. Go after the dark matter in whatever field you choose to explore. There are unknowns all around us and they are just waiting to be discovered.”

PricewaterhouseCoopers LLP (PwC) recently published the “2012 issue 1” edition of its quarterly journal, “Technology Forecast.” This issue is chock full of perspectives and case studies about analytics — definitely worth a read. PWC has generously enabled readers to create their own custom PDF containing one or more articles; the full issue is a whopping 134 pages.

PWC Technology Forecast 2012 issue 1.PNG

In this report, PwC defines the term new analytics as using “a rigorous scientific method, including hypothesis formation and testing, with science-oriented statistical packages and visualization tools.” PwC’s take is that business leaders who embrace the new analytics will be able to create cultures of inquiry that lead to better decisions throughout their enterprises. PwC made the case that new analytics enable an organization to quickly iterate and investigate numerous questions to improve its decision-making capabilities — such as making decisions related to complex, real-time events management, or making possible new, disruptive business opportunities such as the on-location promotion of sales to mobile shoppers.

I agree that data is an extremely important source of insight — of course! But as I was reading through the report, I found myself wondering, “What about the human aspect of decision making?” What about the questions, opinions, experiences, expertise, and persuasiveness and negotiation skills of people? Decision making is an inherently collaborative activity. People don’t sit alone by themselves in front of a computer and make data-driven decisions. (Usually, anyway.) We talk to people. We get others’ perspectives and views.

In this report, PwC really downplayed the role of human perspective in decision making. There are only a few mentions of the word “collaboration” and they generally refer to companies that have a culture of inquiry needed to achieve collaboration between IT and other parts of the business, or about better visualization and tablet interfaces leading to improved collaboration between business analysts and data scientists.

In my view, for an organization to develop a widespread culture of inquiry, people need to be able to communicate easily within the context of their decision apps. They need to be able to explore data together, whether they are online at the same time or not. They need to be able to easily co-create analytic apps so they can get to answers more quickly. So in my view any definition of next-generation analytics can’t be just about the data — it must also include the insights that reside in peoples’ heads, and provide a mechanism for unlocking and spreading this institutional knowledge.

Filter Blog

By date:
By tag: