Data speaking to you

    NLG (Natural Language Generation) is a task of Natural language processing which helps in generating natural language from a machine representation system. It is a translator which converts a computer based representation into a natural language representation. NLG has existed for a long time but it is only recently that commercial NLG technology had become widely available and self-service.

     

    We have heard that a picture speaks a thousand words, but when we come to make sense of data, visualization alone may not tell the whole story. A well-designed interactive chart or visualization is always a great way to see an overview of data and trends However, simple narrative text can help describe, highlight or explain features in ways that many users will find helpful. And for certain cases, such as writing reports and reviews, detailed descriptions may be essential.

     

    Qlik provides a product called ‘Narrative Science’ (This extension is free to download). This product integrates easily to provide a textual interpretation of your visual content. Narratives can draw attention to significant features that we may not immediately understand, such as distributions. In this way, we can learn the visual language for the future. Yet the text descriptions also help to keep us grounded in the details in case we are prone to over-interpret what we are seeing.

     

    Finally, visualizations simply do not tell the whole story in your data. They cannot capture the flow and context of a human conversation which is, in fact, our most fundamental form of collaboration. Their clarity and their specialized visual language illustrate the story, but they are not the story we are trying to tell.

     

    This is why narrative interpretations of data are so powerful in a report or review. Your understanding of the scenario unfolds over time. Our verbal language is highly tuned to assist memory and understanding through this unfolding. And, when it comes to debate and disagreement - the most important and informative aspect of any collaboration - verbal language is our natural tool of choice.

    It takes more than a picture and less than a thousand words too!

    Let’s consider an example:

     

    ns1.PNG

    From this visualization we can summarize:

    • Revenue seems to be improving
    • Revenue is higher by $2 million dollars from Jun 2012 to Jun 2014
    • There is some seasonality

    These are interesting but not enough and lack some specifics.

    When we apply Narratives from Narrative Science to this chart:

     

    ns2.PNG

    So now the analysis can be done as follows:

    • Total revenue over the period ($90M over 30 months)
    • Revenue ranges ($1.1M to $3M per month)
    • Revenue improved 333% over the time analysed, and the biggest jump between Jan 2012 to Feb 2012 by 110% and $1.1 million dollars
    • Peaks of revenue and net growth peaks can be seen

    This is a dramatic difference in data narrative.  No more approximations, no more guesses, no need to go and create further analysis. We can understand a much clearer picture, in context, of what data is trying to tell us.

    Why Qlik is well suited to work with these NLG technologies is because of its QIX Engine. The QIX engine creates a subset of data (called a hypercube) from the overall in-memory model to stream to the NLG engine. Unlike other NLG integrations, it’s not a separate query to a database, but a leverage of the data in context being analysed.  This hypercube contains the information not only of the selections (filters) that are being applied, but by correlation the associations of the selections as well as the non-correlated values.  This is quite unique, as any basic query tool can pass filters – but Qlik being able to include correlated (or uncorrelated) values provides a much richer analysis.

    Qlik is the first to have this capability!