10 Replies Latest reply: Feb 25, 2014 2:11 AM by Alexander Nerush RSS

    QV and Elastic Search - bringing in data from external sources, visualising in QV

    Göran Sander

      It seems most extensions focus on enhancing the visualisation capabilities of QV, but what about the opposite?

      What if you want to use extensions to call for example external REST APIs to bring in medium volume data sets into QV from other sources, injecting them into QV's internal data model, and then visualising them using QV's built-in charts, tables etc?


      Is this possible?

      I have done some work around raphaeljs, HighCharts, d3.js and other charting libs, that works well, but before diving into the above use case it would be interesting to know what has been done by others.


      The concrete background here is that Elastic Search ("ES" for short) has proven very capable for dealing with large data volumes. It offers horisontal scalability (running out of space/CPU cycles in one server? Just add one or a few more nodes to the Elastic Search cluster!). It could also be interesting to explore other similar tools in addition to ES.

      A very interesting use case would be to get dimensional data from ES, allow the user to make selections in QV, then fire off a query to ES, get the results back right away (second-level (or less) response times possible), and finally use QV's regular charts to visualise, slice and dice the data, as we usually do in QV apps.



      There are JavaScript libraries for accessing ES data, so that part should be covered.

      But what about injecting external data into QV's internal data structures? Is that possible, or is that a one-way channel (from QV to the extension)?


      There are of course Direct Discovery (DD) too, which is really intended for this use case. But DD relies on ODBC (or possibly JDBC or QVXs??), and most of the horizontally scalable databases do not support any of these interfaces.

      Or is the preferred way to write some glue code that links DD with ES (or other similar system)?