1 Reply Latest reply: Feb 14, 2018 12:48 PM by Petter Skjolden RSS

    QlikSense on BIG DATA - Hive on Hortonworks

    Miguel Semedo

      Hi people.

       

      I'm trying to develop a bunch of apps that allow us to analyse data on a big data Cluster, on Hive.

       

      The first steps were realy easy but now I have a few doubts about the foundations of what I am doing here:

       

      • due to the large volume of data I'm trying DIRECT QUERY. I have mixed feeling on this. What's your opinion about it?
      • I explored some documentation about "On-Demand Apps"; it can be, potentialy, a good solution to my problem. What's Your experience on this regarding BIG DATA?

       

      This is just to start the discussion about these themes.

       

      Thanks in advanced for your help/opinions.

       

       

      Regards,

      Miguel

        • Re: QlikSense on BIG DATA - Hive on Hortonworks
          Petter Skjolden

          Direct Query is a kind of "legacy" solution still provided by Qlik but there is no ongoing development taking it further. I would try to avoid taking this approach.

           

          Two other approaches could be used:

           

          1) On Demand App: As long as the drill-down approach by having a selection app and a result/detailed analysis app works well for your use-case it is quite good. However it is important to be able to create the right kind of Hive queries and know the limitations and understand the architecture of how On Demand Apps work.

           

          2) Advanced Analytics Integration: by using the Server Side Extensions (Server Side Expressions) you can use for instance Python as a "middleware" to connect to Hive and build a better solution than the DIRECT QUERY approach is with full control of the queries and much easier debugging.

           

          The two approaches could be combined. The first approach could be more than a two level solution - it could be expanded into three levels if necessary.

           

          If you have a look at the demo site that Qlik has built for Cloudera - many of the approaches could be used also on Hortonworks although Impala and Kudu probably wouldn't be very relevant for a Hortonworks solution.