is it possible to Analyze data at the Source Itself using Qlik ?

[ramyasaiqv]

Hello Qlik Experts,

I just happened to attend a senior exec’s meeting today .They had a question and asked me to do some research and find out if this requirement can be fulfilled using Qlik Sense. The requirement is there is a huge amount of (granular) data in AWS and they want to display only the aggregated (summarized) data in Qlik Sense.  To drill-down to the details they want to go back to AWS and should be able to analyze the granular data the way we do on the Qlik sense UI for the summarized data.

In simple words only summarized data should be brought into Qlik sense for analysis and granular data should be analyzed at the source system using qlik sense capabilities. Is this something doable in Qlik?   I really appreciate your inputs. Thank you,

Ramya

[dwforest]

Absolutely, though Qlik does a great job of allowing analysis of details.

In your load script, write the SQL to do the aggregation.

[SumData]

LOAD

SELECT

SUM(metric)

FROM YourTable

Group By YourKey

;

[ramyasaiqv]

Hi David,

Thank you for your response. I think my question is not clear,  i'm not talking about the data aggregation in qlik sense.

I want to pull and display high level aggregated data in Qlik and analyse the detailed data in the source itself using Qlik ( i'm not sure but big data tools does this i guess). Rather than pulling entire data into qlik and analyze it in qlik, qlik should reach out to the source and analyze detailed data in the source itself. Hope this time it's bit more clear. Thanks,

Ramya

[Ian_Crosland]

Hi

The use case you mention is possible with On Demand App Generation, you can find more details here:

https://community.qlik.com/blogs/qlikviewdesignblog/2018/01/23/big-data-and-qlik-sense-on-demand-app...

[ramyasaiqv]

Ian,

I was looking for something  similar to this. Thank you,

Ramya

[ramyasaiqv]

Hi Ian,

Thank you for the link. gone thorough the link and also  did a lot of R&D and finally looked into the On Demand App Generation (ODAG)examples provided with November 2017 Version. The provided examples uses qvd's  but in real time  big data analysis is it mandatory to generate qvd's  to pull granular data into the template apps? My question is

is it mandatory to create qvd's in doing big data analysis  using  on demand app generation? Thank you.

Ramya

[Ian_Crosland]

‌Hi

No you can go dorectly to the source, the example shown in the video uses SAP HANA , more examples will be posted to that blog next week

Thanks

Ian

[ramyasaiqv]

Hi Ian,

Thank you for the response. Our data source is Amazon Redshift and one of the departments in our organization is exploring different BI tools for their data analysis purposes. The main requirement from them is the tool should support big data analytics. We want to convince the new department that Qlik is capable of handling big data. Our organization already has some other direct qlik competitor tools and we want to push qlik to them (the user base could be over 400). I don’t have much time to do a POC as they want to make the decision next week. So, I’m trying to use the sources and examples available from Qlik website.

Again my question is regarding the on Demand app Generation.

In the 1^st step the Selection app is created only with aggregated data from source-   I’m clear about this

In the 2^nd step for the template app generation for granular data do we need to use qvd’s? Or is it possible to reach out to the source system to pull the data directly without storing in qvd’s? To my understanding Direct Data discovery pulls data into qlik memory after the query execution but in ODAG the data is stored latently in the memory. If you have more info please can you share? Thank you,

Ramya

[Ian_Crosland · Accepted solution]

‌Hi

As I mentioned in my previous comment you can go directly to the source (e.g. Redshift)  without qvd files in the detail app. Direct Discovery is not a strategic solution for Qlik in favour of ODAG.  The template detail app only contains small slices of data leaving the majority at source and can be updated with new selections from the summary selection app.

You can find more details here:

https://help.qlik.com/en-US/sense/February2018/Subsystems/Hub/Content/LoadData/creating-OnDemand-app...

I have also attached more examples and a white paper

Thanks

Ian

[ramyasaiqv]

Thank you so much Ian, really appreciate it.

Ramya