topic Re: Direct Discovery in QlikView

Direct Discovery

Wed, 23 Apr 2014 06:36:23 GMT

Hello All ,

Need understanding on Direct Discovery, what is it ? how we can use it ? with examples.

Re: Direct Discovery

ashfaq_haseeb — Wed, 23 Apr 2014 06:44:14 GMT

Hi,

If you wana try for direct discovery, I would suggest you to try this out with V11.2 sr5 as it has many enhancement.

I'm sure below links will help you to start with.

http://community.qlik.com/docs/DOC-5534

http://community.qlik.com/docs/DOC-3781

http://community.qlik.com/docs/DOC-3710

http://community.qlik.com/blogs/qlikviewdesignblog/2012/11/08/qlikview-112-and-direct-discovery

Hope it helps

Regards

ASHFAQ

Re: Direct Discovery

tresB — Wed, 23 Apr 2014 06:45:15 GMT

See the demo app here: Direct Discovery_Demo_Adworks_11.2_SR5.qvw

Re: Direct Discovery

Wed, 23 Apr 2014 06:59:09 GMT

Thank you Ashfaq, could you please give some examples on how to use direct discovery in V11.2 sr3, i dont have sr5, demo app would be great.

Re: Direct Discovery

Wed, 23 Apr 2014 07:01:26 GMT

thanks tresesco, do you have any demo app for 11.2 SR3?

Re: Direct Discovery

ToniKautto — Wed, 23 Apr 2014 11:25:45 GMT

rohit sharma please notice that Direct Discovery had a major rebuild in 11.20 SR5. This means that the syntax and implementation is not directly compatible between 11.20 SR3 and SR5 (or later). For Direct Discovery feature evaluation it is highly recommended that you do this in 11.20 SR5 or later.

In short the Direct Discovery feature is aimed for big data collections, that does not fit into memory during load. Instead QlikView queries data from the database based on users current selections.

If your production environment is short on memory it is recommended that you evaluate if an increased memory will solve your deployment needs, instead of using Direct Discovery. The simple reason for this is that Direct Discovery loads data during application usage, and thereby adds latency to the user experience. The Direct Discovery performance will be in line with database side processing of the SQL queries and the data transfer time from database to QlikView. Direct Discovery will always be slower that evaluating data that has been loaded into memory during load.

Re: Direct Discovery

Wed, 23 Apr 2014 11:42:59 GMT

Hello toni, I have seen examples on direct discovery, in all of them they have used tables from DATABASE ,can't we load data from exel file or qvd's and use direct discovery on data extracted from these files?

Re: Direct Discovery

ToniKautto — Wed, 23 Apr 2014 11:50:03 GMT

That is correct, it requires a SQL compliant database. The reason for this is that the feature targets big data quantities, larger than would fit in memory using the generic QlikView loads.

Content from files like Excel or text files simply can not become large enough to not fit into a modern computer memory. Once your data becomes to large for handle or store on file, you will also very likely move to storing it into a database.

QVD files stores data in the same format as QlikView stores it in memory. This means that QVD files are intended fro optimized loads, as they can be copied directly into memory. For this reason QVD files are assumed to fit into memory, and will also not be relevant for Direct Discovery. You are not able to store the QVD file unless the data fits into memory first so that it then can be written to file.

Re: Direct Discovery

Wed, 23 Apr 2014 12:02:55 GMT

thank you toni, could you provide some more examples or demo app for more understanding...

Re: Direct Discovery

ToniKautto — Wed, 23 Apr 2014 12:21:27 GMT

The best way to evaluate Direct Discovery is that you install 11.20 SR5 on a test environment, and evaluate the feature after studying the Technical addendum and FAQ. Especially focus on the limitation of the feature, as this will set a clear boundary of what you can accomplish with the current edition of Direct Discovery.

QlikView Direct Discovery FAQ 11.2_SR5.pdf

QlikView 11 2 SR5 Direct Discovery Technical Addendum.pdf

If you do not currently have a database to test with, see the Demo below which will work easily with the Microsoft Adventure Works database, The first sheet in the application has shortcuts to the sample database file and to SQL Server 2012 Express, of which both are free of charge.

Direct Discovery_Demo_Adworks_11.2_SR5.qvw

The three items above should give you a complete walk through of the feature and capability of Direct Discovery. However, if all your current data fits into memory I would recommend continuing with regular loads and skip Direct Discovery for now.

Re: Direct Discovery

Thu, 24 Apr 2014 16:22:44 GMT

Rohit - there really are not any demo apps built against the SR3 release of Qlik, in that version the only DD feature that was implemented was the ability to pre-aggregate fields, using the MEASURE keyword, in the src system then load the result into memory. This was a great start for some simple use cases, but SR5 is where the majority of the key functionality was delivered.

Re: Direct Discovery

Fri, 25 Apr 2014 05:54:45 GMT

thank you robert, and i would actually want to know why we should opt for Direct Discovery ?,what difference does it make ?

Re: Direct Discovery

Fri, 25 Apr 2014 13:41:34 GMT

There are 3 basic use cases Direct Discovery is designed to exploit, and the general benefit with each is that you can provide access to more data than can comfortable fit into memory, because while Qlikview scales very well it can't keep pace with the growth in data size, from millions of rows, to billions and beyond. The use cases are:

1) Drill to Detail - For a large table with say 100+ columns, load only the critical fields into memory, and leave the majority of the fields in the source system. This saves as much as 90% in terms of memory space, but still allows the user to explore all of the data

2) Historical comparison - Provide access to deep historical data, again without loading everything into memory. The QLik App can focus on analysis of the last 6mos of data, but the user can also compare to 2yrs or 10yrs ago.

3) Realtime Analysis - Because Direct Discovery can reach out to source systems with real-time, adhoc queries, Qlikview can now combine the best of in-memory performance with direct query, answering the question "what is happening now?"

Re: Direct Discovery

Tue, 29 Apr 2014 12:04:33 GMT

hello tresesco, i downloaded the demo also downloaded the Adventure work sample database but its still getting ODBC error and table not found error ??

Re: Direct Discovery

Tue, 29 Apr 2014 12:05:28 GMT

hello ashfaq, i downloaded the demo also downloaded the Adventure work sample database but its still getting ODBC error and table not found error ??