Qlik Community

Qlik Data Catalyst Discussions

Discussion Board for collaboration on Qlik Data Catalyst.

edifiayo
Contributor

delivering data approach

dear all,

we are working in a approach how to deliver data to our business users. We are currently working in two approaches:

 

1. Delivering QVD files by cluster (i.e. sales facts tables, production facts tables ... etc plus dimensions)

 

2. Providing QVS -which combine the fact and dimensions- to business users.

 

Both approaches implies too much workload for our tea, since we need to prepare the data in advance (i.e. extraction, transformation).

 

Any of you have another approach?

 

Best regards,

 

Edi

 

 

Labels (1)
1 Solution

Accepted Solutions
Partner
Partner

Re: delivering data approach

I think QDC is an extremely viable option. I heard that they would be releasing a stand alone version of QDC, which doesn't rely on a hadoop cluster, which could consume QVDs. However, this version was released in June 2019. I have not heard of a free lightweight one at all. However, if that were the case, that would be super cool!

 

In the meantime, I think you could leverage QVD standardization and ODAG to come up with a partial solution. You could have the selector app be their QVD/data selector, which would only associated to other relevant QVDs. Then have the template app generate the script utilizing the selections to create the data model. I think that could be a really cool utility for the less technical people.

Blog: WhereClause   Twitter: @treysmithdev
6 Replies
Partner
Partner

Re: delivering data approach

Are you trying to remove the responsibility of ETL from your team? 

Unless your business users are technical, then this will be a hard task. You can try to train them on the Data Manager in Qlik and have them be self-sufficient to a degree. However, you may be inviting a bigger mess later on when each person joins the data differently and are coming up with different answers.

Qlik Data Catalyst can add another beneficial layer to this. I think it better fits your use case of business users shopping for data. You will have some of the same problem though if they are Preparing the data sets with the ETL tool. The benefit is data is more reusable and can be documented better. For example, if a data set is built incorrectly a comment can be made to it. Also, with the data profiling it can be pretty easy to tell when data is way off.

It's hard to say without fully understanding your situation, but I think if you implemented QDC and your team populates the core data sets, then that will suffice. The users can then make minor modifications in QDC and pull it directly into Qlik Sense.

 

If you have any questions or need help, feel free to reach out.

Blog: WhereClause   Twitter: @treysmithdev
edifiayo
Contributor

Re: delivering data approach

Hi,

Definitely, We are not trying to remove ETL tasks from our team.  My question was what would be the best or the most recommendable approach to deliver data to End-Users.

So far we tried to approaches:

1. Given them access to the QVDs via folder connections

2. Given them a QVS (script which combine facts tables and dimensions)

Both above generates workload to us, since we need to create all this QVDs in advance. That is the reason I wanted know if others face the same issues.

Best regards,

Edi

 

daniel_dalnekoff
New Contributor III

Re: delivering data approach

We are in a similar position of trying to change our delivery as well.

I wanted to try to keep users out of the data load editor as much as possible, but I don't see a way to do it and not severely limit their ability to get data from multiple QVDs or a single pre-built domains (like you described - sales/production/etc, essentially domain specific data models).  

One of our ideas was to have sets of QVDs that they could select from to drop into the script that would have keys & naming structures to make it as error-proof as possible.  We would drop these into a folder and only give most users access to that set of QVDs.  There would still exist the opportunity for a user to try to rename and re-write the joins, but it could be a start.

QDC looks very promising.  I thought we heard there would be a free lightweight version (QVDs only, limited capability) coming out in Q3, but nothing yet.  We are in the process of getting more information about the product. 

Excited to learn what other ideas you may have and how any progress is going....   

Partner
Partner

Re: delivering data approach

I think QDC is an extremely viable option. I heard that they would be releasing a stand alone version of QDC, which doesn't rely on a hadoop cluster, which could consume QVDs. However, this version was released in June 2019. I have not heard of a free lightweight one at all. However, if that were the case, that would be super cool!

 

In the meantime, I think you could leverage QVD standardization and ODAG to come up with a partial solution. You could have the selector app be their QVD/data selector, which would only associated to other relevant QVDs. Then have the template app generate the script utilizing the selections to create the data model. I think that could be a really cool utility for the less technical people.

Blog: WhereClause   Twitter: @treysmithdev
edifiayo
Contributor

Re: delivering data approach

Hi!,

many thanks for your comments and suggestions. 

As so far we do not have the QDC, we are trying to deliver QVDs on demand by using a excel table. it works sth like this:

1. The user go to the excel file and select the KPI/measure of their interest.

2. Then, the QVD generator app takes the values selected by the user. In the load statement of this app, it takes Measures and Attributes as variables. In this sense, the QVD created and stored reflects the fields selected by the user.

Load

$(vMeasures)

$(vAttributes)

Table Control.JPG

 

As you can observe, it is made by using an Excel file. I would like to know your opinion and know if there is another approach.

Thanks

Partner
Partner

Re: delivering data approach

You can do the same thing with ODAG. Have the users select Measures and Attributes and generate a child app, which would pass the selections into the load script and load what they are looking for. This would make it more dynamic for the users.

Blog: WhereClause   Twitter: @treysmithdev