Q&A with Qlik: App Optimization

[Troy_Raney]

Environment

• Qlik Cloud
• Qlik Sense Enterprise

Transcript

Hello, everyone, and welcome to Q&A with Qlik.
Today, we're going to be talking about App Optimization and
My name is Troy Raney.
I helped create videos and webinars like this one, and we've got a great expert panel for you.
Ben, why don't you quickly tell us a little bit about yourself?
Sure. Thanks, Troy. Good morning, guys. I'm Ben Anderson, principal analytics, presales architect here, part of the pre-sales team. Before joining Qlik just about 8 months ago, I had been supporting customers in the US federal government, as well as commercial customers for the past 10 years. building analytic solutions with Qlik and other tools, as well as workflow automation solutions. So thanks for having me here this morning, Troy, to hopefully get some questions answered.
Great, Eamonn.
Yeah, good morning, everybody, or good afternoon, wherever you are. Eamonn Harrington, just like Ben, I'm a principal analytics pre-sales architect here. I've been at Qlik now for almost 4 years, and then prior to this I was a Qlik customer. So most of what I what I know I learned by doing it myself.
For five or six years, and then I made the jump over to Qlik. So, yeah, happy to be here, and uh… looking forward to it.
Great. Thanks, guys. Well, when you registered for today's session, you had an opportunity to submit some questions ahead of time. So we're going to start with those.
First question today: How to find the sheet or object loading time. Is there any built-in function in Qlik Sense?
Yeah, it's a great question. It's probably one of the kind of like hidden gems in the cloud platform. So the answer I'm going to give here is really just with respect to the analytics in the cloud. But one of the things built in is this application performance evaluation tool.
Um, and what that allows you to do is for any application, you can run this performance evaluation. We're actually going to jump ahead to where we show this in the in the video of our previous.
talk here. you can run this performance evaluation, and it will give you exactly, uh, exactly what this question refers to. So, sheet loading times, object loading times, that sort of thing. It's a great way to.
you know, kind of take a first step into all right, where do I need to kind of focus my efforts to to to make some optimization? And it's… This answer, at least, is also probably a nice opportunity for me to call out what Troy is showing right now, which is, if you didn't get a chance to catch the previous session, we do have a community post and a YouTube video of that session, so…
About 4 minutes in, you can see all about the performance evaluation tool.
Yeah.
Great. Please do check that out. I've… Posted the link to everybody, and… Yeah, Eamonn goes through a demo of that tool as well as the.
app analyzer and how those two in conjunction can really help you troubleshoot your… poorly performing apps and make some great improvements. So, uh… Do check that out. All right. Next question.
I've been separating my data manipulation scripts from the visualization apps. Is that a good strategy?
So the direct answer is it can certainly be a great strategy. To answer this question in the context of.
UI optimization, UI performance optimization, right? Splitting up your.
Transformations, script executed via the script, right? And your UI application isn't necessarily going to really have any impact on the UI performance itself. But this can be a very good strategy. So having a multi-tiered.
Extract, transform, and load pattern with your scripts and your applications is definitely a best practice specifically for reusability and self-service. So if you have transformations, right?
Um, that ultimately create a normalized or a star schema, right? For a data model. Separating out that transformation and subsequent QVDs, for example.
It can be a great practice, because then you can have multiple UI applications downstream that reuse those QVDs, and you're reducing the redundancy of code and potential mismatches across those UI applications.
While that pattern's not necessarily going to achieve any type of performance improvements from a UI perspective. It's certainly a best practice from organization reuse, reusability and self-service.
Great. That's a good answer. Thank you. A next question… How can we identify those fields that are not used in our app? I mean, without using tools like Modio or QSDA.
Yeah. So I will give a shout out to QSDA, as it is a great tool. But as the questioner suggests, there are alternatives. So I'm going to post this link in here. But there is a automation that you can download from GitHub.
Um, that basically goes through your application, looks for unused fields and generates a drop field statement.
Um, at that point, you would just put that drop field statement into the application, and then there you go. The nice thing about this as well is that, uh, because it's just a drop field statement at the end, uh, it's like very… lightweight, if you will.
you know, it's easy to implement, and then if you ever wanted to take it out, it's easy to take it out as well. So this would be my… uh, sort of my go-to here. If you wanted to do something other than using QSDA.
That is a great tip. Thanks. Next question. Is there a better operator to use instead of an if statement, since those tend to affect performance? Does match work better?
So similar to our other questions, I think there there's several answers and considerations with this. So.
When we're talking about if statements in the UI, within the context of a measure and filtering within a measure, it's always going to be more performant to use set expressions.
Okay.
So number one, if you're using if statements within a measure to do any type of filtering, switch that to set expressions, you're going to see, you know, in some cases, potentially drastic performance improvements within that measure.
Um, now, if we're talking about certain cases where you want to have some type of interactivity, say, user toggles a variable or a selection, and you'd like that measure to conditionally change using an if or a match, or even a pick match.
Uh, type expression. Um… The performance differences between those three types of expressions, you're not really going to notice any difference between those . Within that particular context, it really comes down to typically a preference in terms of the developer, right?
If statements can oftentimes become pretty cluttered, especially when you're nesting if expressions, uh, they can be difficult to read while, you know, pick and pick match can oftentimes.
be a little bit easier to read and comprehend.
All right, that's a good answer.
not necessarily a performance improvement, but readability and understandability improvement there.
And one thing that Eamonn demonstrated in the tech expert talk about performance is basically moving all those if statement.
expressions out of the front end and into the back end really sped things up, and separating them out, making unique fields. Yeah.
Those are also good tips. Check out the recording.
Next question. When a large data set is loaded, what are some options to optimize the app for users?
So, big data.
Yeah, I think I'm gonna… Troy, I'm gonna piggyback on what you just said, which is check out the recording. So that's a really good question. It's kind of, uh, it's almost like the unsaid kind of question beneath this whole recording on, uh, optimizing app performance, but.
Yeah.
To give a quick answer to that, there are some major things that you want to look at when you're dealing with large data sets, particularly.
kind of cardinality and… dropping unused fields, right? I mean, it seems like such a simple statement, but it is the truth.
If you're bringing in a ton of data, do everything you can to bring in less, right? And then one of the things that you'll see in the recording is we had a very dramatic example where we had a date field that was timestamped all the way out to a millisecond, which made it extraordinarily like inefficient in terms of the data model.
breaking up that sort of thing, simplifying things where they can be simplified will will give you really, really good kind of, like, return on on your time investment there.
Uh, but in general, those are sort of like the data model considerations. Pretty straightforward, but a lot of return on your time. And then on the front end.
Once again, you'll see more in the recording, but definitely things to consider, as Troy and Ben were just speaking about, you know, keeping if statements out of the front end. Also, maybe on really, really taxing visualizations, adding in some calculation conditions so that things don't have to.
load all the way for each user, that sort of thing.
Great. Next question. Oh, and I just put up one of the resources from that, our help site documentation all about optimizing performance. That's a good resource.
Next question. One large fact table which might get sparse but has short ways to data or multiple or multiple dense fact tables.
User performance versus reload time versus RAM footprint. What would you guys suggest?
It's a big question. It's very philosophical there.
Sure.
Hey, man, I can take a crack at this first. I mean, I think in general, you know, what what we've seen is that concatenated fact tables, which I think is kind of getting at.
the initial scenario here, which is one large fact table. That approach is going to be more performant than multiple fact tables.
You know, associated via bridge tables. the ability to have all of that, or that approach of having all of those fact tables concatenated into one is going to enable you to have more of a start, standard star schema, right? Approach, which is going to.
perform much better than having to, uh, potentially on the front end, create virtual joins across multiple fact tables via bridge tables, which would be the alternate approach using multiple dense fact tables.
So in short, you know, the concatenated single fact table should provide better performance.
Okay.
And then from… from an overall reload time perspective.
that's going to be very dependent on that particular use case in terms of reload performance, but that front-end performance should be better with one fact table.
And ultimately, that's what we care about most, especially when apps are in the clouds. Is it good for the user? Then it's good for everybody.
Next question. Can an app perform faster if loading the same table, same number of records and rows from QVDs instead of directly from the source database.
SQL.
Right.
Yeah, so I'll answer this in two ways. I guess it depends on, uh… where you apply the word performance. If you're talking about front end performance, there's really no difference. Once the the kind of load routine or load task is finished.
All data is created equal, so to speak, right? The data is in the application makes no difference where it came from originally.
That said, if we're talking about backend performance, absolutely QVDs are going to be much more performant, right? So one of the great things about QVDs is the read speed is so fast. And this really, really becomes advantageous, of course, when you're using a table in many different applications, right?
Because, you know, in a simple example, let's say I had a single transaction table, I bring that table into a Qlik application.
Well, actually, if I make a QVD first, that might be more time total, right? I have to bring the table in, I have to make a QVD, then I have to load that QVD. But as soon as I start loading that single table into many different downstream applications, making a QVD is going to save me quite a bit of time, not to mention quite a bit of strain on my source database.
And that's where you really see those performance gains. So generally speaking, using QVDs is almost always the right approach. But as I mentioned, you know, just in interest of being comprehensive here, front-end performance, really no difference.
But back in performance, QVDs are almost always the right decision.
Great. Thanks. Next question. Are there plans to release an update that allows us to copy multiple objects at once into another sheet?
It's not quite the topic for today, but I don't know if you guys have heard anything. Otherwise, I was about to talk about ideation.
That would have been yep, yep.
Yeah. All right. Well, if you go to Qlik Community under support.
You'll find this great site called ideation. It's where everybody, customers, employees, partners, can go in here and see what product feature requests and improvements have been suggested already.
If you find one you like, you can upvote it.
say, you know, that is a great idea. And you can submit your own ideas and that would be an excellent place to go. I know our product management team weighs the input from customers heavily when they're trying to decide what resources to allocate to improvement. So definitely.
take a look at ideation. I'm going to put the link here for everybody.
These are great questions. Keep them coming. Next question.
Yeah.
Yeah.
Oh, Troy, real quick, sorry. The only other thing I would add to that is, I think the question was about copy and pasting objects, right? I would assume the questioner already knows this. But for the benefit of of them and maybe others, you can copy and paste whole sheets as well. So.
I… like I said, I assume… I assume the person already knows this, but just in case, um, you know, you don't have to copy, you know, a want to make a copy of this sheet in a different application, I'm doing, you know, first my bar graph, then my table, then my KPI. You can just do the whole sheet.
Yeah. There you go.
So worth, worth mentioning.
Thanks. Is there something like the App Analyzer, which was demoed in the Techspert Talks recording for QlikView documents?
So there… is kind of. So… There is, and we can put this link in the chat from Rob Wunderlich, also the person who made QSDA Pro in the previous question. There is this QlikView document analyzer.
all of Rob's stuff is great. This is really, I think, the the closest thing to the app analyzer that we have in the from the monitoring apps.
This is not, you know, something that Qlik has made, right? So this is not a supported tool. This is, you know, from the from the community. But definitely worth checking out. And I think.
you know, as far as… as far as the question goes, probably the closest thing, uh… at least that I know of to to what the questioner is asking for.
And I see one make someone making a comment here to get the QlikView to Qlik Sense conversion tool.
We've done some… I've done some sessions highlighting that tool about how to migrate from QlikView to Qlik Cloud. And we're going to have another one.
There we go, the end of April. We have another text for talk talking just about how to migrate QlikView to Qlik Cloud. So we're going to be highlighting all the tools in that one. So please register for that if you're interested.
Next question. I see many customers using Hide when null functionality on calculated dimension columns. Could you address performance concerns or benefits of using this approach versus Set Analysis?
It's an interesting question. I hmm… I'm going to take a stab, and then, Ben, if if you if you want to, you know.
Keep me on the rails here. Um… Ultimately, I don't think there's going to be much of a difference. I would always… I prefer to use Set Analysis when I can.
I think…
The nice thing about the the high column condition on a table is that and and you know you can see this yourself in in just, you know, kind of normal usage.
There's a legitimate performance benefit there, for sure, right? So I think this is probably very use case dependent. I'm actually trying to think of, like, where would I.
be using this versus Set Analysis. I think it's going to be.
a little bit of six and one half dozen of another. I'm not sure that there's going to be a huge difference here… difference here.
Oh, I see.
Yeah, okay. Yeah.
Yeah, I think it's very use case dependent. I haven't seen a comparison of the 2 techniques. I mean, so if your data naturally has, um… If your data naturally has null values that you're suppressing right? But I think in the chat we might have an example of where it's a calculated dimension. You know, I think in that particular case, you're going to see poor performance with the calculated dimension using the hide.
Yes.
functionality as opposed to Set Analysis. Set Analysis, you're going to see, you know, higher… greater performance, and then also, from an interaction and kind of best practice perspective when you're talking about interaction selections, right? If you have a calculated dimension.
You know, it can cause some issues with how the users then interacting with the data when they're making selections.
Eamonn, I don't know if you have anything else you want to add.
No, I totally endorse what you just said, and I take back my ambivalence, and now say that Set Analysis is much better. Thank you, thank you to the person asking the question for kind of specifying an example there. I would, yeah, unless I really, really had to do something like that, I would not.
I'd much prefer Set Analysis.
Great. Our next question, is there a preferred or better performing data model pattern when calculating as of date filtering? They want to use multiple hysterized fact tables.
Yeah, data modeling again.
Yeah, so as always with data modeling, at least in my experience right is a lot of times as BI developers.
We don't necessarily have a lot of control over the source systems, right? So then we have to make decisions based on that reality. So in an ideal world, you know, this type of as of.
challenge would really be met with, you know, a solution at the source. So your source table would be tracking changes to transactions via.
offsetting transactions, right? So, you know, if you have a transaction of July , , right? And there's an adjustment to, you know, the unit cost, or the price, or whatever that fact was as of, you know, July , .
There's an adjustment maybe in , right? That adjustment would be an additional fact or transaction right within that fact table.
Now, in reality, what happens a lot of times is the transactions themselves are changed and there's no record of those transactions changing in the fact, which, you know, creates this challenge that this individual is probably alluding to.
So you know, there are multiple ways of approaching this from a data modeling perspective.
Okay.
potentially one of the easier ways of approaching this is just having snapshots. So you're taking snapshots of your fact and dimension tables. You know, one approach would be just taking a snapshot of your entire data model.
on certain on a certain schedule, right? Daily, weekly, monthly, right? And then you're essentially just concatenating those snapshots together, and then having a snapshot date column that you can use on the front end to do your as of filtering right? So, you know, you're defaulting to today's data.
But if you wanted to look back in time as of, you know, a previous date, you can do that, and it all works via snapshots that are concatenated together. You can take different approaches to that, right?
kind of mirroring your type 2 slowly changing dimension approach, but doing that with a fact table.
Where you basically would have transactions anytime a transaction changes, you would have a… two day columns, right? A start and an end, or, you know, start and an effective date, that type of approach with each transaction.
And then you would be able to use those two date columns to do your as of filtering. That type 2 approach can potentially be a better or more performant approach when you're talking about large data volumes, because the periodic snapshots, obviously, are just going to continue to grow across time, unless you're going back and cleaning them up and dropping older snapshots.
Um, and there's a lot of redundant data, right? What percentage of your data is actually changing from one snapshot to another? It might not be that large, so of a percentage, right? So you can really increase that data volume, whereas that type .
approach is more optimized, where you're only bringing in additional records when there's a change to that historical transaction. So just 2 different approaches there.
Alright, I… I feel like our next question is got a similar answer.
Next question is, I have a question around loading multiple files in a loop. If I reload all 40 files in a loop, which usually don't update all the time. Is there a way I can load only the updated files.
and update the existing table with the new data instead of reloading the entire data again.
And again, making my load time huge. I was trying to when I was reading that while you were giving your answer, I was starting to think about, like, change data capture and the whole, like.
data integration side of Qlik Cloud. And the advantages with that sort of setup, but… I don't know.
Yeah.
So there's definitely that side of it. Of course, that, like, you know, kind of strays into other areas of, you know, it's like other products that Qlik sells. But that is absolutely a great call out like, Hey, we've got we've got other products that, uh…
They're designed for that. Yeah.
you know, kind of solve this… solve this natively. I'm trying to think, and I'm just gonna think out loud for a second here. And Ben, I wonder.
Okay.
if there… because obviously the way that you would do this, right, is you would look at the file and check the last update date of the file, and then say, if the last update date of the file is greater than the last time that this application ran, then use the file otherwise don't.
There must be, but I'm not sure off the top of my head if there's a way in the Qlik load script to check the update date of a file.
That might be a.
One of our audience members is suggesting the file time function.
Yeah.
Oh, there you go. Well, okay, so… This is this is in relation to if you're in the cloud. But there is a file time function that will tell you exactly that right? So if I'm storing my QVDs or even my flat files, you know, whatever the case might be in the cloud.
There's a file time function, and then you can sort of just use the logic that I just described, right? Create a variable in your app that stores the last kind of complete reload time, and then the next time you reload using that file time function versus whatever value was stored in that variable to just decide, okay, do I bring it in or not?
Nice. Great. Next question. Does applying section access to a document affect its performance?
I can take this one in. So the bottom line is with section access, you have two considerations. The first consideration is going to be on load. So when a user clicks to open up an application, right? When a user clicks to open up an application.
that you're basically your section access logic is then executed at that point in time, right? So the app data is reduced at that point in time. So, uh, the initial load when you're using section access.
will take a marginally longer time to load the app than without section access. But after you've then loaded the app as the end user, you can see potentially significantly.
faster response times, if you will, depending on the specific use case, right? So let's say you have a particularly large application. Let's just large on the smaller side. So maybe 5 gigabytes, right? If you have 5 gig.
application, and you have said analysis, say, for a sales team that's, you know, dispersed globally, right? You might reduce your data set for a given salesperson by 90%, right? So you're going to potentially see significant performance, uh.
impacts on the front end UI after the individual user has opened up the application, opened up a sheet, right? Because they're only seeing, you know, 10% of the data in their case.
Yeah, all the charts will load a lot faster, it'll move faster once you're in there, because it's only… Yeah, only loading a portion of the data. That makes sense.
Exactly.
Great. Next question. Can removing sheets from my application improve performance at all, or is the performance more tied to the overall data consumption usage within the app? So I guess they're asking if they have.
a ton of sheets. Does that affect performance, or…
So I think in the way that this question was asked, the answer is no.
If I said, for example, this application is five gigabytes in RAM, and then this application had 10 sheets, or this application had 1,000 sheets, there's really no difference there.
Um, the one thing that is, of course, worth mentioning is, you know, like, you could have a single sheet that it itself is, you know, poorly constructed, or whatever the case might be, and it performs poorly. But if you have an application that, in general.
is in need of optimization. Really saying, hey, there are too many sheets. I need to remove a lot of them.
Is in itself a performance increase so much as you might do it for, like.
cleanliness or aesthetic, you know, like, I know when I was a customer, um.
we had a lot of self-service. Some said too many… too much self-service, and that was… that was often a problem in air quotes, like, you know, this application has a thousand community sheets, right? So you might want to do some cleanup there for.
sake of organization, but in terms of peer performance itself, there would be no difference.
Or no meaningful difference, at least.
And I was just thinking, like, the performance evaluation tool in Qlik Cloud is an excellent tool for identifying if you have a sheet.
That might be dragging your app down. And that was it.
Absolutely, yeah, yeah. If you have a, uh, it's a great callback to that tool and, you know, one of the things that you can do in the cloud. But absolutely, yeah, if you do have a single sheet that's… you know, really, really kind of an outlier. The performance evaluation tool is a great way to A, see that quickly, and B kind of quantify exactly what's going on.
Yeah. And next question.
Oh, it seems like we may have addressed it regarding the.
Does tag field hidden, which makes it field data not searchable, reduce the size of the app RAM explosion, or improve and improve overall performance. We've kind of… talked about that, right?
Do you want to address it?
Okay.
We might have hit on it a little bit. I think in terms of in that vein of an approach, right? Similar approach. High prefix isn't necessarily going to impact the performance of the application.
But there's another approach using search exclude statement. So you basically are excluding fields from being indexed, um.
Hmm.
From, you know, the search index, right? And when you're talking about performance from a reload perspective, you would then see more performant or faster reloads because you're not indexing as many fields or all the fields within your data model.
Right? And then, you know, you're marginally decreasing the size of the overall index. So you have to the point of, you know, RAM, you know, you are decrease, slightly decreasing the memory.
footprint there. as well.
Great. Next question is back to QVD fields.
Given compression of parquet files, is there advantage over QVD files?
It's an interesting question. So I did a little bit of testing with this myself. You know, I don't know, 6 months or a year ago.
Um. Two… two things. So Parquet files in my testing were more compressed.
but had slower read speeds. The difference in compression was not dramatic. I couldn't recall the exact numbers, but I think we're talking about something on the order of.
You know, maybe %, something like that. And then the read speed as well was not too terribly different, but slower. So it's a little bit of a balancing act in terms of, you know.
Which do you prefer? The other thing that's worth mentioning, of course, is that.
you know, let's say you're storing your QVD layer, or in this case, parquet file layer in S, and then you wanted to use it somewhere else, and that sort of thing.
Parquet files obviously have the ability of or have the advantage, rather, of, you know, being readable by other systems versus a QVD file.
there might be some reason to, uh… to use Parquet for that reason. But if we're limiting our our kind of focus here to just within the bounds of Qlik Sense analytics. It's sort of, you know, gain a little bit here, lose a little bit there, not not a huge difference.
Okay. That's fair. Next question, how to monitor Qlik consumption. And I guess we're talking about Qlik capacity.
Yep, so for Qlik Cloud Analytics or capacity-based customers, there's several options here. So, you know, the the.
best options, right, are within your administrative activity center on the home page. You'll have all of your capacity metrics right? For a month, or for whatever period those are measured on. You can see your usage against those metrics.
on that page. And then also within the administrative activity center, you can then distribute your detailed consumption reports. You know, internally to other admins or to whoever you want to provide access to.
Um, those detailed consumption reports within your your tenant. So the detailed consumption report is an actual Qlik app, right? So you can use space permissions and things like that, just like as you would with any other app to provide users with access to that.
So in addition to both of those resources, you also have the Qlik Cloud monitoring apps, which Eamonn went into during this month's tech expert talk a little bit. And within those monitoring applications, you know, you can get a little bit more into the weeds with.
Yeah, there's a lot there.
consumption metrics, as well as other non consumption type metrics as well, just in terms of like usage and things. So definitely take a look at those. Troy has the community post up on.
loading the Qlik Cloud monitoring apps.
And basically, it's an from Qlik Cloud. It's an automation. There's a template. If you just search for monitoring apps under automations, you'll find it.
Great. Next question. Oh, it sounds similar to one we've had before. Are there any times when loading directly from the source is a the better option as opposed to a QVD.
kind of address that. But do you want to recap quickly?
Yeah, I would say the only time that loading from the sources is better.
is when you're only going to use that table.
in a single application. As soon as you start using that table, whatever that table is, in more than one application.
storing it in a QVD or in some sort of flat file first is almost always to your benefit, whether that be for purely performance reasons, right? And remember, we're talking about just load time performance, or if you're in a capacity environment, you know, how can I sort of minimize my total capacity usage, right?
loading from the source and then storing in a flat file is going to be more efficient than loading from the source twice, or even worse, three, even worse, four times, right? So that's really… Those are, you know, kind of like, those are the broad principles. The last thing I'll say is that, you know, once again, and it's an important point to stress in terms of app performance. Once the data is in the app, it all performs the same. So there's really no difference there. But.
you know, sort of making your loading routines as efficient as possible.
really only makes sense if you're using that table just once.
Great. Thanks. Next question. Is there a way to allow users to select blanks on straight tables?
It's a little more of an app development question, but…
Yeah, I mean, but this can play into performance, too. I think, you know, and in terms of blanks, I'm assuming questioner means null values, right? So null values can simply select null values within a table.
And, you know, Eamonn, check me on this one. But, you know, I've the approach that I would suggest is handling your null values in the load script, right? Which is also going to potentially have performance impacts if.
doing that transformation handling in the load script as opposed to the front end. So, you know, oftentimes null values from a business perspective.
actually have some type of meeting, right? So what I would suggest is, you know, when I say handling the null values, right? So for a given field, right? If a value is null, so writing an if expression, if a value is null, then, you know, that means no sale, or whatever that means from a business perspective.
Um, replace that null value with an actual value that reflects what that null value truly means on the business side. And then, obviously, from there, you have a value that is selectable on the front end.
Great. Thank you. Next question.
Creating too many security rules or improper security rules, could that affect the application or dashboard performance?
Yeah, it's a good question. So I'll start with too many security rules. I I suppose.
sort of like in an extreme scenario. I've got.
security table with hundreds of millions of rows or something. Like, that would definitely have some performance implications, because ultimately you're loading the security table in memory as well.
Um… For practical purposes.
really, you know, the security table itself isn't going to be too terribly impactful. You know, if it's a difference in sort of logic between.
Every user is in section access for a single row, or every user is in section access with, you know, 100 rows.
once again, assuming that we're not talking about, you know, an extremist, this section access table has 100 million records or a billion records, then really, there shouldn't be too much of an issue there.
If you have improper security rules, I suppose it's.
depends on what improper really means here, but my biggest concern would not be performance. My biggest concern would just be, hey, are these users actually getting the correct data sets, right? Um… But you know, for the most part, section access should have, you know, as it relates to performance. You know you it might be a little bit slower loading an application initially, right? Because the section access has to be applied. But once that section access is applied, then the application itself is.
usually faster than an application without section access, because I'm only giving you a smaller set of data.
Right.
So I think… Uh, that's a… that's a ramble, but in summation, unless the security table we're talking about is sort of, like, um, comically large, I think you should be okay.
Great. Next question. When does ODAG or On-Demand App Generation become necessary over a well-optimized QVD model?
Eamonn, you want to think? Yeah.
I can take a stab at that. Yeah. So it's a really. I like that question a lot, and I'm gonna. You're not going to like my answer.
Right. So… ODAG becomes necessary, I think.
you know, functionally in two ways, right? The first is I've crossed some threshold over which it is now functionally and technically necessary, right? In other words.
When it gets too big and the users complain.
Ah.
Um, like if we're talking exactly like, well, so that's the second one. The first one is just, Hey, I've got a five gigabyte or a 10 gigabyte app size limit and without summarization and ODAG, I just can't fit all the data in here. So I need to have some other strategy.
Okay.
But then, Troy, that's exactly where I was going. The second way that this becomes necessary is really just the users think this app is too slow now, right? I think ODAG is a great, great solution for.
you know, basically any time where where the use case allows it it to be, right? So things like sales analysis or inventory analysis, or financial use cases, right? Like, really almost all the time you could be using ODAG. But.
Yeah.
To me, it's always a scenario where it's certainly more convenient to not use it, right? From a development perspective, from a user perspective at times. So, really, the drivers for me would be, hey, technically I've crossed the line where I don't have any choice but to start using alternatives like ODAG.
or be more realistically, the users say that the app is slow, and it's time to… you know, I've kind of… I've squeezed all the juice I can out of the regular optimization. It's time to go down an alternative road and start thinking about ODAG.
All right, we're in the last 15 min. We'll see if we can get through all these questions. A next question.
Would you recommend using a default bookmark or action if you want to enforce certain filters like current year and month?
Yeah.
So this might be getting into next month's texture talk. But I'll answer it anyway. You know, I think in this case, it can often depend on.
Your users, their… how comfortable they are with Qlik, um, and your design preference, right? So, the benefit of using default bookmarks or actions is that those are selections, right, that are visible to the end user.
Right.
So they… if they understand how Qlik works, they understand the selections bar, they can see that, right? It often, when you're talking about default selections and default logic, right? Using bookmarks actions is a great approach.
Alternatively, right, you can use Set Analysis and default your dates and things like that using said analysis logic with dynamic variables and things like that.
Sometimes that's not preferred, right? Because then that logic's a little bit hidden from the end user if you're not using appropriate labeling and things like that. But that's certainly another approach that you can take. It's really up to you, and potentially something you will be getting into next week or next month.
Yeah. I'm sure this question will come up, but they'll.
Great. Next question. If we avoid multiple concurrent users, will that help in the performance improvements?
This question came up in the Texpert Talk.
Yeah, um, so it's an interesting question. The answer, I think, depends a little bit on.
what sort of Qlik deployment you have, right? If you're deploying on premise, then there is always this.
Right.
you know, always this reality of, hey, I only have so much server resources available to me.
One of the nice things about the cloud is that it will horizontally scale to your sort of requirements, right? And I think this question gets at a real.
kind of, uh… base reality of BI development, which is sometimes you're hindered by your own success, right? I've got this application, but I only have a finite amount of server resources, so therefore, I hope it's successful enough, but not too successful that it slows everything down, right? And I would just point to.
point to the cloud as a good sort of maybe thing to think about if if you find yourself in that kind of situation or mindset where you're really, you know, thinking about.
Yeah.
Would it be better if not as many people use this application, right? Because ultimately, that's that's sort of what's being said here. So the direct answer to the question is. Yeah, if you know, if you're on premise, you know, the less people hitting a server, the better. But that better has a question mark at the end of it, because.
Right, it… Exactly. Exactly. Exactly.
Yeah, it better performs, but ultimately, you want people using the product if it's helping them, so… Yeah, that successful app. But as you pointed out in the text for talk and just now touched on it like Qlik cloud solves that problem for you because it will dynamically give you the resources you need when you have multiple concurrent users.
Yep. Yep.
Next question. Creating duplicate security rules or improper security rules. Oh, yeah, we had that question. Thank you very much. That was pre-submitted. We got that one. All right. That was all about security rules.
Next question. In a published app. We used to see our trace lines on a load when reloading in the app, line by line, showing what QVDs had loaded in the reload summary.
Now the reload summary ignores all trace statements and just gives a total of rows loaded.
Why did we lose the detailed reload summary?
Amon, do you know if there's been changes with that, or.
I don't know the details of that.
You know, I I don't. I don't. I this this sounds, um… This sounds pretty familiar to me. I am on my other monitor going to look at the load history of an app I've got right now.
Though my recollection of this issue was a kind of.
where the logs get, like, it was basically, like, you have to have a couple more clicks to get the download… to get the detailed log, uh, but I'm opening up a log right now, and I see… Let's see.
No, I mean, it still looks… virtually the same.
Yeah, so I think there is…
Yeah.
Although I don't have any trace statements in here, to be fair. I'm not… I… I don't know if you want to try this one, Ben. I think I would just be speculating without looking into this more myself right now, unfortunately.
Yeah, I think we'll need to… potentially follow up if we can, Troy, on that, that one.
Okay, that's fair. That was a very specific one.
Next question. I have found performance issues with section access in the load. It loads faster when you put the section axis on the last sheet than it does when it loads on the first sheet.
You guys have any familiarity with that?
I hadn't noticed that, but…
So yeah, that's interesting. Yeah. So so I don't know about you, Ben. My like my sort of.
Standard operating procedure when writing load script is to do the same. I always do section access.
as the very last thing. Um… I don't know that… It should affect performance, but I also wouldn't, you know, be like, hey, your experience is incorrect. There's no difference. If you're saying there's a difference, I'm sure there is. I think loading section access as the last thing is just a good practice anyway.
Um, it makes sense to me, often the section access that I'm building kind of is reliant upon, okay, first get all the different combinations from tables that already exist, you know, that sort of thing. So I think it's… I think it's good practice regardless.
It's not something that I had run across myself, but it's something that I will keep an eye out for. But I would say, like I said, I would say keep keep doing it last anyway. To me, that makes the most sense.
Yeah, I would agree. I mean, from a readability perspective, at least from, you know, reading a script in the order in which it's executed and flows, right? I think it it makes sense to do that, but… I have nothing to add. I've never tested this personally.
There you go. All right. Next question. How do we calculate capacity planning for expected user load?
I've heard that there's some. Yeah, go ahead.
So… I would have to imagine that this is related to on-premise Qlik sense, because other or QlikView, I suppose, because otherwise, like.
Yeah.
Hey, the answer is you don't have to. Good news. Um… because this is all about, yeah, how many EFI 500 users at once versus , users at once. This is very, very specific to what you're doing. I would encourage you to reach out to your account team and maybe talk to someone about, hey, these are the apps that we're running. These are how large the apps are.
This is how many people were looking to… to have it once, that sort of thing.
those kinds of, like, you know, those general variables that I just outlined, right? Particularly how many apps you're talking about, how many concurrent users, how many of these different apps are open at once.
Um, those are the things that they'd want to talk about and then get you some sort of rough estimates on.
you know, total server capacity that you would need.
Great. Next question. Is there an easier way to change the order of columns in a straight table?
Currently, it's quite difficult to rearrange them.
So I would have to go through and… test this myself. I know we've had some recent quality of life improvements, if you will, around adding.
fields to the straight table. But I don't know if we have any other changes in mind to make this particular task easier.
So if you use. Eamonn, do you?
Yeah, so I think… I think we're getting help from someone in the chat, which is greatly appreciated. They're telling us that if you use Chart Exploration, you can drag to rearrange the columns. That would be, like, my default way of doing this. My assumption is based on the wording of the question.
That the person asking the question thinks that that is not a good way to do it, or perhaps there are too many columns in the table.
to… for that to be reasonable or or something like that. But that is absolutely the way I would do it. And certainly.
Yeah.
Yeah.
I think the only way you can do it now if you're not the author of the table, and even if you are, it's virtually the same thing, just by hitting the edit button first. So, you know, whether it's through the exploration tool or whether it's through the edit menu, just dragging those columns around and then, you know, hitting apply to lock in that.
that ordering.
Great. And next question. What's the best approach to use the AGGR function, aggregate.
When filter on those fields, like all AGGR field are selected.
Way to only select the field inclusive of AGGR Logic.
I'm not sure what that's asking.
Yeah, I'm not sure. You know what? It might be good to bring that question to the next Q&A in a couple of weeks, where we're going to have some people who are really good at expressions and diving into the details of front down stuff. But.
Yeah, I would just suggest a lot of times, at least for me personally, I kind of need to see an example right to understand how to walk you through the question. So if you are going to bring that to the next Techspert Talk.
Yeah.
Um, I'd suggest, you know…
And I would also like. Plug the forums on Qlik Community because we have, you know, over a quarter million.
Members on here, so if you have a question that's pretty specific, put it in here, and you'll probably get some feedback fairly quickly.
So that's a great resource at least. Next question. I only got a few minutes left. Are we currently provide a self-service dashboard and have enabled the work stream feature for our users. As a result, users are creating a large number of sheets.
Which is causing the total sheet count on the overall app size to grow rapidly.
Is there a way to control or restrict how many sheets a user can create? Oh, that's an interesting question.
I don't believe there is a way to like you certainly there's a way to restrict users creating sheets. But you know, in a binary fashion, right? You can or you can't.
Mm-hmm.
I don't believe that there's a way to cap the number of sheets a user can create. You know, it's like, hey, you can do 5 sheets only, that sort of thing. The one thing… Yeah, that's… that would actually be a really good ideation.
There might be something for ideation.
as, uh, as Troy's showing on the screen here, as I'm sure most people are aware, this is this is where you can come in and submit.
or vote for and kind of give attention to ideas for product enhancements. The one thing I would say is that, as I as a kind of, uh.
mentioned in a previous answer. The size of the app is really not affected by.
Uh, the number of sheets, right? The sheets are really just metadata. The size of the app, in terms of if we're talking about size as far as, like, a base RAM size is concerned, that's really the data model itself.
Now, that's not to say that too many sheets isn't a problem, because as I was also saying before, it can become like a organizational and aesthetic issue where I open this app up and I have no idea how to find the sheet I'm looking for because there's a thousand sheets in the community section.
So I wouldn't be too concerned about the size of the app as it relates to RAM, but yeah, definitely something for the ideation forum, I think.
Well, see if we can squeeze in. A couple more questions here. Can data islands decrease performance if they're used to solely toggle different variables, KPIs based on selections?
One solution is to replace them with a variable input object, however, this cannot be fully styled on an on-premise server as of May 2024.
Yes. So so, and Ben, keep me honest here.
Data islands can decrease performance, because every time you make a new selection on the data island, you're you're basically.
telling the engine to recalculate because it doesn't know that it's any different from any other kind of field selection. So islands are, as the questioner is kind of alluding to, extraordinarily useful for making kind of, you know, selection boxes for users that they can.
you know, control the sheet. But they can become.
Um… detrimental to performance. I can't speak right now to the styling aspect of that question as it relates to variables, but I would agree that a variable input is a kind of technically more efficient and will give you a better performance than an island.
Great. And last question we have, does using variables within Set Analysis statements impact performance on pivot tables specifically?
Yeah.
So I'm assuming this is implying a negative impact performance potentially within pivot tables and I nothing comes to mind as to why that scenario would.
You know, using variables within Set Analysis would impact performance within pivot table specifically.
I think I… the only… the only way I think it would be truly detrimental would be.
Um, you know, it's like what's in the variable, right? If the variable is a big nested if statement.
That's… that's gonna be that's going to be a negative. But, um… you know, sort of like to the root of the question, the difference between a variable and not a variable, I don't think it should be really any impact.
All right, guys. Well, we are out of time. Thank you so much for all the wonderful questions we've had today. Thank you for joining us. Thanks to this great panel. Eamonn, Ben. This has been wonderful, and hope you all have a great rest of your day.
Thanks everybody.