Below you can find an overview of the available connectors with linked 'getting started' articles. For some connectors, you will find on the community more in-depth articles and examples that you can easily upload to your tenant (How to upload and download automations )
Hello everyone and welcome to the October edition of Techspert Talks! I’m Troy Raney and I’ll be your host for today's session. Today's presentation is exploring Qlik Application Automation with Jeffrey Goldberg. Jeff why don't you tell us a little bit about yourself?
Thanks Troy. For those of you who don't know me, I’ve been with Qlik now for almost two years on my second tour. I worked for Qlik for four and a half years in pre-sales. I focus now as a product manager. I’m certainly focused on Qlik Application Automation, but I’m really also focused on APIs, integration, identity management and anything that really has to do with enhancing the platform either through extensible components, extensions like visualization extensions, and the APIs that support those things. So, that's why Application Automation kind of falls into my realm because a lot of this is; while it's a no code tool which we'll talk about in a minute; is all driven by the APIs that are available in our platform; as well as other cloud platforms.
All right. Today, we're going to be talking about what Qlik Application Automation does; how it works; we'll get to see a little bit of a demo there; some best practices; and really we're going to be focusing a bit on how to troubleshoot common issues and what to do when things don't go as expected. Jeff, for those of us who aren't familiar, can you tell us a little bit more about what Application Automation can do from a high level overview?
Absolutely! The thing about Qlik Application Automation is it's not just a feature. It's actually its own unique product within the cloud platform; which is called an Integration Platform as a Service. And we've now integrated into Qlik Cloud. And what it enables you to do is: leverage analytics as part of an application integration experience. When we looked at Blendr, and brought it in and made it Qlik Application Automation, what we really wanted to do is: we talk about Active Intelligence, but how do we make it real? How do we help accelerate decision making and make it so that people can reach that state of Active Intelligence without having to do manual work?
So, it's kind of replacing a lot of scripting and tasks with visual blocks, so you can kind of see how it's mapped out.
Yeah. So, it's a great way of thinking about it. It's basically a visual way of scripting your daily workflow. There really hasn't been a good way without being a programmer to connect different applications that you use every day. So, you know using Hubspot and Salesforce to migrate or move contacts between the two systems. And you need to make sure that data stays in sync between the two of them.
This is where Application Automation can come into play, because it can take care of doing that work for you without having to involve someone from the IT organization or having to know how to code in order to facilitate that that kind of experience.
And what are some popular use cases for it?
A lot of what people are doing right now, since we've released Qlik Application Automation is analytics automation and the analytics operations, right? Sometimes this is called Passive Intelligence.
So, there's really like four different types of use cases. What we want to get people to is that state of Dynamic Actions, right? Where you're leveraging analytics that drive data decision-making in an automated way, but recognizing that there's a good amount of upfront work that has to be done in order to make sure that you're processing that data and getting it to a state where you can go ahead and look at dimensions and measures, perform selections on that data using the blocks that then allow you to do the evaluations which enable you to then trigger the actions in the downstream business applications you're integrating with.
So, based on the data you get to actually trigger things and initiate workflows?
Cool. How should someone get started using Qlik Application Automation?
Yeah. So, let's just jump right in, right? So, I’m on Qlik Cloud, and I am in the Analytics area of the platform. So, when I want to start building automations, I can just go to Add New, and choose New Automation. And then I can…
Give it a name.
And that brings you right into the Automation Editor. And the Automation Editor has three different sections. There's the Block Menu, which is on the left and you can see that there's a number of different connectors and block types there. So, we have basic blocks lists. We have some advanced blocks cloud storage blocks and then we have connectors and you can see it's showing the connectors that I use most often in my other automations.
Yeah, that's cool.
There's a number of other connectors down below that you can choose from. And we're adding new connectors on a monthly basis. And then when you want to access those connectors, you can just click inside, and that will bring you to the actual blocks that are available. And you can search for blocks through the search menu or you can use these different tags that are up at the top to filter the list and get started.
What's the center called?
The middle is the canvas. This is where you actually drag and drop the blocks. And then, when we drag and drop the blocks, on the right hand side, that's the block configurator.
Okay. Can you walk us through setting up a reload automation?
Yep. Type “Do” in the search, and I drag and drop Do Reload. First things first, you see the blue pulsing square? That just shows that when I let go of the mouse button, it's automatically going to attach to that point in the flow.
And the details of that on the right?
Yep. Now, because I’m using the Qlik connector here, I’m already authenticated to my Qlik Cloud tenant, so there's no need to provide any credentials.
I can go right into, and I can search for my apps here. I have this app called TC. You can see that I have the app grid here. And then I can say I want to wait for this reload to complete, or I can set it to run asynchronously. If I set it to run asynchronously, then what I can do is add another Do Reload. And I can go over to here and do another lookup, and I can look for my second app. Oops. I can do, I do look up and I can look for my second app. What happened just now is: you see that red box that's around App ID? The reason why that turned red is: I started typing and then I deleted. It's basically telling me: I can't run this flow without supplying an App ID. What's nice about the platform is it actually tells you intuitively when you've got errors in your configuration that you need to work through; and we'll show a couple more of those.
I actually love that part, that it kind of tells you right away the things that are missing.
It's really nice, right?
Before it creates an error. Yeah.
Yeah. So, I’m going to also do a start reload and continue here. And what this is going to do is: it's actually going to kick off the reloads for these two applications at the same time. And there's another block inside of here called Wait for Reloads to Complete. And you use this block to basically say “I want to wait for these two reloads to complete before I do anything else,” right? Another way that you could do it though; you could go back and; one of the things that's nice is: in the canvas you can just drag and drop. And whatever is not in the sequence doesn't get run. I can go ahead and I can go into the Basic Blocks here, and I can add a Condition.
And if I switch this Do Reload to wait for the reload to complete, then I can go into the Condition. I can choose from the input, you'll see that there's the output from the Do Reload ready to use as data that I can use to drive the continued execution of the flow. So, in this case, that output from the Do Reload, you can see there's the App ID, and there's also the status, right? So, if the status equals Succeeded, that's when we want to run the second reload. And we'll also wait for this reload to complete. And if I want to I can add another Condition here to do something else or send a message to Microsoft Teams. Drop that in here. And I can also drop another one here at the bottom, and I can even change the flow, right? So, the editor is really nice and flexible. Let's say you want to just grab a hold of a whole chunk of an automation. Wherever, whatever block you click, and you disconnect from the flow. It's going to take all of the blocks below it.
That's a nice tip for how to use the canvas and mix it up.
So, once it's connected, how does the send message work?
Go into here, and you'll notice I already have a connection to Microsoft Teams, but I’m going to delete that and you can see immediately I already have this…
And if I go ahead and I try to save this, it's basically throwing (before I even can save) an Invalid Block message. And it's telling me: you've got a problem with this block. This block doesn't have these fields filled out; because you don't have these fields filled out, it's not going to work. Now, I can go ahead and save it anyway.
It's nice to have that option.
Yeah. It's nice, right? But if I go and I go and run, and then you're going to get this error. And it's basically going to say “oh, I couldn't find Microsoft Teams, not either never existed or may have been deleted.” What you need to make sure that you do is: first you need to make sure that you have a connection for Teams. Specifically, sometimes you have to have different permission sets. When you choose to link, it will pop-up this box. It will ask you to pick an account and then it will connect. So, now that that connection's all set, you can see that it's there I can go into the inputs. And you can see, I’ve got the red that basically is telling me: I need to fill this out.
It's looking for what profile to use in Teams, I guess?
Yep, exactly. It's looking for the different team that you want to send the message to.
And I want to send this to the output channel, and I want to say: the first reload completed successfully. Save that. And you can see, I still have an Invalid Block, because I have this Team's Message, but let's just go ahead and remove this one. And now you can see that it will save appropriately. And if I run this; if you go into the overview, you can see: what your last run was; you can see what your history is; and you can see we have our failed item. We have this one that's running, and once the automation finishes, you get this really great Per Block run. Which doesn't necessarily put the blocks in order, but you can go in and you can look at the execution; and you can see the information that was sent into the request, and then what came out of the request which is pretty cool when you're trying to troubleshoot different things that are going on.
Yeah. So, if something failed like one of the blocks, could you actually see where it broke down there?
Yep. Yep. So, what would happen is you would see in the Per Block if there was an error where it broke down, and you would also see that the error is here. But you're right; you could come back to the Per Block, or better yet you could go to the Chronologically tab. And what this will do is: it actually shows you the sequence of everything that was kicked off. So, and it will give you a preview as well. So, you can click on the drop down here, and you can also see the same information. So, I’d recommend using the Chronologically tab.
And can we take a look at what the message it's sent to you in Teams looks like?
Absolutely. I’ll move it over.
That's very simple, but yeah, it's nice to know what it looks like. Now, I understand there's a ton of possibilities with Teams and with Slack that we don't have time to go into today, but back to this reload: what can affect the performance of these reloads?
So, depending on the size of your applications, they may take a long time to reload depending on how much data you're actually reloading. But it could also be a result of the reload queue that sits inside of the cloud platform. Depending on when your request goes into that queue, that's when it's going to run. So, if you've got a lot of concurrent reloads happening at the same time, you may find that your load ends up running later on.
Okay, now I’ve heard the terminology and I’m hoping you could explain the difference between a task chain or automation and an orchestration?
It's a great question Troy. I actually don't see a whole lot of difference between a task chain per se and an orchestration.
But then it's about focusing on operational efficiency and productivity. There is aspects of data movement. It's really about moving small bits of transactional data in support of business processes. It's not about like bulk data movement. When people are talking about orchestration and orchestration of pipelines, they're talking about “we're moving data,” right? We're not just doing this. When we think about orchestration, right? It's directing automations within end-to-end processes, right? So, if you think of just an API call opposed to going to a screen and clicking three things in order to do something; it's very much event driven or dependency driven versus “oh, I’m just sick and tired of having to click five buttons at a time.” I think that's the way a lot of people think about the differences between the two things.
Now getting back to SaaS, suppose there are several automations built without those wonderful emails or Team messages/notifications. How can you view all of the automations have been created and see which ones might have issues?
Yep, great question. If we go out to the Management Console. Inside of the Management Console, you will see that there's an Automations tab. And you can go ahead and see from here all the automations that exist on your tenant. And you can see when the last run; whether the last run failed or not. So, if we go and say “oh we want to look at Icons.” So, let's back out here, go to the Catalog, Icons, right we can see here's our last run brings us right to the error. We can go into the Chronology. I only have one block in here. So, that's the, that's the issue. But I can then also look at the full history here. So, it's not that I have to go back to the Management Console; that's just one place that you can look at it from.
Okay, is there a way that users can manually trigger a flow from an app?
So, there are some different ways that people have done that. Using the Action Button can use the navigation to the website URL, and there the URL of the automation in there. And the way that you get that automation is by going into Run Mode here and choosing Triggered. There's an example here that represents the URL for this automation.
When you use that website URL, it pops up another page. Now let me just go into my copy of Consumer Sales app. This is the app I use to test with.
So, I understand you've been improving the Action Button, since it's open source.
I’m working on updating this button, so that instead of going to navigation and choosing ‘open a website or email’ you can actually go into the button here, and you can choose Execute Automation. And then you'll see the list of automations that exist in your tenant.
So, it's picking up all the automations you've already created in your tenant there?
It's picking all of those up and so then you can choose: “Oh. I want to do Hot Dog Alert.” And you're also going to be able to choose “Oh, I want to Include My Selections.” So, if I go ahead and I do this, and then I do this, and I press this button, and then I click on this button, and then I go out to my hub, go and look at my hot dog alert; you should see that it started, right? Now, because I’m just testing this because this is still something that I’m making sure works properly; all I’m doing is passing inputs and we'll have videos once it comes out. But you can see I’m passing into this automation the App ID and the Bookmark ID. When you do that selection check box, it creates a bookmark. And now you can go ahead and reference that bookmark inside of the automation for that App ID, and now you have the starting point of your flow. So you can start working with different charts and stuff like that inside of the application, and getting the data that you need.
That's really awesome.
Yeah. To be clear: not out yet.
What are some best practices or details that admins should be aware of when managing automations to avoid potential issues?
Yep. So, automations today will only run for a maximum amount of one hour.
You can't create a runaway automation that just runs forever. Now there are some other types of execution that have some potential consequences; and you want to make sure that when you're building your automations, you're thinking about it, right? In the case of something like this automation where we're using a triggered web hook, right? It's great that we're using a triggered webhook URL, but let's say I’m putting it someplace where it gets called a lot.
You're going to queue up a lot of automations, and they're all going to run. Now, automations don't run simultaneously. Automations always will run one at a time. The thing is if you have like a really small automation like this one which runs for like one second; and you are triggering this URL like crazy, then you're going to hit your threshold of available automations. Because the way that we offer this is a number of automation runs per month. So, as you're building your automations. you want to be cognizant of the use case.
Yeah, and let's address one issue you brought up there. Is Qlik Application Automation free?
There is a free tier for Qlik Sense Enterprise SaaS customers.
And Hybrid Data Delivery customers. The free tier gives you 5000 runs per month currently with the ability to run 5 automations simultaneously. That's what we call concurrency. But we do have 3 different offerings that you can purchase that expand your capacity. So, you can buy more what we call “automation runs.” So, you can go from 5,000 to 20,000 or 60,000 or a 100,000 and you can contact sales to get more information about the pricing of those.
That's great. Can you show us another example of what's possible with Qlik Application Automation?
I would love to. In fact, what I want to do is show how you can take advantage of analytics to compel action in other systems.
Let's say I’ve got a Qlik Sense app. I’m looking at sales data, and I’m also looking at this Account and Opportunities detail. You can see I’ve got these Marketo scores and I’ve got these Hug ranks. What the Marketo score is reflecting is the interest that someone has in the materials that we're sending them. What the Hug rank does is it shows how we (you know fictional company) are engaging with you.
So, a high Marketo score means a highly engaged customer, and the Hug score shows how well engaged the company is with that customer. So, this helps identify sales opportunities?
Exactly. I know that there's a certain threshold combination between the Marketo score and the Hug rank where it’d be cool if I could actually make automatically the sales person gets an activity inside of Salesforce that they have to act upon?
So, I see you've completed a flow would you walk us through how this works?
Yep, absolutely. I’m using an event which is App Reload Finish. So, I can go over to my Run Mode and I choose App Reload Finished. And then, the first thing I’m doing is I’m making sure that this is equal to the app that I’m looking at which is my Sales Pipeline Analysis which it is.
And I can start selecting Stage, and I’m looking for accounts and opportunities that are in champion stage.
Once I have that, I want to get the straight table data from this table. Now, I’m selecting that field value first is to reduce the amount of data that I have. The get straight table data only will bring back a 100,000 cells, right? And that's a limitation inside of Qlik Engine. You wanna definitely make sure you have an understanding of the selections you wanna make on the data. If you try to grab something that's going to have a million cells, automation's going to fail.
That's a good troubleshooting tip.
Once you have your straight table data, you add another Condition, right? And this is where you can add your Thresholds. So, in here I’m saying: well, if the Marketo scores equal to 110, and the hug rank is equal to 9 (and this is just for the sake of demo), I want to create a task. And inside of that task, there's specific fields that have to be filled out; like the What ID, the Subject of the task.
Yeah, all those fields and Salesforce that are necessary.
And you can see I can actually use data that I’m grabbing from the straight table in the message that I want to put inside of Salesforce. And then, I’ll also send a Hero Card to basically say “Hey, you've got these records. Do you want to go and look at them?” And we can actually embed the links so that they just click the button.
Yeah, that's so convenient.
Yeah. We have a playlist looking at how we construct this in real time. And you can see inside, there's this Active Intelligence Salesforce walkthrough.
That's a great resource for learning more. By the way, the link to this playlist and more will be included with the recording of this session.
And when that application reloads, it does all of this automatically now.
So, that lets people easily take action on those insights which means a lot less manual work. How does all look from the application side?
When this automation runs, it actually creates these activities right here and I can go into the task and I can basically see “Oh, Jeremy Webb from Tristique Associates has a Marketo score greater than 100, but a hug rank less than 15. Contact them and engage them about our new Application Automation capability and Qlik Cloud.”
And that was created by the automation?
Yep. And we can confirm that it worked because we can go into the automation here. And we can see this is what we sent. And then we can see here's the out, which is a Success=True.
Can we see again what that looks like in Teams?
You can see that it's created these items. There's that Active Intelligence. But one of the nice things is as you're actually building this there's ways that you can actually troubleshoot your flow by moving blocks around, right? So, if you do find yourself in a situation where you have to do some troubleshooting, you have a lot of flexibility in the editor to move blocks around to kind of troubleshoot what's going on.
I see running the test runs of individual blocks as well. That's really cool.
Yeah. Test blocks are really nice as well because they basically give you back sample data. So, like let's say: you're doing something for the first time. You don't want to have to do a full run. You can do a test run on a block based on information. So, if I go and I hit “Do this test run” here, what it will do is: it will run everything. So, let's go ahead and grab this out. You can see it didn't like my Condition because didn't get the App Reload Finished. So, we'll switch this back to Manual Mode, right? But if I do a test run, it will only run to the block that I’ve done the test run at.
So, in this particular case, it labels the output as a Test Run, and it basically says that it made the selection. And then, I can also do the test run here. It will go ahead and it will show “Oh, yeah okay. So, it brought back that test run information.”
Great! Now, it's time for Q&A. Everyone please submit your questions to the Q&A panel on the left side of your On24 console. First question: is there a function to allow developers to request application promotion from one environment to another?
Oh, that's a good question. We are getting a lot of - we are receiving a lot of feedback about things like “Oh, I want to do app promotion. I have two tenants and I want to basically say: this is where I do my development. This is where I do production.” Right? In a lot of cases, we have customers that use the same tenant, right? And they have different spaces. But in the event that you really do have two tenants, currently with Application Automation you notice (and I mentioned this at the beginning) when we first… Let me go back to the - you'll recall when we earlier - when I talked about the use of the Qlik Blocks. One of the nice things is: you don't have to worry about setting up your credentials because it's bound to the tenant that you're actually working with. Now, that's really nice because you don't have to do that; but what right now we don't have the ability for you to connect to another tenant like with an API key using the same set of blocks. So, what you can do is: you can use the call URL block (as an example). And say let's say: you're going to run this automation on your production tenant as the as a solution that's going to perform the app promotion, right? You could use the call URL block with an API key to contact that the development tenant; reference the app; export it to base; and then import it as base into the production tenant. So, that is one of the ways that you can do app promotion. We are looking at a number of different ways to streamline that process. Either through automation or through the core cloud UI. Stay tuned for those updates as time goes on, and we learn more about how our customers use Application Automation.
That's great. Next question was: what happens if a flow fails? Is there a way to trigger an alert email to the admin?
Yes, actually. If we look at this example right? So, we have succeeded, but if it didn't succeed you can easily go ahead and use something like the Mail Block, right ? If you wanted to send a mail (an email), you could use a Mail Block. And you do have to configure that with your own SMTP server. It doesn't use the SMTP server of what's in the Management Console. So, you do have to specify that again. But you could also send a message with Slack. You could send a message with Teams just like we did previously, right? I can copy this block and I can paste this block, and I can drag it into here.
It's great that there's so many options available for communicating.
Yeah, exactly. And that was one of the things we wanted to make sure of is; that there's so many different ways, right? You can use if you subscribe to Mailchimp, you can use Mailchimp. If you subscribe to SendGrid, you can use SendGrid to send messages. Out there's lots of different ways that you can go ahead and produce messages based on whether or not your automations are succeeding or failing. The other thing that you can do is: there is an Error Block, right? And when you put the Error Block in, this will actually create the message. And you can choose to stop the automation or continue the automation. And by doing that, that's what's going to show that will show up, right? If I come back out into the overview, if we look at things like the Chronologically or the Per Block, and you have an error; you can actually specify setting up that error.
That's kind of interesting. Next question: referencing Qlik Data Transfer can trigger a reload of an app. Is there any plan to trigger an automation instead when the data is synced?
I’m not sure about Qlik Data Transfer. We are working with the data integration team for Hybrid Data Delivery to create APIs and events that will turn into blocks inside of Application Automation that will respond to events based on data changes. We also are looking at when you're landing data inside of data files which is what the Qlik Data Transfer tool does. There should be APIs and events that that we can hook on to. So, I’d say stay tuned. Stay tuned for that.
Is it possible to write data to Qlik Cloud?
In terms of writing data to Qlik Cloud specifically; certainly you can go ahead and grab data from different data sources like SQL Server, like Hubspot, like any of these connectors. They're all producing data. You could do something like writing a load script manually inside of here because there is the possibility to set a load script inside of an app. But if you're looking to move like data files, currently there's no way to move data files in the Application Automation tooling similar to Qlik Data Transfer. That may be changing in the near future. It's all - it's a matter of making sure that the APIs are public and guidelines compliant. But there's numerous different ways you can bring data into the cloud through APIs. And you don't even need to use Qlik Blocks in order to do that, right? You can just go ahead and use APIs to create files or move files and move data around to different sources and targets if you're trying to integrate Application Automation to do non-Qlik things (which is a totally perfectly fine use case).
Where can I find an overview of all the automations I’ve created?
Yeah. So, in terms of the overview, right? The best place to go is the Hub. All automations that you create are in your personal space. So, if you go over to the Catalog, you can click on all filters and you can choose Automations, because automations are first-class citizens just like apps or your monitored charts. And you can see them and then you can go ahead and you can look at your list if you need to see what their performance is if you've got access to the Management Console. The Management Console gives you access to look at that list. We have had some feedback come from customers as well as the field saying it would be good to have a mechanism for supporting that in this UI as well. So, that people can see what they've got running and have more of an overview. So, something we're thinking about.
How can you tell which app the automation is for from the editor?
Ah, yeah. So, you really can't. Because automations aren't necessary (like I just said) they're first-class citizens, right? So, they're this they kind of operate at the same level as an app. They're not child object of an app like a chart or a story or anything like that. Because an automation doesn't even need to interact with Qlik Sense. It can interact with Hubspot, and Salesforce, or Marketo in Salesforce, or ServiceNow. Based on other information that's coming into the system (either through another data connection that you're making to) like a SQL data source or from another application in your Enterprise portfolio. So, it's important to understand that if you are trying to associate automations with the apps that they're tied to, your best bet is to go into the edit of an app, and add tags, right? So, you could add a tag that aligns this; like I’ll just align this with GMS. And now, if I go ahead and do a search on all things with tag GMS, you're going to see I’ve got copy of Consumer Sales and then I also have my first automation. And that would be…
That's a smart way to do it.
That would be the way that I would organize it.
Okay, last question: I’m looking for a way to send ‘task fails/success emails’ to specific users. Is there a way that can be done through automation?
Absolutely. If you go back to using with Slack and Teams a little harder because the way that they're structured, you typically will send information to channel. Although you can send in Slacks to (I believe) to specific users. If you're sending an email, it's just a matter of knowing the email address of the person that you want to send it to. And it could be that you're, let's say you've got Section Access inside of an app. If you leverage the Users Table, let's say you're an admin on that that app. You're going to see everything. I would use the Section Access table and use the email as your user identification field. And that way, you could pull that email address out; and you could send emails that way. Or you could have a list that you sideload either through a text file or Excel or a database should do that.
That's really smart. Well, thank you very much, Jeff. This has been great.
Troy, it's been an absolute pleasure to spend some time with you today and go over the platform. I know we showed a demonstration of some Active Intelligence, and then we also went over some different places where you're going to spot errors, and stuff like that. There is a tremendous amount of content that we've actually produced so far. You saw the YouTube channel. There's also our community. We have an Application Automation and integration forum on community. And then we also have a number of how-to articles so that you can learn how to use the different patterns and blocks, and you can go ahead and upload those patterns directly into the editor so that you can save yourself some time. Thanks for the time today to chat.
Thank you everyone. We hope you enjoyed this session. Thank you to Jeffrey for presenting. We appreciate getting experts like Jeff to share with us. Here's our legal disclaimer. And thank you once again. Have a great rest of your day.
Q: We have upgraded Qlik Sense from June 2020 Patch 4 to May 2021 Patch 5 , we have installed bundled version, PostgreSQL version 9.6. How to create separate DB node from bundled version and how to upgrade postgreSQL from 9.6 to 12.x version?
As always, my recommendation is to test this in a test environment before applying the process in Prod.
Q: Can we have more than one windows version in the Qlik Sense cluster nodes?
A: It should be technically possible but not recommended and perhaps not even supported. It is important to keep your cluster as identical as possible as there could be feature/settings difference between Operating System version causing inconsistent behavior.
Q: I tried an Update, but it hangs durring removal of older patch.
A: It is possible to manually remove the patch and then update to the newer patch. Sometimes is happens that for some reason the installer is not able to perfom this operation successfully.
Q: Do the streams and applications carry over to the new version?
A: Yes, they do along with all your settings.
Q: Hello, is there anything specific when upgrading a multi-node environment (in terms of installation and troubleshooting)?
A: In a multi node environment it is recommended to stop all nodes and start by upgrading the central node. If you are using the Qlik Sense failover feature, make sure which node is having the role of central node before stopping the services. Also, when you stop your nodes, always stop the rim nodes first and then the central node at the end.
Q: We are multi-node and multi-environment. Whenever upgrading, each node takes a fair amount of time, is this normal?
A: Yes, and this is most probably because of the dispatcher service which covers multiple subservices. In addition to that it is possible that a 3rd party software such as an antivirus is scanning each and every upgrade operation like copy files which as a result slows down the whole process.
Q: I get the error qlik notification service already exist. Support says its under investagation. Would upgrade to the a newer version fix this?
A: I wouldn’t recommend upgrading a non-working environment except if advised otherwise by the support team. This is a risk to make things worst.
Q: Is this the same process for QlikView Server?
A: While there are some similarities especially when it comes to the steps prior to the upgrade such as testing, system requirements and backup. However the actual process is different. There are articles and help documentations covering this such as: Upgrading and Migrating Best Practices for QlikView Services
Hello everyone and welcome to the September edition of Support Techspert Thursdays. I’m Troy Raney and I’ll be your host for today's session. Today's presentation is Troubleshooting Qlik Sense Upgrades with Bastien Laugiero. Bastien, why don't you tell us a little bit about yourself?
Hi everybody. So, my name is Bastien Laugiero. I’ve been working at Qlik for almost six years now focusing primarily on Qlik Sense on-promise; and I recently moved to a new role as a program delivery manager focusing on our SaaS platform.
Okay, and what are we going to be talking about today?
So, today we're going to talk about Qlik Sense upgrade and troubleshooting approach. The first thing I would like to talk about is the requirement and preparation for a successful upgrade. Then we're going to go through a couple of demos: one that shows an upgrade failure and another one that shows a platform down after upgrade. So, basically a successful upgrade, but a platform that won't come up. And there's gonna be some takeaway at the end of the sessions, some articles being shared and so on.
Okay. So, when it comes upgrading Qlik Sense, what constitutes a successful upgrade?
There is never any guarantee that an upgrade is going to be successful, right? We are talking about software here. So, there are however key factors. It’s the amount of preparation we put prior to the upgrade. That's basically going to guarantee at least a recovery in a working state.
And what kind of preparation would you consider required for an upgrade?
A very simple step and often missed is to read the documentation, right? So, you can look at upgrading Qlik Sense and you will end up here. So, here the whole process is being described. Something that you really not want to miss is to look at the system requirements, the precautions, backup and so on. But everything is detailed on this page here.
Okay. Help is a great place to start. Where else can admins look for guidance?
Take a look at our Knowledge Base and Release Notes. So, this can be found in our Community. If you go under Community.com, you've got Support and Release Notes here. You can also search in our Knowledge Base. For instance: “Upgrade to Qlik Sense August 2021” and see if there is any results.
How do customers really know what version of Qlik Sense they should be upgrading to?
That's a good question. So, if you go to Qlik.com, if you go under Support and Download; you're going to end up in our Download site. Select your product “Qlik Sense” and you will have all the versions available.
So, the latest should be on top there?
Exactly. And that's what we see here on the screen, right? August 2021, and we've got the initial reason the patch one available.
And where can you find what's in the release, like features or fixes you might want?
For that, you can go to Help.Qlik.com, under “Qlik Sense on Windows” you're going to find this “What's New in Qlik Sense,” and you can select the version that you are targeting to upgrade, right? We're gonna talk about August 2021, and you've got a list of different features and so on.
Yeah, a lot of new features there.
The last thing I always recommend to do is: to look once again at the Release Notes, because the reason it contains the defect fixes that are available as well as new features, settings that needs to be enabled to enable potential features. So this one is very very rich in terms of information.
Okay, and if someone like with this version that's the latest, it already has a patch. Should we upgrade automatically to the latest patch? Is that possible?
So, no. We're going to take a quick example here. Let's say that: you want to upgrade from Qlik Sense May to Qlik Sense August patch 1. You cannot upgrade directly to August patch 1. You will need to first upgrade to the initial release, and then to the patch.
So, once you are on the initial release, so let's say August 2021, you don't need to upgrade to patch 1, patch 2, patch 3, patch 4. You can go directly to the latest patch.
But you need to start with the initial release first?
Now, I know that in IT in general, but especially when upgrading Qlik Sense, the most important step is backing up. What specifically should be backed up so that Qlik Sense can be restored if anything unexpected happens?
The first one will be the product backup, so Qlik Sense backup. You can go into our Help site. And here you've got the process to actually backup your Repository Database, important files. That's the first thing you need to do. We also provide some Qlik Fix video on how to do that step-by-step as well to follow here. The second thing that you can do is to take a server backup or virtual machine snapshots. This is often easier and faster to restore from this type of backup.
What kind of failures can occur when upgrading?
There are two type of failures in my opinion. The first one will be: so you get a pop-up that shows that your upgrade failed, and we're going to see that in the next demo. The second one is the installer says that you've upgraded successfully to target release; however, when you start up your services, you can't access the Hub, you can't access the QMC. So, the platform seems to be down even though the Windows services are up and running.
Okay, well, let's jump to the demo. What did you want to show us first?
So, here we have this virtual machine running Qlik Sense November 2020, and we're gonna basically upgrade it to Qlik Sense May 2021. As you can see here, I’m not going to jump to February in between. I’m going to go directly to May. I’ve been taking all the precautions that I was talking about before. I’ve checked my system requirement. I took my backup. Now I’m just gonna run the upgrade. Always run your installer as an Administrator.
And you've downloaded it locally and now you're just running it?
Exactly. Alright so, here we're gonna click on Yes, because we want to run the installer. Click on Upgrade.
I like that the installer recognizes the pre-installed version, that's cool.
Accept the terms. Service account information. And here, enter the Repository Database password for this demo. I’m going to just put the wrong password. Just a quick note on that: I’m able to make this upgrade fail, because I have stopped all the services. Normally, if the services are up when I’m actually running the upgrade, there will be a check of the super user password. And if it's wrong, the installer will tell you that. We want to install the latest bundles. We accept the terms for the bundles. And we're going to start the upgrade process. So, it's going to fail at the Qlik Sense Repository Database. Here, we've got our upgrade failure. So, in most cases, you're going to see actually a meaningful error message here that you can actually search then into our Knowledge Base. And we can just search for “Password Validation Failure.” We can just filter on “article only.” And here, you will end up with different articles related to that error message.
What if the error message wasn't so helpful or clear is that one? Was that a link to the log files from the upgrade?
Exactly. So, this is a link to the installation or upgrade log file. So, this is important, because if you have to reach out to a Support to help you troubleshooting this issue, they will often ask you for the installation or upgrade logs. So, these are the logs that we are expecting to receive. So, if you click on that link, you're gonna end up on this folder here. So, this was the logs that were created during my initial installation on November 2020. As you can see, it just gives us the release name here. So, there are a lot of different logs, and the reason is because Qlik Sense has multiple services to upgrade. And each service will be represented by a specific log file. So, this is the order it went through. So, the first one as you can see it's PostgreSQL, so Qlik Sense Repository Database, and then it will continue here. If it fails at any time, let's say it fails at the Scheduler; so, it's not going to go through the other services. However, you will see a rollback log file on top of each of the ones that were already upgraded. So, you're going to see this these five here, plus an additional five with a dash roll back at the end. So, what you - the log that you want to open is actually the latest one that don't that, don't have this rollback…
Exactly. If we go back to PostgreSQL. So, there was no rollback here, because it just failed at the password validation. So, it fails right at the start, so there was no rollback to perform. But this is basically the latest log that we've got from the installer. So, this is the one we're going to look at, right?
So, when you're looking at a log file what do you usually search for to zero in on what is important?
Yeah so, you can search for “Value 3.” So, “Value 3” it's an error code that will trigger a rollback basically. The other thing that you can search that is always associated with that return Value 3 is “Error Code 1603” here. If you start to read a little bit above that error, you can find back our Password Validation Failed error message. However, you may have other error messages here that you can search in our Knowledge Base.
Okay so, if it fails, you're not sure what the message means, you could theoretically copy that message that you find here in the log file, search in a Community, and hopefully find more information? But this is definitely a log file that Support would need if they if you require some additional assistance?
My advice here is that: when you send logs to Support, don't just select the latest log. Take all the logs that are associated to your upgrade basically.
And it's pretty easy to see which ones they are, because in this in this case, we're upgrading to a different version and it's designated there in the first part of the log which version wrapped it into.
Now quick question: since this upgrade failed, is the November 2020 still active and functioning on this system?
Yes. In fact, we can take a look at the services here.
So, it didn't really affect the current installation.
No, it didn't. The installer is built in a way that if there is any failure, it will roll back to the previous state. However, it may happen that this rollback is not happening correctly, and therefore this precaution of taking a backup becomes critical.
Yeah, I totally understand working in IT that everything starts with a backup. Okay, you had a second scenario you wanted to run us through. Tell us about that?
All right. So, we're gonna look at scenario where the upgrade is actually successful, but you are not able to start your platform. My services are started; however, when I try to launch the QMC, I’m just getting a 404 error. So, well, the first thing I want to do since I am on the Qlik Sense Server here; the first thing I want to test of course is if I’m facing any DNS or network issues, I’m going to try localhost. And I’m going to basically end up with the same problem, which means that my Qlik Sense platform is down here. The troubleshooting approach is a little bit different, right? Because we we're not going to look at the upgrade or installation logs, because those who are successful. What we want to look at is the Qlik Sense service logs.
Okay, taking a look at the services, would you help explain what all the services do exactly?
So, we've got the Qlik Sense Repository Database, which contains basically the entire configuration of your platform. So, this one is the first one that needs to function basically for the other services to pick up the configuration. The next ones are the Repository Service and Service Dispatcher. And then you've got the remaining ones which are the Engine, Printing, Proxy Scheduler that all depend on the Repository Service to be started to function properly. When you start the services here in Windows, there is an entire startup phase that's happening in the background we're going to look at through the logs.
So, since the Repository Database starts first, is that where you would begin looking in the logs?
Exactly. So, there is a much easier way for this one at least to to see if this one works fine than looking at logs.
We want to connect to the database, right? This is a Postgres database running in the background. One thing that we can do is to actually try to access the database. We usually do that with pgAdmin. There is an article actually explaining how to install it and configure it. It's called “Installing and Configuring pgAdmin4.” So, you can find that in our Community Knowledge Base.
And that is a free utility, right? That people can download?
Exactly. So, what we want to do is to basically try to connect to this Repository Database. So, I’ve already have a pre… connection that is already created, but the setup is very simple, right? So, the host is “localhost” because my service is installed on this machine. Got Postgres, and I will put my super user password during the setup. If we try to connect here, I’m gonna do “Connect Server,” it's gonna ask me for my super user password. And here, I can see that I can access my databases. I can open them, the connection here is working fine. So, I can assume that this service is actually functioning properly.
Since the database is running fine, what's the next step?
Will be to look at the Qlik Sense Repository Service, right? For that, we need to look at the Qlik Sense logs.
And where do we find those?
So, the Qlik Sense logs are - you can find them under C: > ProgramData > Qlik > Sense > Log. Here you've got all the logs for each services.
So, each service has its own log folder?
Exactly. Got the Repository folder, and we're going to look at Trace and System. So, if we look at this log, just scroll to the bottom here. We can see an error. .
Ah, right there.
What we are seeing here, if we look at the couple of lines above is: it's trying to connect to the QSR Database. It's waiting 300 second for a response, which is five minutes. So, we can assume that this service is not able to actually connect to the QSR Database; and that's most likely the root cause of my issue. So, once again, this is just a troubleshooting approach. You may have different error messages in there. The idea is just to take this error message, search for it in our Knowledge Base and find some potential root causes and solution.
In this particular case, so, I just modified manually the connection string the Qlik Sense Repository Service so that it will fail to connect to my database. I can quickly show that. So, if you go under C: > ProgramFiles > Qlik > Sense, you've got something called Repository > Util > QlikSenseUtil.
Okay, so what is this utility?
So, this utility it's basically providing several information: such as your connection string, your service cluster setup, and more. We usually use it for service clustering connection string editor. So, if you start to read your connection string. So, this is my QSR connection string here. What I’ve done is simply modify this port so that it won't be able to reach my QSR Database, because the default port is 4432 and not 5432. If I start to save this, and exit; I should now be able to actually start my services.
Now you're restarting because you actually changed that connection string, right?
Exactly. So, I need to re-trigger a connection attempt.
Okay, and once that's finished, you should be able to connect to the QMC?
Then yes, I should be able to access my QMC if I’m not facing any other error of course. There should be a line in this log that says “Startup Phase Completed.” So, you can just search for that, and there I can find it. So, that means my Repository started correctly.
If that started up fine, but we still weren't able to access the QMC or Hub, what would be next?
The next service that I would look at is my Proxy Service, because my Proxy Service is the one that delivers the web interface.
Okay, so, that actually hosts the QMC?
Exactly. I can take a Qlik look at my Proxy logs. So, same thing: Proxy > Trace, and then we've got System Proxy.
And I noticed before there are a couple warnings there.
Yes. So, that's because of the startup phase, right? it tries to connect to my Repository, but since my Repository was not up and running; so, it failed, but it's retrying, right, until it can. Then you can see the last line on this log is “Proxy is Running.” So, that means this service started up fine.
It's always important to look at the entire context, and not just stop at the first error or warning, because it might be actually a false alarm, right? So, if we now try to access our QMC, and our services are working, and I can connect to my environment.
Great. And would you say those are two of the most common issues that people face when they're upgrading: the upgrade fails or the upgrade succeeds, but the services don't work correctly?
Yes. Exactly. Now, the root cause can vary from case to case.
From version to version as well, because our R&D team is working very hard on trying to fix those potential use cases, but there might be new ones coming in other release, right?
Just as a reminder, we will be sharing the links to everything that's been discussed today along with the recording. Bastien, are there any other resources you'd like to highlight?
So, I just wanted to highlight a couple of things. Here are a few links around our System Requirements for Qlik Sense, how to back-up your Qlik Sense site, a link to the Release Notes and the upgrade process. And finally, I’m highlighting here a couple of articles that will help you troubleshooting your Qlik Sense installation or upgrade if any failure occurs. So, it's kind of a summary of what we just discussed today.
Okay, great! Thanks. Okay, now it's time for Q&A. Everyone, go ahead and enter your questions in the Q&A panel on the left side of your On24 console. Bastien, which question would you like to address first?
All right. So, the first question is: how to prevent Qlik Sense Enterprise on Windows from rewriting config files such as C: > ProgramFiles > QlikSenseproxyproxy.exe.config, which as for example: the keeper live timing parameter.
So, unfortunately this is not possible. So, when we do a major version upgrade, these files are being replaced or overwritten with the new ones corresponding to the updated version. My best advice here is to actually keep track of these changes you are making in these config files, to then update them post upgrade. Do not replace these configuration files from a previous version to a newer one, because they could be different, like new settings for example or deprecated settings. So, it's very important to keep the original file and just change the setting that you had configured before the upgrade.
That's good to know. Next question?
Do all nodes in a separated environment needs to be disconnected from the Repository before proceeding with the upgrade?
So, in a multi-environment, it's recommended to stop all nodes and start by upgrading the central node. So, if you are using a Qlik Sense failover feature, that's a Qlik tip here: make sure which node is having the role of central node before stopping the sources. It's very possible that actually a rim node became a central. And this one you will either need to move back the central node role to the original central. You will need to consider this stream. Know that the central node and basically be the first one to upgrade. Also, when you stop your nodes, always stop the rim node first, and then the central node at the end. When it comes to the rim node, there is not really any - it's not really important in which order you are going to stop your rims. But the central should be the last in any case.
That makes sense. Next question?
Please include any tips or suggestions you may have regarding certificate changes that occur during a Sense upgrade.
So, there are normally no changes to the certificate during an upgrade. They're not being touched. It has happened in the past for some specific releases that the certificate needed to be renewed as a post upgrade activity, but this is always documented in the Release Notes; so, with step-by-step instructions as well. So, finally the certificate is also part of the backup that you take before upgrading, so if sometimes it happens that something can go wrong; and in this case, you will have this backup to cover you. But in general, there is no change on the certificate that's happening in Qlik Sense.
Okay, next question?
So, we had a lot of problem in the past with broken certificate that then needed to be recreated reinstalled, so that's why we postpone upgrade unfortunately.
So, this issue - it's often environmental, and can be caused due to many different reasons such as: a security policy applied on the server, or third-party software such as antivirus blocking or blocking the files basically or corrupting them. So, as mentioned before, my best advice here is to make sure that you are backing up the certificate before upgrading. Which means that if they get corrupted post upgrade or at any time, you can safely restore them without any problem.
That's interesting that it's not, that happens people, but it's not actually Qlik Sense that's affecting it. It's just being affected by the environment.
Correct, and that's the thing with Qlik Sense. It runs on Windows and is very dependent on the overall environment that it's installed on.
Okay, next question.
Upgrading multi-node Qlik Sense from November 2019 to February 2020 was unsuccessful. Some users can log on, some cannot. How to check the root cause in the log file?
So, it's very difficult to say to say right now without any clear symptoms and clear message. So, what I will do is: based on your question, it sounds like the upgrade was successful from the installer (which was what we discussed previously in the demo), but now you are you are facing some post upgrading issue, right? In that case, I would just suggest to focus first on the symptoms that the users are experiencing. For instance: is there any specific error message or specific behavior that they experience when it faced with the problem and start from here. Alright. You can search this behavior or error messages in our Knowledge Base. That should give you a good point to start, here.
And if it has to do with logon issues, what log files or what service log file would you recommend looking at?
So, if that's pure logon, I will first check the Windows event log. So, that depends also which authentication method you are using, right? Let's say that you are using Windows authentication in this case. The first thing to know is that Qlik Sense does not take care of the authentication. So, if you use for instance Windows, you will need to check the Windows event log security at the time the user had the failure to see if there is an error messages there. If you are using single sign-on, you probably need to check your in the single second side as well.
Very good, thanks. Next question?
All right so please cover upgrades related to Postgres Repository sitting on node outside of Qlik.
So, the process is no different, right? Qlik Sense will automatically detect where the database is and will proceed with the upgrade. However, note that Qlik Sense will not upgrade your Postgres. You are responsible to upgrade it if you wish, making always sure that the version you choose is actually listed under the Qlik Sense system requirements for the version that you are running of course.
If moving from one version to another and you are currently several release out of date, do you need to install each releases in turn?
The answer is no. You do not need to upgrade to each release in between. However, if you want to install a patch, you need to first install the initial release associated to that patch. So, let's just take a quick example: you are running Qlik Sense February 2018 which is a very old version, and you want to upgrade to Qlik Sense May 2021. You can do that. You don't need to upgrade to each major version in between. However, if you want to upgrade to Qlik Sense May 2021 patch 4, you will first need to upgrade to Qlik Sense May 2021 initial release; and then apply the patch. And here the for the patch, same applies. You don't need to patch to apply the patch one by one. You can go directly to patch 4.
Thanks for clarifying that. So, next question.
Where can we find the latest patch?
So, this is something we quickly saw in the presentation before. You can find it in our download page which is accessible from Qlik.com in the Support area of that website, right?
Do I need to apply all patches or just the latest?
This is kind of related to the question we just answered before. So, no you don't need to apply all patches one by one. A patch is cumulative. So, which means that you can simply install the latest one and all fixes from the previous patch will also be included.
The next question is: we got error 1068 “The dependency service or group failed to start” with the Repository Service. What does that mean?
In my experience, so, this error message happens when you try to start one of the Qlik Sense Services in the service console, the Windows service console; and this basically it's very possible that the service account does not have the right privileges to start the services. I know that there are some articles that use about this error here. So, we'll just search directly for the error message in our Knowledge Base, and you should find information on what caused this and what is the potential solution.
What right does the QS service account require? Does it also need a license?
So in terms of Windows privileges, it's recommended that the Qlik Sense service account is part of the local administrator group on the Qlik Sense server. However, if for security reason this is not an option; so, it's very often that your IT department will not allow you that sort of privileges. There is a way documented on Help.Qlik.com to run the Qlik Sense services without admin privileges. When it comes to the license, the Qlik Sense service account does not need the license. The license is really about accessing the Hub, opening and creating application, so you don't need to have a license to access the QMC and do administrative tasks there.
That's good to know. Okay, we're getting near the end. Next question?
So, the next question is: usually I test future new releases of Qlik Sense server with the desktop version. Since there is no desktop version anymore, how do you guys test new versions before upgrading your whole cluster?
That's an interesting question. So, and here we are really talking about how deeply you should test your environment before upgrading. So, my recommendation here is to always have a Qlik Sense server test environment, because Qlik Sense desktop is very good to test your application and how they will behave with the updated version. However, something that you also need to consider testing is the server part of the software. Potentially the overall upgrade process as well, so that you are familiar with all the steps that needs to be taken when it's time to upgrade your production. So, testing application it's of course needed, right? But you also have the server side, the server parts of your upgrade that also need to be tested, in my opinion.
Is that like a license they could get from their account manager to have a test license?
Yeah, this is - you can reach out to your company to see what your options are, but yeah definitely having a test license is possible for a given period of time here.
Okay, we have time for one last question.
So, is the upgrade similar for QlikView?
Interesting question. So, while there are some similarities, especially when it comes to the steps prior to the upgrade such as the testing the system requirements, the backup and so on. The actual process is actually different. We are talking about two different products that work different way, right? So, there are there are definitely articles and help documentation covering this. So, you can search in Help.Qlik.com for the upgrade process, and you can also look at our Knowledge Base for upgrading and migrating best practices for QlikView.
I hope this was useful. Thank you all for attending and have fun with your upgrade!
Thank you everyone! We hope you enjoyed this session. And thank you to Bastien for presenting. We appreciate getting experts like Bastien to share with us. Here's our legal disclaimer. And thank you once again. Have a great rest of your day.
4. Q: How many user email addresses can be in one group?
A: You can add as many users as you want to a group, but the e-mail address is assigned only to the users, not to the group. Each user has a specific identifying e-mail address and up to 3 optional alternative addresses.
5. Q: Is it possible to drop the reports to a network folder?
A: We don’t have a feature to import users directly from Qlik entities. You can send them to excel and then import the users form there. You must be sure to respect the order of the fields in the standard Excel template to import users.
7. Q: Is there NPrinting API s available for dev ops? Where can we find more information about that?
8. Q: Which version of NPrinting used in demo? and whether previous versions and current versions of NPrinting are compatible with both QlikView and Sense?
A: The version of Qlik NPrinting used in the demo is the latest one: May 2021 release. Starting from Qlik NPrinting 17, you can connect to both QlikView and Qlik Sense. We recommend that the Qlik NPrinting version is the same as the Qlik Sense version.
9. Q: Any plans to fully integrate NPrinting into Qlik Sense?
A: There’s a plan to implement more Qlik NPrinting capabilities in SaaS editions of Qlik Sense, while the current version of Qlik NPrinting will be maintained to support Qlik Sense and QlikView on premise.
10. Q: Currently we have to maintain 2 set of users. One for QS and one for NPrinting. Is this the expectation?
A: This is correct, NPrinting users and Sense users are not necessarily the same.
11. Q: Any work being done to stabilize NPrinting? We have to restart NPrinting after every 1 week otherwise all tasks get stuck. We have got same feedback from other teams within our firm.
A: The latest version of NPrinting are stable, you should upgrade to the latest version. If you have any issue related to Qlik NPrinting performance, please open a ticket to Qlik Support that will be happy to help you investigating this issue.
12. Q: When the Qlik Sense server is down. Does the NPrinting still generate the reports or the Qlik Sense server should be online to do this?
A: NPrinting needs to connect to the Sense server that must be reachable and responding.
13. Q: Hello, I would like to know if there is a way to stop a task that is running (I have an issue and I have tasks that have been running for several days and I am not able to stop them to reload) Thanks!
A: Please, go to Admin>Task executions and click on the “Abort” button near the task name. If needed, you can also stop the NPrinting Engine service.
14. Q: How section Access works in NPrinting? If you can answer that would be helpful. Thanks
A:This article explains how to set up section access. Notice that when you use section access, the NPrinting engine user must be an ADMIN in the SA definition in the script. Moreover, for Sense connections NPrinting applies SA by re-opening the Qlik Sense application once for each user. This makes impossible to apply several optimizations in the task executions: so reports require more time to be created when Section Access is in use.
15. Q: Do users need to have a QlikSense license to receive emailed reports?
A: Generally, this is not required. NPrinting users are not necessarily the same as the Qlik Sense users. A token must be assigned only to the NPrinting Engine user and to the other users working with Section access.
16. Q: If you have created a condition” chart has value”. How about in a leveled report, on dimension has no values. At the moment, the level is created, by the chart is empty.
A: The condition is evaluated on the filters. Levels, Pages and Cycles are not considered, I.e. the condition is evaluated after that the filters are applied but before the generation of the reports and the application of levels. Consider that there are two kinds of Conditions: Reports and Task. For the seconds, even the report filters are not considered, just the task ones. You can find more information here.
17. Q: On the advanced search function for the filters, is this case sensitive? i.e. does it need to be Beverage or beverage.
A: Yes, it is case sensitive
18. Q: What's the best way to keep your import task up to date? For example, we may need to add new users or remove users that are terminated. Also, a user may change its group throughout the year.
A: You can do this by choosing the right option in the import task. You have options to automatically apply in NPrinting the changes you do in the Excel file, so that if a user is edited or removed in Excel, the same happens in NPrinting as well just by re-running the import task. Here is more information.
19. Q: When will NP allow us to duplicate publish tasks, filters, etc? A lot of time could be saved if this functionality was present instead of creating a brand new task or filter.
20. Q: If you have NP report run but it contains no Data, will NP print the report or will NP generate an error?
A: Empty data is not considered an error, so the report is supposed to be generated. You can use a condition to block the generation of the report if a chart is empty.
21. Q: How is the performance in QlikSense server side, if we have many reports NPrinting? Impact in RAM or CPU?
A: This depends on the complexity of the reports and of the Qlik Sense applications. NPrinting needs to perform a selection in the Sense app for each filter and cycle and then it has to export charts and images. The formulas in the app charts must be re-calculated at each selection and this can require much CPU if they are complex.
22. Q: In complement to my question, I tried to Abort it but I get an error message : Something went wrong while aborting execution of the task 'AP042_R2_MPS_A03'.
A: It is possible that an internal process could not be stopped correctly. In this case, I would stop the NPrinting services after aborting the task. If the problem occurs frequently, please, contact Qlik Support.
23. Q: What are potential problems when setting up an import task? My Nprinting, for example, seems not to be able to access my C:\ drive.
A: I think this is the only potential risk. Be sure that the NPrinting service user has access to the folder where you save the Excel file.
24. Q: How to have object ID from Sense when you don’t have Title explicit label?
A: Open the sense app and add /options/developer at the end of the URL and press enter. Right click on the object you want to import in NPrinting and choose “Debug.” You can use the Object ID in NPrinting.
Hello everyone, and welcome to the July edition of Support Techspert Thursdays. I’m Troy Raney and I’ll be your host for today's session. Today's presentation is Unlocking the Power of Qlik NPrinting with both Andrea Bertazzo and Giuseppe Panella. Andrea why don't you tell us a little bit about yourself? Hi, I’m Andrea. I’m working in Qlik Support mainly on the design team and I’m focusing on NPrinting since I joined Qlik; but I was working NPrinting also before in the former Vizubi team, the team that created NPrinting. So, yeah this product is part of my working story, and I would say a nice part of it. Giuseppe? Yeah. I’m Giuseppe Panella and I work in product content and media team; producing videos for the Qlik Help site and Qlik Help YouTube channel. And I joined with Andrea with Vizubi team seven years ago. Great, and kind of high level; what are we going to be talking about today? Yeah. Today we are going to introduce you a bit to Qlik NPrinting, that is the reporting and distribution solution for Qlik Sense and QlikView. And it enables you to create, manage and distribute reporting projects. Qlik NPrinting enables organization to easily create reports using data and analytics from Qlik Sense and QlikView in formats such as PowerPoint, Excel and Word. Okay. Or using the built-in the editors like HTML or pixel perfect. So, from a high-level, how does Qlik NPrinting work? Yeah. Qlik NPrinting helps users to schedule and generate reports. Can send them to a variety of recipients with filtered information, so every user can have customized information. And you can also deliver reports using email or distribute to corporate directories; or you can publish in NewsStand web portal. That is a portal where you can download subscribe and view the available reports for user. All right. I mean, I know it's a wonderful product for people who are really looking for some more static reporting and a very pixel perfect, high quality way. Could you set-up for me a scenario on how users can take advantage of NPrinting? Yes, actually. What we want to show today is a story of an enterprise. So, the Acme Corporation one has two different sales teams they want to send different information. One for the Beverage sector; and one for Food sector. Okay. Each team will receive different reports depending on the team they belong to. All right. So, we start with creating an app in NPrinting; and then we create a connection to Qlik Sense. And once we have created this connection, we are going to start populating NPrinting with the users. So, in this case, we are going to create two different groups for two different sales teams. One for Beverage and one for Food; and then we show you how to create the user manually; and how to add role to this user. And then, the best way to do this in NPrinting so importing from Excel and using an import task for importing user in NPrinting. Yeah, we create a filters, so people will receive filtered reports; and then we are going to create the report that is an Excel report, that we are going to distribute through email and to the NewsStand. Okay, great. Before you show us how all that is done, what version are you going to be demoing? Currently working with the latest version of NPrinting. That is the May release. Okay, are NPrinting and Qlik Sense installed on separate machines? Generally, yes. I’m NPrinting a Qlik Sense should be installed on different machines, because NPrinting and Qlik Sense essentially need to use as much as resources are available on the machine. Okay. We asked to install them separately. Okay, and Giuseppe was mentioning we needed to start with an NPrinting app? Yeah. What is an NPrinting app, and how is that different from a Qlik Sense app? The NPrinting app is essentially a container. We are going to create this NPrinting app for our Acme project. So, an NPrinting app is what contains everything related to this project. We will see that we will add our report, our users, our filters. Okay. So, how do we get started? Just creating the app. So, we click here on apps. And click on the app button. Write a name. You can add a description, but it is optional; and click on Create. The app is ready. You can see it there. Okay. So, what information do we want to send? What we want to do is to go to this Qlik Sense application; and to send a report containing this chart to our users. So, we have to create something connecting NPrinting and Qlik Sense. So, you're setting it up to send that chart to a specific set of users? Yeah, exactly. Okay. Yeah. We had the app and to create the connection click on Create Connection. As before, just give a name. Is there a best practice for the naming conventions to use? Well, you can add many connections to the same app. Okay. It's better that you add a name that reminds you to the content. That makes sense. Description is always optional. Connecting to Qlik Sense; so, we choose this. We have to add a proxy address and Sense IP address. Just go back. The proxy address is just the first part. Remove this. Okay. Then the Sense app ID. Keep it and remove the rest. Okay. And, we have our connection ready! Of course, we have to add the connection to our app; and then specify a domain, and a username. This is the user that is connecting to Qlik Sense to get the data. Should that be an admin account like the user running the NPrinting Engine Service? Yes. Exactly. It must be an administrator, so that it has full access to Qlik Sense. Okay. And click on Create. When you do this, NPrinting starts connecting to Qlik Sense and import the data. It is running now. All right. If you are connecting for the first time to Qlik Sense; remember that you have to import the Qlik Sense certificates. So, that's an important step on importing the certificates from Qlik Sense, but you've already done that? Yeah. So, I have done this some time ago. In general, you have to do this every time you install NPrinting or you upgrade to another version. Okay. So, we've created the connection; what's next? Yeah. Now we are going to create the group to organize the user in two different groups. Right. To create the group, we are going to Admin, and select Groups. In this case, we are going to add the Beverage group. Okay. Click on Create. Okay. And then we're going to create the second group; that is the Food group. After this, we are going to create the first user. So, we're going to show how to do it manually; but best practice is to create an Excel file to import thousands of users. We're going to insert the email that is required field, and the name of the user. Fill in the password. Since you are going to use the NPrinting authentication, we must insert the password. Choose the time zone. Deutsch. When done. we click on Create. So, now we add Role - are used in NPrinting for security. So, in this case, we're going to add the User Role. Okay. So, we've gotten our user Ann with an E, and we've given her a User Role. Is she already in a Group? We are going to assign this User to the Group. In this case, in the Beverage group. Okay. And when done, we are going to create a Filter. Create Filter. So, we add a name to the filter. In this case, is Beverage Germany. Then we have to select the app; and select Create. And then we are going to add Fields. We select the connection to Qlik Sense. So, I see it's picking up all the different fields that are available in that app. Yeah, exactly. These are all the fields that are available in Qlik Sense. We are going to select, in this case, the Region first. Okay. A specific value. In this case, Germany. And then we are going to add the second field; in this case, the connection is the same; but we are going to select the Product Group. If we write Beverage, you can see that you have two different fields. And so, we are going to use Advanced Search, in this case. All the fields that contain the ‘Beverage’ word will be included. Okay. So, the filter will include both ‘Beverage’ and’ Alcoholic Beverage’ for Germany? Yeah, exactly. Creating a user manually is a very useful exercise; but if you have hundreds of users in your company this is not feasible, because it takes too long. Exactly. The most powerful way to create your user is to import them from Excel. Okay. You can download this template from our tutorials. You can see you have three different sheets. The second one is Users. Here you can just add the email address of each user, and the user name. Remember those are mandatory fields. And here you have all the other options that are available when you create the users manually. And you can just add the values. Okay. And do these field names match exactly what NPrinting uses? Yes. It is important to keep exactly the same name in the Title. Okay. And exactly the same order. That's why we recommend to download the template directly from our Help page. Yeah, that makes everything easier. Yeah, it makes everything easier and you're sure that it works. And at the end, you can see that we have a column for the Roles, a column for the Group; so, the user is automatically added to the group that we created. And we have a column for Filters. But those filters are new, right? And that's maybe the most powerful part of this process: Importing users from Excel; because you can define your Filters, and they are imported and created automatically. Here you see we have report in the same name that we have here in Users. And here on values, we specified region as before. This field, we can specify the value that we want to select with the filter. Okay. So, each user has its specific region. Moreover, we can also specify the Advanced Search. So again, we specify the name of the field that we want to filter; and then we use this syntax; for example, to select Beverage for the Food group, I added this ‘Excluded’ tag. This means that we keep all the other values. Great! So, how do we import all of this? Go to Task and create an import task to do this. Okay. Give a name; add a source. So, here we have to enter the path. You have different options. I recommend to select ALL the options. And when everything is done, you just click on Run Now. Generally pretty fast. And when it is done, you can see that on the Users menu, we have a lot of new users automatically imported by NPrinting. All right. That's great! On Filters tab, we have all our new filters. If you open any one of them, in the fields you see that we have our Region selected; but also our advanced search with the selected ‘Excluded’ tag flagged. That is pretty great to have all that created with one simple task. And if you need to update or change anything, you would just make the changes in the Excel file and then re-run the task? Exactly. You make the change there and you re-run the task. So, now we have the connections, our users, and the filters. What's next? It's time now to create the Report. The report is the file that our users will receive. To do this, let's go on Reports, and click on Create Report. Okay. As before, just add a title. Between the different formats, we choose Excel. We want to create an Excel report. And we specify the app; as always, everything belongs to the app we created. Just click on Create. So, the report is saved. We can click on Edit. To create our file, NPrinting will ask you to choose the application. Application is NPrinting Designer. This is a free software. Okay. And you can download it from our download page and install on the local machine. So, just click on Choose Application; and open the link to Qlik NPrinting Designer. All right. So, it's firing up. You can see that NPrinting Designer is embedding; Excel in our case; but it's also showing us a lot of NPrinting options. Would you also use NPrinting Designer if you had chosen one of the other options like HTML or PowerPoint? Essentially all reports are working more or less at the same way. We want to send our users this chart. So, to show the image, just click on images. And click on… So, these are all the objects that exist in the Qlik Sense app? Yes. Those have been imported when we created the connection. Okay, and I remember that was called Total Sales. Exactly. Just write Total Sales. That was the one that we needed. All we have to do is to drag it into the template. This tag will be replaced with the image. Okay. We can also do the same for the table. So, we can import the same in this case as a straight table. It gives us the possibility to drag each single column one by one, but also all together. And this option is quite nice; because it allows us to use Excel formulas. For example, to calculate the total automatically during the generation of the report. That's really cool. Now the rest of the z is zero, but it will be populated later. I want to add something more. I want that each user can see the sales emerging for each city in this region. To do this, I create a Level. I click on Add Levels. Levels are cycling on all the values of a specific field. Okay. So, it's like a cyclic group. Exactly. So, if I select City for example; and I put the level here, everything between those two tags will be cycled by city, and I will see my image and my table with the value of each city. Okay. And saving is always good. Do we need to actually run the task that generates the report? Exactly. So, to create a task, we just go to Task; click on Create a Task. We always have to put the name; a description is optional; and as always, a task belongs to specific app. And click on create. The task is there, but we have to populate it; first with the report. You can add as many reports as you want to the task, each belonging to the same application; we have only one. As output format, we choose Excel. What were the quality options there? The quality option is for reports in PDF format.
Okay, and I guess higher quality means larger PDF file size? Yeah, exactly. Okay. We have to add our users. So, go there click Add Users or Groups. You see we have a lot of users. Fortunately, we created the groups. So, instead of adding each user; we can just add the group. We have one for Beverage and one for Food. Okay. So, this task will distribute that report that you made to the users in those two groups via the NewsStand? Exactly. And what is the NewsStand? NewsStand is a web portal. Each user can connect to the NewsStand and see the report directly there on the browser. Can we also send the report via email? Yes. In Destination, we can choose NewsStand by default selected, but we can select also email. Okay. If we do this, users will receive an email message. Of course, the email message can be customized. You always have to add the subject. Okay. You can also customize it with username, report name or variables from Qlik Sense. And the reports will come as an attachment to the email, I see. Yes, the report is an attachment to the email message. Is it possible to actually embed the report inside the email? Yes, it is; especially if you create HTML reports you can embed them. Everything is explained in our tutorial. I guess the next step would be to trigger this task? You just go to Trigger; give a name; and here you schedule to run it, for example every week, maybe on Monday. Okay? Okay. You can also specify the time. Maybe I want to run it at 8, and just click on Create. NPrinting will automatically start the task execution every Monday at 8. And the next option there, what are Conditions? With Condition, you can you can be sure that the report is run only if certain conditions are met. Okay. For example, if a certain chart has no value then it's better not to start the task at all. So, for those kind of needs, you can create your condition. Everything is explained in our tutorials. So, there you can see how to create and apply conditions in NPrinting. Great! Has the task been triggered yet or can we trigger that manually? Yes. To trigger the task manually just go to Overview and click on Run Now. Okay. And task will start. You see that it's running here. We have some warnings, so something is not as we expect. Andrea, if people encounter any errors along this process, how can they find out more information about what might be going wrong? So, the first thing is to check tasks that are not completed, that have errors or warnings. Just click on the task. Okay. And I can see some warning and error messages. Okay, so this is pulling information directly from the log files? Yeah, this is taking information from the log file. Okay. Even if the log file is always more accurate for a more deep analysis. It is better to have a look at the log. Sure, but this is nice to get kind of a quick preview right in the console. So, it looks like our email server is not configured correctly. Exactly. I have to reconfigure my SMTP server, because otherwise I cannot send email messages. Okay, so where do we find those service settings? In Admin Settings, I have a lot of options – Email. Yeah. I see it just hasn't been set up yet. It has not been set up. The address and the port essentially. There are some other warnings. To investigate those further, where can you find the log files? The NPrinting logs are saved in C: > ProgramData > NPrinting > Logs. What I really need to troubleshoot problems with the execution of reports is the Engine Log. Okay. And I go to the end to see what is the error message. The NPrinting Engine, that's the service that's actually running the task? Exactly. The NPrinting engine is in charge of generating the reports. It creates the file. And when you look at this, what are you looking for? I look for errors. For example, in this case, I have a CEF rendering exception. Okay. This means that there is a problem with the image. If you don't know what that means, what would you do next? I can copy this. I can just Google for this error message; and you see that there are a lot of articles that can help me. For example, this article shows me how to troubleshoot the problem. You see it is reporting the keywords that I see in the other message: GRPC and CEF. And here, I can see all the steps that I can perform to fix the problem. It's quite long article. Following this article, you are pretty sure that you fix it. That's great! Can we open up NewsStand to see what the finished report looks like? Let's connect the NewsStand. And in my case, I can use ‘localhost’ because I am on the local machine; just need to change the number of the port. Question: is somebody wasn't on the machine where NPrinting was installed, what would the URL be? You have to specify server DNS and name. Essentially, you use the same address that you use to connect to NPrinting web tool, yes. And just a different port: 4994 instead of 4993? Exactly. Just add 1 to the port. I had this user, for example; so, I can use this email; and you can add this password. So, that each user can see only his specific records. And here is the report that NPrinting created. I have a preview where I can quickly check if everything is fine, but if I want to see the entire report, I click on Download. The report is downloaded. Here, I can see I have the first chart with the data for the entire region; the table with the total calculated by Excel; and then data for each city in Spain, in this case. With different totals of course, because those are filtered city by city. Ah, right, because you had it cycle through every city in the city field. Yeah, exactly. That's great! Before you mentioned a tutorial. Can we take a look at that? You can find all the information about the things that we have talked about today in the tutorial attached to this presentation. Okay. So, we have collected the Qlik Help articles and videos, so you can go and look in details what we have talked about. And we have more information about how to create reports; and different reports; filters; and so on. Well, this is really great. So, you've kind of collected all the critical information we've talked about, where people can find more documentation and additional instructional videos. This is really great. And everybody, this is going to be sent to you as an attachment to this video on Qlik Community. So, please take a look at the attachments. Okay. well now it's time for Q&A. Please submit your questions to the Q&A panel on the left side of your On24 console. Andrea and Giuseppe - which question would you like to address first? Okay. The first question is: when will NPrinting be available for Qlik Sense SaaS? Okay, and well at the moment, well, there is no plan for having NPrinting connected to Qlik Sense SaaS, but some of the features are already been implemented in the Qlik Sense SaaS. Last month has been released Subscription feature in SaaS edition of Qlik Sense. More feature will be available in the next months; and Qlik NPrinting will be available as a standalone product for Qlik Sense and QlikView on-premise, while in SaaS will be developed a new feature to cover the reporting features of SaaS. Okay. Second question is: doesn't NPrinting consider Section Access when sending records? Of course, if you have Section Access, you have to specify it in the Connection. You have the option to click for Section Access here > applies user Section Access for reports. When you do this, be sure that the NPrinting user you are connecting with, I mean this user is Admin in the Section Access definition. So, each report can be opened and then the selection of Section Access will be applied to each user that you have defined in the User section; so, here, right? That makes sense, yeah. Consider that Section Access may increase the time necessary to create your reports, because you have to open the Qlik Sense application or the QlikView application every time for each user; so not only once for all the users, but once for each user. So, it takes a bit more. Okay. Next question is: does NPrinting need to be installed in on a separate machine? Well, even if there are no conflicts between NPrinting and Qlik Sense / QlikView services; NPrinting must be installed on a separate machine in the same domain as Qlik Sense / QlikView. Having the two software on the same machine can cause resource problems independently of the amount of the RAM or CPU that are in the machine. Thanks for that. Next question. Does NPrinting work the same with QlikView? NPrinting works with QlikView and Qlik Sense at the same way. There are just little differences and the procedure is exactly the same. You will not notice any difference working with QlikView or Qlik Sense connections. Great. Okay. When will be able to auto-trigger NPrinting connection from QlikView QMC? Actually, this is not in the roadmap; but you can add this idea in the Ideation portal of community. Okay, another question is: is it possible to schedule tasks based on QlikView reloads? It is possible; but it is not an internal feature in NPrinting. So, you have to do this on your own by using NPrinting APIs. We have several posts in the community that you can follow. Essentially you can ask NPrinting through APIs to start reloading or executing a task just after a successful QlikView or Qlik Sense reload; but this is something that you have to do on your own. We have the documentation in any case in the community. Okay, so it's not ready to do that out-of-the-box, but it is possible to set it up using APIs? Yes, exactly. All right. Next question is: when will be able to trigger publish task when connection is done. Actually this is more or less the same. You have not a built-in solution in Printing, but you can use APIs to trigger connection after the connection is done with Qlik Sense; so, quickly. Okay. NPrinting engine is offline even when all the tasks in knowledge base are followed. This is a question. Maybe more a problem reporting by a customer. So, NPrinting ‘engine offline’ means that something probably has not been done correctly during the installation. The first thing I would suggest to do is: to check the NPrinting engine logs to see what is the error message. Be sure also that you have correctly installed the NPrinting certificates; and that you didn't forget the password. If this doesn't work, you can contact Qlik Support to ask for help. We will be more than happy to help him. Great question: how can I create a reporting in NPrinting using multiple Qlik Sense apps? Well actually, you can create an app in NPrinting, and you can add several connections to different Qlik Sense apps. And then once you have connected to different Qlik Sense apps, in the in the Designer, you can create the Template using different sources. So, you can select the Connection in the Designer and pick-up the object from Qlik Sense that you need. Okay. So, in the NPrinting app, you can create multiple connections to different Qlik Sense apps and then in the Designer, just pull in the objects or charts that you want? Yes, exactly. You can even connect to a QlikView document and a Qlik Sense app in the same NPrinting app. So, you can build a report using two different sources. That's really cool. Thanks. Another question: I have a Qlik NPrinting Excel report. I am able to generate a task and email is delivered with attachment or not. How to identify if the email was sent with attachment? So, in this case, NPrinting automatically stop the generation of reports when the reports is empty. You can monitor this using a Condition. So, you can create a Condition on specific charts; so, that if the chart is empty, if it has no data, the report is not generated at all. So, no email message is sent if you don't use the condition. You send an email message without the report, but maybe it's better to stop the report generation. And they may send it by using a Condition in this case. That's a really great tip. Next question is: where I can find information about configuring certificates? Basically, you can find all the information related to these topics on Qlik Help site, yeah. And you can, here, you have all the information related to the connection; the requirements, and how to install certificates. It's even got a video, that's great. Okay, we have time for one last question. Yeah, last question is: how can I install NPrinting to not use HTTPS, but just use HTTP? As before, you have documentation about it here. You can find it on Qlik Help site, and you see you have also a video related to this in the Qlik Help YouTube channel. In the Enabling HTTP, you see that there are step by step guide that shows you how to do it. Simple. Great! Okay, thank you guys. Thank you all. It was a really nice experience to share with Giuseppe. We went both through NPrinting from the beginning to the end; and we have learned how our documentation can help us create our reports and tasks quite easily in NPrinting. Thank you all. Hope you enjoyed the presentation, and the content. Thank you everyone! We hope you enjoyed this session. And thank you to Andrea and Giuseppe for presenting. We appreciate getting experts like these to share with us. Here is our legal disclaime; and thank you once again. Have a great rest of your day.
Q: Can we make App specific Data Job alerts (Success/Failure)?
A: Yes, if you would like to get an alert when a specific app has been reloaded successfully or otherwise has failed, then you can use “System alerts”
Q: From whichQliksenseversion onwards alerts are available?
A:According to the help page:Minimum recommended version of Qlik Sense Enterprise on Windows is February 2019.
Q: How can I get an alert if a QlikView job fails to load, or if an NPrinting job fails to execute?
A:Not possible, for now Qlik Alerting is only available to set it up with Qlik Sense, starting from QS February 2019 .
Q: Hello, I have a questions regarding the conditions. They seemed to only use operators, but can we apply filters or selections to the alert?
A:Yes, you can apply filters andconditions.
Q:In DATA alert, can you edit an ADDED MEASURE?
A:If you mean if you can edit a measure once it has been created, yes you can do itafter from the QA webconsole.
Q: Is it mandatory to have different machines for Qlik Sense and Qlik Alerting?
A:Not mandatory but best practice is to have them in separate machines.
Q:Can you loop through Selections to test condition on each?
A:Yes, using filters and more advanced conditions, you can find some examples in our Help Page.
Q:Since it's included in SaaS, does that mean there's no added cost?
A: Yes, exactly, for Qlik Sense Enterprise SaaS there is no extra cost, but it is only available for SaaS Enterprise, not in Qlik Sense Business.
Q: Can Custom notifications be used to send alerts in another language?
A:Yes, sure, it can be sent in any language as they are custom notification so you only need to put whatever text you like.
Hello everyone and welcome to the June edition of Support Techspert Thursdays. I’m Troy Raney and I’ll be your host for today's session. Today's presentation is “Getting the Most from Qlik Alerting” with Patricia Silva. Patricia why don't you tell us a little bit about yourself? Yeah, sure. So, hello. My name is Patricia. I’m a Technical Support Engineer based in the Qlik Barcelona office, and I have been working for Qlik since 2019, and my main focus are Qlik Sense, NPrinting and Qlik Alerting. Okay, and what will we be covering today? We will go through what Qlik Alerting can do; difference between SaaS and Windows; demo with SaaS; a demo with Qlik Alerting on Windows (on-premise); and troubleshooting tips. Great! So, for people who are kind of new to the product, can you explain what is Qlik Alerting? Yeah. So, Qlik Alerting is a separate product that can provide enterprise alerting for your Qlik Sense deployment. Okay. It offers capabilities for users to create their own Alerts on the data they have access to in Qlik Sense; so that way, they can easily spot outliers within their data by configuring Alerts. And then when another fire, it can be directly sent to user phones or email; whatever you have configured. So, how does it work from a high level? I know we're going to get into the demo, but can you show us the workflow? Oh great. Yeah, sure. So, in terms of workflow, on the left-hand side, we have the enterprise data sources flowing to Qlik Sense application. And in the middle, we have Qlik Alerting. So, it will look into those Qlik applications, master items and measures that are present in in Qlik Sense applications; and it will allow administrators, users from Qlik Alerting to directly create Alerts using the web application. From Qlik Alerting, you can apply conditions and when these conditions are met, then it will directly send to email or to your mobile phone. Like text messages to mobiles? Yeah, exactly. That's really cool. Now you mentioned that it's an additional product. Is it free or is it a feature or an add-on? How does that work? Yeah. It is a different product. So, it is not free of charge, but you only need to make sure that when you buy this attribute; you need to check that it's indeed added to your Qlik Sense license. So, that way Alerting can check it. But yeah, it is a separate product from Qlik Sense. So, once you've added the Qlik Alerting tag to the Qlik Sense Enterprise on Windows license, what type of Alerting capabilities are available to each access type? So, from Qlik Sense side, we have Professional, Analyzer, Capacity and no license access. For example, if you have a user that has no license, so here we can see what you were able to do on Qlik Alerting side; so for a professional license, you have all capabilities. This is just at a high level. You can find more information in the help page. Okay. So, yeah here's a brief summary. Okay. I appreciate that we can get more details from the documentation, but that's nice to have that overview. What types of Alerts can Qlik Alerting generate? You can generate Data Alerts. So, that means it will look into Qlik Sense data application, then filter by dimensions. Okay. You can also do System Alerts; which means every time an app reloads, you can choose between: success, failed, abort, system abort or in progress. So, you will receive an email about the status of the task. Okay. So, like you could have an admin, be aware of when a data update fails? Yeah, exactly. You can also configure Broadcast Notifications. For example, when your system is going to have maintenance; it's just a reminder of anything that you would like the users to know. So, with Data Alerts; that first one you mentioned there. I imagine when you're looking at the Sense app and all the data that's available; it could be a little complex. How do those work? For Data Alerts, we have two ways of distributing. One is Broadcast Alert. So, this alert, what it will do is go to the Qlik Sense application and it will be this user who is the creator will check this value. And if it meets the condition, it will be sent directly to User 1, User 2, User 3, User 4. And on the other hand, we have Managed Share Alerts. Is that the one that can validate the users through a Section Access table first? Yeah, exactly. So, it will take into account section access. Patricia, what's the difference between Qlik Alerting on Windows and Qlik Alerting with Qlik Sense SaaS? So, yeah. In Qlik Sense SaaS, it is already included. You only need to enable them from here. You have to go to Qlik Sense Management Console (QMC), and you just need to make sure in Settings that you have enabled Data Alerts, here. Ah, right there. Yeah, and if you want to distribute those Alerts, not only to this space but also to an email, you need to be sure that you have Email Server configured. Okay, can you go ahead and show us how to generate Alerts in an app here? Yes, so once we have enabled this setting, we can just go ahead and right-click on any visualization. So, we can go here: Alerts > create new Alert. So, let's give a name. Okay. Yeah, and we just need to select a Measure which is going to be: Sales Total. This is a Master Measure that I’ve just created. You can see here: the Total Value for these measures. Go ahead and click Next. We select a measure which is: Sales Total; and we are going to select Greater Than 25 million. And does it automatically do that preview? Yeah. So, we go ahead and click next. And we are going to say when data is refreshed. You can do it daily, weekly or monthly. Okay. And that's it. Alert created. That's nice. So, we can also go here; and this has been created. You can disable here, edit, … Great. So, I will go to Data Load editor to reload again. So, you're doing a manual reload to kind of trigger that Alert? Yeah. So, we can see that we have received an info alert here. Okay. So, pops up there notifications. Did it also send an email? Yeah. We received here the Alert which has been triggered, because it's above 25 million; and we can click here and… It takes you right back to the app. Yeah, exactly. That's great! You've shown us how Qlik Alerting works on SaaS. It's very user friendly. I understand with Qlik Sense on Windows, there's a button that needs to be installed. Can you show us how that setup looks? To do that, we need to go to the QMC, because we have an extension. When you set up for the first time; Qlik Alerting, it comes with the installation. So, we just need to go to C-drive, Program Files, Qlik Alerting, and the extension is just here. So, that's after you download it. Yeah. So, we just go ahead and go to Extensions… Import. So, we just go ahead and Import. We can see it has successfully been imported. So, now we refresh just in case. We go ahead and click Edit Sheet. We go to Custom Object, and it will appear here as Qlik Alerting. There it is. So, we just drag and drop. We need to give here the name of our Qlik Alerting Server which is this one. So, we don't need to copy HTTPS because it's already checked here. Okay. So, that's the separate machine where you have Qlik Alerting installed? Yeah. Exactly. So, as we can see this is failing. And this is normally because we don't have right certificate installed. We will now install it. Okay. So, first thing you need to do is: to go to the QMC. Go to certificates; give a name. Let's say “qlik.serverr.domain.local.” Include separate key; and you export them in PEM format. So, once they are exported in your Qlik Sense machine; you import it to Qlik Alerting machine, which I have already done here. Okay. So, when we have that, we need to go to the Certificate Manager. And we're on the Qlik Alerting machine here, right? Yeah. Truster Root Certificate Authorities. We go here, and import. So, to be able to see it, we click ‘all files,’ and we select Root PEM. Okay. So, we click Yes. Import was successful. So, now on the Qlik Alerting machine, you've imported the certificate from Qlik Sense. So, they should work together now? Yeah. Now I will refresh. So, there we go. Now we can see the button to Create Alerts. Wow! Great. Shall we make an alert like we did in SaaS? I will just do it simple. Okay. We won’t filter by object. We will just select this Measure, which is a master measure. Okay. Select ‘Geater Than,’ Manual Value. You can do Manual Value, Measure, Previous Scan or Set. Previous Scans: it compares if the data has changed from the last scan value. To select Manual Value, which will be 25 million. All those zeros. Yeah, yep. I saw there you could apply a second condition as well. That's interesting. Yeah. So, for now we will select Every Time the application reloads. Create Alert. So,once the Alert has been created, you can go to the Detail View. So, this logs you into the Qlik Alerting console? Yeah. So, now we can see here the Alert that we have just created, and we can also Edit. So, you can add more conditions if needed in the future. Okay, and if you made changes here, it would still apply to what you created from the app? Yes, exactly. So, we can see here that we have selected On Reload; and the distribution, if we select Advanced by default, it's just Broadcast Alert. But if we select Advanced, we can change to Managed or Broadcast, and also select recipients. And Managed, that was the one that could use Section Access? Yeah, exactly. That it will take into account Section Access in term of distributing that data. Okay. You can also Customize Notification, which we will do next demo. Oh, nice. Save. So, you're gonna go into the QMC and trigger a reload? Yeah. I see the reload was successful. And because you've already configured your email server, that triggered this email Alert? Yeah, exactly. This is the default email but you can receive custom notifications. Right! Can you show us how to set up Customized Notifications with HTML code? Yeah. So, to configure Custom Notifications, we have to go to the other we have created before. Click Edit, and we go to Notification; and we will give this a name. And in my case, I have created a simple HTML code which (if you can see it) has kind of variables here. Can you show us where on Help.Qlik.com we can read more about those types of variables? So, we have a list here: Dynamic Text Variables. So, I’ve used not all of them, but some of them. So, you can see I have selected Alert Name. We will retrieve here the first name of the user who is going to be distributed; and the Conditions we have used to trigger this alert. Okay. I will just copy and paste it here. Okay. And I will trigger this time the alert from Qlik Alerting web console. And there it is. I see it picked up all those variables. Yeah. And looks very nice. Okay. Now, I have a question about email and the users that Alerting is accessing. Now, does it get all the users from Qlik Sense, or how does it pull its list of users that it could send notifications to? Yes. So, Qlik Alerting has this functionality; so you can sync your user who is your Qlik Sense site. Okay. And here we can see that this user has Analyzer, Analyzer and Analyzer. Okay. And we can also give an email, if it has not been defined in Qlik Sense Side; you can then just go ahead and edit and just change the email. It's cool you can edit the contact details from the Qlik Alerting side if it's not what you're expecting. Yeah. So, I noticed in the URL at the top that Alerting is using port 4552. Is it possible to change the default ports? If you wish to change these default ports, you have to make these changes in two places. You might wish to use port 80 for HTTP and 443 for HTTPS; which can make it easier for a user. Okay, and would you say that's kind of best practice to change it to those? Yeah, exactly. Can you show us where those locations are that need to be changed? Yeah. So, we need to change it first in the UI. So, we go to Admin, and Config. All right. Then, we change it here so 80 for HTTP and 443 for HTTPS. We click Save. And then we open the Services. And we will stop Qlik Alerting Gateway. Okay. Once it is stopped, go to C-drive, Program Files, Qlik Alerting, Config. We open the Default.JSON. And just in a notepad editor? Okay. Yeah. And as we can see here in the 5 line, change the HTTP port just as we did in the UI. So, 80 and 443 for HTTPS. We click Save, and then we go back to the Services. And just start Gateway again. So, now if we go back to Qlik Alerting; as you can see, there's no need to add the port every time. So, if we go back to this application, this is going to fail. Because now we have changed the port. Is it having trouble communicating with Qlik Sense right now? Yeah. The thing is that now; as we have changed the port, Qlik Alerting is not in that port anymore. So, we can do it directly here. So, we can just say 443; which will fix it. Yeah, how can we make that a more permanent change, so it applies to all new Alerting buttons as well? Yeah. So, we can do it from the Dev Hub. Yeah, what is the Dev Hub? It's a tool that comes with Qlik where you can edit extensions; also create extension; and create mashups. So, it's basically for developers. Okay. And in our case, we need to change an extension. So, we will go here and open a project, because it's an existing project which is called Qlik Alerting. This is the extension that we imported before; and we need to go to the third tab: JS/Definition. And here where it says ‘localhost,’ we will just copy the address of Qlik Alerting with no HTTPS. Okay. And Qlik Alerting Port, we will select 443. So, we click ‘Save.’ Go back to the application. All right. So, now if you were to add it back; it should work, fingers crossed. So, if we now add this; it will appear, because now we have defined the DNS Name and Default Port. That's fantastic. So, you've explained that pretty clearly how it works. What if we've gone through all these steps, and we're still seeing an error message? How can people troubleshoot this? They can check that they are running really in 443; and also check that they have copied this with no HTTPS, because if I go like this. So, we only need to give that name and the port. Okay, but if people aren't really sure where the problem lies, how can they start troubleshooting? Yeah. For example, with HTTPS you can also use the developer tools, which are located here on these three dots. Then we go to More Tools, and Developer Tools. You can also press F12. So, in that case, it failed to post because we are doing a Post Request here. So, you can see ERROR NAME NOT RESOLVED. This is because you are using HTTPS, which you already have defined here. If you remove that, this is just adding the same here. Right. That's a great tip for learning more. So, for example, the error message that we had initially when adding the Alert button. How can we search for more information about that error? Yeah, exactly. So, then we can go ahead to Google and just say ‘Qlik Alert and could not retrieve,’ which was the error message we were getting. We can open this article; and here we can find more information, detailed information about this error, and what you would see on the console tab. Yeah. I see that's an article from the Support Knowledge Base that shows why those messages are shown, and how to resolve them. That's really great. Patricia, one of the Alert types that you discussed before in the presentation was System Alerts, like an alert to admins if a reload fails. Could you show us how those are set up? We can create system Alerts. We go to Alerts here. We have Data. Wow we are going through creating a System Alert. Create New. So, we will Select the Task Manually. Yeah. These are tasks that exist in Qlik Sense, right? Yes. So, we will check if this reload fails. Maybe, we should set it to Success for this demo since we're not expecting it to fail. Yeah, because we just want to show how to receive and configure System Alerts. So, we click Next. Save. Okay. Let's do a reload. So, here we go. It was success. That's great. And I see there are links to the Sense QMC and the Alerting UI. Now if that were a failure and you wanted to find some more information, where are the log files located? Yes. So, we have two paths where we can find Qlik Alerting logs. But starting from June 2020 release, we can find more detailed logs in C:, Program Data. So, there's a log folder there for each Alerting service? Yes. So, those are the logs. And if I open one, you can see it has a Timestamp, Hostname, Version,… So, we have really detailed information here. Great! So, these are curated logs that are relatively readable. You mentioned another location? If you wish to have more information, you can go to Program Files, Qlik Alerting; and for this service, for example, Email Messenger. We have this folder called Daemon. So, we can see this ‘.err’ where we can see errors, but… Yeah. I see that's just a string of commands with no time stamps or anything. Yes. Well, that can be helpful. Hopefully people won't have any issues where they need to go that far, but it's nice to know that the information is there. Are there any other tools for administrators to keep track of all the Alerts that are generated and how they're doing? Yeah. Qlik Alerting includes a Qlik Sense monitoring app that could help you understand and visualize what is happening within your Qlik Alerting environment. Okay. How do we set that up? Yes. First thing we need to do is to create the Qlik Alerting Data Connections. So, to do so, Qlik Alerting includes a BAT file that can help you create and configure these Qlik Alerting data connections. Oh, that's great! Where do we find that file? Yeah, we can find it in C:, Program Files, Qlik Alerting, Setup, Monitoring-Connection. And we have these BAT files. So, we just run it. I will run it As Administrator. We give the username of the root admin. Okay. And now, if we go to QMC, and we go to Data Connection; we should see something called Alerting: monitor_alerting and monitor_alerting_login. Oh, that's great that that one batch file created those for you. Exactly. It is easy. Next thing we need to do is to import the monitoring app. Okay. We go to Apps in QMC, and we will import it. And this app comes with the Qlik Alerting installation? Yes, from Program Files, Qlik Alerting, Setup, Monitoring Connections; and it's here. So, we open it. It has successfully been imported. So, now we go to the hub. We open that. We just need to change one variable to be able to match or Qlik Alerting server name. Okay. So, we need to look for the following variable. It's called: ‘vServer.’ We can select: ‘search in all sections.’ So, first place where we can find this is here. So, we just need to change it too match our Qlik Alerting name. Okay. So, it's just getting the name of the machine where Qlik Alerting has been installed? Yeah, exactly. So, we change the port to 443. There's no need to add it here. So, we now click ‘Load Data.’ All right. So, now it's loading all the information about the tasks the Alerting tasks, and Alerts that have been triggered, and created; and making that visible in an application? Cool. Yeah. We click ‘Close,’ and we can now go to the sheets. This is an overview, but you can also check more information here. So, all of the Data Alert Distribution… There's a lot of data here. This is great. Yeah. So, if anybody was looking for some more information, what are some resources that they could use? So, yeah. If you wish to find more information, you can go to Help.Qlik.com, and just look for Qlik Alerting product. We also have a guide here. So, you select - be sure that you select your installation here. Which version. So, I will select; yeah. I will select May 2021, which is the latest one; and click on Guides. And if I click here, it will just open Qlik Alerting Guide. Great! So, if you wish to have more information, you can find it here. It's nice to know where the documentation is. Okay, well I think it's time for Q and A.Go ahead and submit your questions through the Q&A panel on the left side of your On24 console. Patricia, which question would you like to address first? Okay. So, we have a question here: it's only Windows authentication Supported, other IDPs like Okta? No, for the moment Qlik Alerting only Supports Windows authentication. Okay, and next question. Yeah. Is it possible to install Qlik Alert into connect to QlikView? So, no. It's not possible. You can only connect to Qlik Sense on Windows and SaaS as it comes already as a feature, so no need to install anything. But it's not possible to connect to QlikView. That is definitely a nice advantage of having SaaS; that's already set up already there. So, do all services need to be in the same server for Qlik Alerting? Yes. There is no option to separate them. At the moment, only MongoDb and Redis; and from February 2021, Redis is not necessary anymore. So, only those ones. All right, I see another one. Is there a limit about users that can be imported to Qlik Alerting? No, there's no limit. The only limit that we have is that we can create groups up to 1000 users. But there's no limit to the number of groups you can have, no. Okay, next one. Why there are two different ports to access to Alert portal? Yeah, because one needs for secure communication which is HTTPS; and the other is plain HTTP protocol. So, you can have one of them closed in your environment; so, that way you can modify the default ports to 443 to HTTPS and 80 HTTP. Great. So, can you install Qlik Alerting in Azure? Yeah. At least for Windows; if you have a virtual machine that is running in Azure, yes. You can install it there. Okay. So, we have another one. Is it recommended to reboot Qlik Alerting and Qlik Sense after installing Qlik Alerting? There is really no need to do so, but in case your Qlik Sense license do not get refreshed in Qlik Sense side, then you might need to reboot Qlik Sense to refresh Alerting attribute. But this is from Qlik Alerting in June 2020. What version of Qlik Alerting should I be running? So, I would recommend running in the latest one; or at least, let's say if you are in Qlik Sense February 2021, at least I you should be running Qlik Alerting on the same version of Qlik Sense, or at least one after one or two after. I mean not one below from if you compare to Qlik Sense. And now the latest version of Qlik Alerting is May 2021, which is already available to download. Okay Patricia, we have time for one last question. Yeah. So, last one would be: where can we find more custom notifications templates or examples? So, you can find more examples in in our Help page. So, if you go to Data Alerts, here. And Managing Custom Notification, you can find a lot more complex. You can look for elements. You also have examples of how to embed web events into emails here. Right to Left Language Support; so, you have a bunch of examples here that you can use to customize your Alerts. That's really great. Well, thank you very much Patricia. Yeah. Thank you everyone for joining to this Qlik Alerting session. I hope it was useful information for you all, and yeah thank you so much for joining. Thank you everyone. We hope you enjoyed this session. And thank you to Patricia for presenting. We appreciate getting experts like Patricia to share with us. Here is our legal disclaimer. Thank you once again. Have a great rest of your day.
Hello everyone and welcome to the May edition of Support Techspert Thursdays. Today's presentation is Top 10 Visualization Tips for Qlik Sense Enterprise with Patric Nordström. Patric would you kindly tell us a little bit about yourself? Thanks Troy. It's good to be here. I’ve been with Qlik since 2017. I came aboard with Qlik with the Idevio acquisition. I’ve been working in the Qlik echo system since 2007 or something. I’m the product manager for Qlik Sense Visualizations and Qlik GeoAnalytics. Okay. So, that's what I do. Great, thanks. And what will you be sharing with us today? Most of my time I spend talking to customers, listening to their problems and suggesting different solutions; coming up with requirements for R&D. In many cases, the customer doesn't know the product that well so. There's so many things that you actually can do with the standard settings inside of Qlik Sense; so I see it as a bit of a mission to remind everybody what a great tool it is and how many things that we can actually do within Qlik Sense without having to build something very special, because it's so flexible. Yeah, I know you have a lot of wonderful tips here. I don't think we'll go through them all, but I thought we'd start. Since being an American, I carefully watched the presidential election recently, and trying to explain the intricacies of how that works can be challenging. And I understand you have a solution for that? Well, I was playing around with decision trees lately. It's useful in many different applications and one of them is for visualizing the outcome of elections. Okay. I was looking for some public data to play around with and the election is of course quite fun to play with. Certainly. We have a couple of charts that are suitable for this type of data. One of them is the Org Chart. It's just a table with nodes and information to the parents. The parent would be the Node ID; just a table linked to itself. And that's a default out-of-the-box diagram, right? Yeah, so it comes in the Visualization Bundle. So, it's over here in Custom Objects > Visualization Bundle. Okay, there it is. So, it's meant to visualize company rosters, but it can easily be adapted to show the outcome of decisions. This tree shows the possible outcomes of the swing states. Okay, and how would you interact with this? Yeah, so this is how we start. And then we decide whether we think the Democrats or the Republicans will win in Florida. So, if we think the Republicans will win, we will come to the next swing state which is Pennsylvania, and we can see the current state. If Florida is won by Republicans, Pennsylvania's by Democrats and then the Democrats win Ohio, and the Democrats wins. So, it's a nice way to show a rag tree, not completely balanced. As soon as any of the parties win, it will actually show that in a darker color. Of course, the Republicans had a different position to start with, so they have to go through a number of elections before they call it a victory. Right. So, it's a nice way to traverse a tree. It doesn't show the whole tree at the same time. If you're more interested in the complete decision tree, you can instead show them in a Radial Denogram instead, which is the same type of tree, but I’m now showing all the nodes at the same time. Okay. The Org Chart showed the beginning of the tree in the center, but this one is expanded to show all the branches in the tree. You can highlight the possible wins for the number of ways the election can go, well estimated to go. And you can also highlight when it can be a tie; which is quite… Yeah, rare. …interesting that it actually can be a tie. The last was the Org Chart. What's this it in this case? I’m using the Map Chart instead. And this could be good for displaying any kind of decision tree, right? Of course. I mean you could use this for any type of decision tree data where you have some kind of pattern, where you have a yes or no decision. One outcome leads to another, and you want to show the different types of possibilities. Okay. I’m just playing around with radial coordinates, but you could also use the same tree and have a different coordinate calculations, and then you would have a flat tree; and then you could play around with it if you want to have different types of perspectives: up or down, or left to right. It's just the transformation of the coordinates. I love that putting it on a cyclical group, so you can switch quickly between different visualizations, and see the same data shown different ways. That's cool. Yeah, it saves space as well. I mean most of the time, you have many ideas when you want to build a dashboard. And the first thing that you do is that you try to cramp into nine different graphs and diagrams and tables and everything at once. You win in both how the user experience the app, and how well you can perceive your message by showing bits and pieces of the information at a time; not revealing everything at once. I know a lot of times especially with data from the election, people are trying to apply it to a map. But do you have a way of focusing more on the data points, but still having some sort of geographical elements to it? Yeah, so that is also something that I’ve tried to build out, showing some kind of simplistic way to show geography. Yeah, exactly. We do have a Map Chart and you can see the geography is exactly how it is; but you want to show KPIs that doesn't have that much to do with geometry. Yeah. So, this is a way to present a simplistic way to show maps as squares or hexagons or circles. In many cases, you have to do this by hand. Instead of doing that, I used an Excel spreadsheet as a template. So, I organized the states in a row-column order, just to place them and then I can move them around. It's not exactly how the Earth looks like. Yeah. It serves its purpose to show some kind of simplistic map over the U.S. instead. Exactly. It's kind of elegant in its simplicity, I like that. Yeah sometimes, it's the actual geometry gets in the way. And by modifying this grid, you want to do it for other countries, you can just change the names of the states for the region that you're interested in and reload it. And you will have this type of cartogram also for your special region. That's great. Now, you're obviously an expert on charts and visualizations. In your experience, when users first notice a diagram, what is it that stands out to them? What do they focus on? Yeah, so that is another thing that I usually bring up all the time is that: the first thing that they look at is the title and the subtitles. So, they spend more time reading that instead of reading the plot, which is quite surprising to us; that they are so focused on the actual charts. You don't really know what the user will make out of the dashboard, but I think we can come up with better titles. I mean the easiest way to do it is just to have the dimensions showing ‘Sales by Year,’ say ‘Shares by Category,’ some kind of explanation. But you can actually do it do it better in my opinion. If you look at the chart, what am I trying to communicate? If it's a line chart showing sales over time, then perhaps I’m interested in the trend. And then it makes sense to show how big was the increase or decrease in the last period. And we can also show the total increase over time. You could do that by building up the titles dynamically using Set Analysis perhaps together with Maximum and Rank to build up the titles. Then the titles will be dynamic. Even if the user is making a different type of selection, or the data is reloaded; the titles will still be valid. We do have a Snapshot functionality with the Storytelling, where you can add Annotations to the chart. Yeah. But this is a way to build some kind of dynamic, that will be true whatever kind of data you throw at it. And you could do this for the Line Chart, but you could also do it for the Pie Chart. If Pie Chart is probably something about shares, so you probably want to highlight the largest share, and then you could have the second and the third. Bar charts is mostly about comparisons. So, then you could have the title express how big is the largest value compared to the second one, or you could also incorporate what the top three represent in the total value. That's a much more powerful title, get a lot more information immediately from the chart than just the default. Yeah. It's an eye-opener at least to me, when I when I played around with it, that it actually carries much more information that. Yeah. I’m lazy at this as well. I usually said just some boring kind of titles, but it really pays off to make it a bit more elegant or try to use that space that you have. It's really well done. You mentioned: you built these dynamically using a Set Analysis. Is it possible to take a look at how those are built? Absolutely. It's not like rocket science. If titles is switched on, I just picked the maximum year divided by the year prior, and then I could see whether it goes up and down. It's not that that hard; especially if you know a bit more how the dashboard should be used. Then you could probably, you could tweak it even further. In this case, I’m also using the new function of the Dimension-based Reference Lines. So, I throw in a little Annotation inside of the chart, and that's goes over here. Okay. I picked the last year like that. Just to place it somewhere, and then I add the title the increase or decrease within the, in the label. And then I have a show condition, in this case to switch it on and off to make the point. Is that, do you know when that feature became available? So, that was one of the latest editions. So, okay. I know that's a common problem that comes into Support actually. We get questions all the time about what features are available in what version, especially when it comes to on-prem customers wondering when they should, to what version they should upgrade. So you have a way of keeping track of that? Yeah, I have to put it in an app; but otherwise I can't keep track of that. So, what are we looking at here? I’m keeping track of all the releases in or all the new features in each release that are relevant to my, to my area. Okay. Dimension-based Reference: already in 2020 September; so it's been around for quite some time. Is this, is that app publicly available? It's not, but maybe it's something that we should have put in, because there's a lot of people asking for it. Yeah. But I tend to meet more and more cloud customers. And they do already have all the functionality. They don't worry so much about which version they're on. I like to keep it also to show the progress, that we are actually adding a lot of functionality to the product all the time. Do you have a favorite kind of chart, or one that you've used a lot recently? Yeah. I can't help myself. I was planning a building project at home. You have to apply for the building permit; you have to get the permit; you have to wait a while before it becomes in effect; and then you have to order the stuff that you want for the building. I decided I want to show it in some kind of calendar, with each one of the months together, with the weeks. And then I added some sliders. If it takes shorter time for me to get a handler, and maybe they are quicker to give me the approval. In many cases, you see Gantt charts for project planning. But in this case, this was for our community. I wanted to talk to people that didn't perhaps play around in dashboards that much. So, they were more comfortable showing data in a calendar, because most people have seen a calendar. Exactly. But what kind of object are we looking at here? This is the Trellis Container. Okay. Instead of having to make seven tables, I can just make one table and build out the functionality inside of it. I have added the days, then the weekdays. Whether it's Monday, Tuesday, Wednesday; and some background color to show if it's a weekend. I added my activity last, the week number and then I have four variables that adjust the timing for things. It doesn't take that much data either. The only thing that I placed was the variables, and then I built up a calendar from the start date to the end date. I’m just auto generating all the dates between max and minimum date. That's really cool, actually. Just be able to quickly see how it all these different decisions and times will affect your end date. Yeah, it becomes really evident. People are asking: when do you think we will actually have this in place? And then it's an easy way to show people, understand that this thing has to be in place before this one, and there's a certain amount of delay time before we can actually have it in place. A common challenge in my department at least one that we face is testing new changes in a system to kind of analyze if there might be improvements and where. How would you recommend visualizing that for that? A and B testing is probably the most popular analysis that you could do. Okay. Of course you could use Qlik Sense both to calculate the statistics behind the A&B test, but also visualizing how well you can rely on the data. You have two groups: one where you let them show the difference or the change; and the other one keeps the old. And then you try to see if you have some kind of confidence that the changes that you made really made an effect on. In this case, it's conversion. So, I like to show it as Bell Curves. So, this is the conversion rate for the control group; and this is the conversion rate for the variance group. And then just calculating conversions between visitors. There's a conversion rate uplift is the percentage of how much better is this in the conversions; and then you can calculate the standard deviation. Said value, P-value and in this case. It's all about the P-value, so if the P-value is below 0.05 then you would have a 95% confidence over significant test results. I added a slider where I can change the conversions in the variance group. So, if I increase it, I have higher confidence; if I decrease it, so few other conversions, I can see that now it's no longer statistically safe, or I can't be sure within 95% that the change is a significant test result. And that Bell Curve is kind of a powerful visualization as well. And it's quite easy to build up in Sense as well. I mean you already have the normal distribution as a function in the engine. To me, it resonates well. Of course, you could show it just as a table. I like the way that it spread out the estimate of the normal distribution. I know in Sweden when you're looking at calendars, and stuff like we were before; weeks is very common. Do you have a way of making custom times, like with weeks instead? When you drop an Excel file inside of Sense, and you're loading it. It will actually create a master calendar for you, and it has some basic settings. But it maybe it's not that super well known that these time-aware charts, when you have a continuous axis, you can actually decide how they should look like. The normal standard way is probably to show years, and then months, and then dates, perhaps. But if you work a lot with weeks then it makes sense to show week numbers instead; which is quite popular in in Sweden. Not so much in the U.S., perhaps. Yeah. When you build up the calendar, you declare derived fields (so to speak), and then you have expressions for the different parts of your time axis. The meta information that goes in (that decides how the time aware chart should look like) is the way that it's tagged. So, dollar axis (that will actually show on the axis) and then you have two different ways to show qualified: that means that this axis will show when you are zooming in. And Simplify: this is the axis that will be shown when you're zoomed out. And you could also have helper expressions that will not be shown in the UI, but you can if you want to build up something more complex. So, now I’m sort of zoomed out. I show the years, and then I show the week number for the first week in each quarter. I can't have all 52 weeks, because then it won't show me any week numbers. As I’m zoom in, you can see the individual weeks. And at a certain time, it will switch over and show me the individual week numbers instead. And if I go even further, it will actually show me the dates again. Oh, wow! Even all the way to times. That was pretty cool. Yeah, as we switch over to week numbers, so we show all the week numbers. We also change the quarters to show both the year and the quarter number. Otherwise perhaps, you get too lost when you are looking at week 27, whether you are on 2020 or 2021. That's good to have that additional reference. You can build up your own fiscal year, perhaps. In Sweden it’s quite common to have fiscal years that start at a different month. Could have your fiscal year starting on May 1 for instance. But there is actually some customers that have a fiscal year that starts at a certain week number. Then it becomes a bit tricky, because if you have a year that started week 36; then it's no longer symmetric. You will have years that are different length depending on, because week numbers are kind of sliding. It's easier to calculate or build up an offset if you start on a month rather than a week. But you could still build something. In this case, I’m playing around and using colors instead. This is a customer of ours that have a fiscal year that started week 36. The week numbers are correct. So if I zoom in here, we can see that they actually starts at one. But it's actually week 36 in this case. Okay. And I’m using the color functionality together with the time axis to show where the fiscal year starts and ends. Now I know you have a lot of tips and recommendations there in this app that you put together and developed. We don't have time to go through them all of course. Do you have any general advice for developers who’re trying to decide what is the best solution to the problems they're trying to solve? Yeah. In many cases with design or anything, that you usually try to add too much stuff. And you're not done when you have no space to add things. It's probably when you should start to take away things that are less important in order to highlight something. So, you're not done until you have taken away everything that is not relevant any longer. And especially if you work with electronic dashboards and not with paper reports; then you have all the possibilities of adding interactivity that can fill the gap. And the human brain usually works really if provided an overview first; and then you get details later. So, you could use when you hover over something, you can get a custom tool tips that provide additional information for that space that you're interested in. Of course, the whole paradigm of Qlik Sense: with making selections and narrowing down the scope that you're interested in, also helps a lot in order to consume a complex data set. Great! Thank you very much. Okay, now it's time for Q and A. Go ahead and submit your questions to the Q and A panel on the left side of your On24 console. Patric, which question would you like to address first? So, we can go from the top. What's the most effective way to display filters? Especially if there is a number of them are used to struggle to understand on how to use them? I would say that the most common used filter is the Selection Bar. So, when they started to use some kind of selections that is the most peoples are interacting with, then you have the possibility of adding filters to your to your page, or to your dashboard to help the user for an initial selection. And that's what filter pane is for. I’m not sure if everybody knows how the filter pane works. Because if I’m adding something to the filter pane, I can add additional fields to it. If I add a lot of different ones, it will actually collapse and help you to keep them organized. If you want to make selections on it, you can still do it; but you can still also you can expand it. So they can get easier full screen view of all the selections. I like to work with the titles with the filter panes. Instead of just showing the title for the field, you can actually (like I did here in my feature overview). So, instead of just showing Year here. I’m showing: if I make a selection, it will actually show me that in the filter pane. Of course, this is also included in the selection bar; but if I make another selection, I have a dynamic title, so instead of showing Year, I’m showing the actual values for the year. It works really well if it's something obvious; like years, because you don't have to spell out that this is a year. Most people recognize that directly. And you can do that with other things as well, so. That's nice. This is 2021 button, and canon, and you could have a look at it that at the same time. So, that's my best tip for selections. I also, if you're making some kind of guided application where you want the user to click and make a certain type of selection; you can help the user to have - you have buttons that you that the user should hit. And you could apply either a selection directly, or a bookmark. Okay. Instead of showing everything. There's also the Global Selector. That will show you all the fields which; I don't know how many users are actually using, but this is a quick way to get to see all the data that is available within your side of your app. All right. Next question? Yeah. Next one is: can we evaluate which visualizations or metrics takes the most time to load? So, in my experience, when you visualize something, it's much about the actual metrics behind. So, the expressions that you make, that is what can actually take time to render. So, I usually try out all my expressions first, before I, or especially if they take a lot of time. I try to debug and try to write better expressions just by looking at them populating a table, perhaps to start with. And know that they are actually producing the result that I’m interested in. Right. That makes sense. Testing is always important. Yeah, I mean especially, you probably start with some kind of subset, and then you load the full data. And it might be behaving a completely different way. So, it's, you have to work with the app through the different phases I think. Okay, next question? Where can we find more information about Qlik Sense extensions and maps? So, it's all on the Help Site. It's organized with the different products, so I would say that if you're looking for reference information, then you will have all the information at the Help Site. Then there's additional information; perhaps more tutorial specific ones. The one thing is the reference documentation, but you sometimes you would need, you want to do something very specific. And then reference documentation might not take you all the way. So, there's the Qlik Design Blog of course. And then at the Qlik Community. We have different forums. So, if you're going to perhaps for Qlik GeoAnalytics, so. I notice you're very active there. Yeah, I try to spend a lot of time here. So, then there's a tutorial section where we could see the in the document section for GeoAnalytics, where you can find tutorial built out tutorials if you want to load a shapefile for instance or if you want to do something more specific. Okay, next question? Can you give an outlook which functionalities from the on-prem version will follow in the cloud edition? Which gaps will remain? So, most of the things we release in the cloud version first nowadays. And then we gather all the new functionalities, and they will go into the on-prem version four times a year. So, if you have a cloud account, it's good to keep track of all the new features; because you get them immediately in Qlik SaaS, so that is a good way to keep track of new things.
When are you going to have out-of-box colors font borders, etc.? So, you might have seen that we have added a lot of features. I think we pushed up more than 96 different features the last five milestones. So, there's a lot of - we focus on three different areas. We push out new charts, completely new charts, new functions inside of the charts, more like accumulations and trend lines, etc. And then we are also adding custom settings. So, in the Line Chart for instance, you could have dashed or solid lines, you could have curved or straight lines. So, that's type of short specific settings. We will also add more common settings to, like that are more common for the short background; perhaps the title. So, that is something we want to add, but and we will add it continuously, released by release. But we wanted to start out with the most important parts first. But there will be more settings like that as well. It's relevant (I guess) for people who want to have more control over their application; and want to build something very specific. You can control these settings already today by using a Custom Theme. So many of these are exposed that way you could also use css to tweak.
And if you want complete freedom to do anything that you want; perhaps you should also - you should build a mashup instead. Leave the Sense application and just embed the charts that you like, and you then you can tweak it any way you see fit. Okay, Patric we have time for one last question. We can go: which is the best way to connect with to see more of these type of visualizations? I would say to stay tuned to the Qlik design blog. There's a lot of nice articles posted over the years on how to design and how to build for this specific app or for. This app with tips, it's part of a series. So, this was part number five. So, there's four other apps that you could, they're all available on the Qlik Community. So, if you search for in Qlik Community and “Top 10,” you probably get a hit for this one. And they are linked together, so you could explore them. The Qlik Community is a great resource for sharing and for finding inspiration to build things with Qlik Sense. That's really great, because I’m sure a common question amongst our attendees is ‘where can we get a closer look at this app that you've been sharing?’ Great. Well, thank you very much Patric! I think this will be helpful for a lot of people. Yeah, thanks a lot for having me here. It's great to be able to spread the word on how things that you could do with Qlik Sense. Okay, great. Thank you everyone. We hope you enjoyed this session. And thank you to Patric for presenting. We appreciate getting experts like Patric to share with us. Here is our legal disclaimer. Thank you once again and have a great rest of your day
Q: What is the difference between "*" and "ALL" in reduction field?
A:The wildcard character * in the data reduction column refers only to all values in the security table. If there are values in Section Application that are not available in the reduction column of the security table, they will be reduced. ALL is just a value I createdandreference in, which I have control overand use to help visualize the relationships.
Q: Qhat is the best practices when we have a section access based on 5 fields?
A: Create a composite key in a link table or fact table. This is your junction box.
Q: Sales person to see his sales plus sales for another departments of other Sales person?
A: Create an aggregate set of facts in your fact table that all users would have access to.
Q: Do you have to reference the long USER ID string in QS SaaS as the USER ID in Section Access, or can you use ADFS usernames?
Q: Can we reduce data by numeric field that is a dimension? Can we reduce data using wilcard? For example by fields that begin with dim*?
A: Any value can be used for data reduction, but wildcards are reserved so you will want to create a secondary table to map your strings.
Q: Is there still a risk of locking yourself out of a qvf app like there was in QlikView?
A: No. You can always open without data.
Q: Although section access works with NPrinting for Qlik Sense, show/hide columns don't work with NPrinting from what I have read.
A: I have used the container show/hide on charts in NPrinting. You could also make your condition in the master item which should propagate correctly.
Q: Can i simply use multiple Reduction Fields?
A: It could create a circular reference, which is generally not desirable or could create unintended results. A composite key is best practice. - Basics for Complex Authorization
Q: In data script: what's the best practice to build up temp section access table by combining different datasources (excel, qvd) before using that under section access?
A:This is the same as building any other table.There shouldn’t beany adverse impact by concatenating tables from multiple sources for the purposes of section access.
Q: What condition did you use to show the Qliklogo vs. the section access data?
A: It was just container similar to what I did with the scatter and distribution chart. I used a text object and a filter pane in a container. It was ‘count(USERID)= 0’ and ‘count(USERID) <> 0’
Hello everyone, and welcome to the April edition of Support Techspert Thursdays. Today's presentation is Troubleshooting Section Access and Qlik Sense Enterprise with Mike Reese. Mike, why don't you tell us a little bit about yourself? Thanks Troy. I’m a senior solution architect with Qlik. My main focus is helping guide customers in their Qlik journey through pre-sales, and I actually have 14 years total experience with Qlik; over six now actually working for Qlik directly and eight in the partner space: consulting, wearing various hats, implementation, architecture design, development, project management, you name it. Great. So, Mike, what are you going to be showing us today? Today, I will be covering Section Access. So, what is Section Access? We'll go through a couple scenarios of role base Access. We'll do some impersonation. So, you'll get a kind of a good understanding of how you could get into an application as a different user; actually test their entitlements and easily flip between the different types of Access that you might need to implement in your security model. So, then we'll also see how that is applied through the load script. And then at the end, we'll go through some data modeling tips that help facilitate Section Access; and different levels of facts, different types of facts, multi-fact table and being able to set up security in a seamless way. And then also, we'll go through our takeaways and tips and tricks. Great. So, how does Section Access work? Good segue. I just have a couple slides to help frame the demo, and then a few more to reinforce what I show. Data authorization using Section Access is implemented using reserve words as field names and values for your entitlements table where Access levels aren't assigned to users within the Section Access part of the script. These tables must contain a minimum of two system fields; Access being one of them. As you come in, a user logs into the application; they're either designated as an ADMIN through this Access keyword, or a USER. And if you're an ADMIN, you have Access to all the data. And if you're a USER, then you have Access to reduce sets of data based on whatever is designated to you in this entitlements table that's set up in the load script. Okay. So, it's all about controlling the data that's shown to the user based on their profile? That's right. So, literally you're setting up an entitlements table and through the power of Qlik’s Associative Model; those associations designate what a user can see and what they cannot see. That sounds like the basics of how Qlik works. What's the difference between the association of any table, and the Section Access table? Perfect. So, to help visualize the mechanics, we have a Qlik data model accompanied by a Section Access table. And all we need to do is associate one field between the data model tables and the Section Access tables. So, in this case we have ‘Department.’ And this would work with any field, right? It doesn't have to be departments; whatever field you want to designate as your security field. And so, in this case, it does work exactly like the associative model. So, then the difference is mainly it's distinguishing between the security table and the application facing data model through the use of the reserve words that I mentioned; that I promised to talk about at least once more through this demo. One of the others that I’ll cover right now is the OMIT; which is used to designate the names of fields that users should not see. So, basically if you're looking at this “John;” he would not see the social security number field. And neither would “Steve.” And so, something unique to Qlik is that you're now you're seeing the benefit of data reduction. That's row level security through associations the way Qlik works natively. And then you're also seeing the ability to apply column level security through OMIT. Okay. What if I have more than one field that I want to omit. Yeah. So, that's actually perfect. So, this is what the data model looks like with the security table. So, if Section Access was turned on, the security tables wouldn't be visible in the data model. And they'd be hidden because they're security tables. But one of the things I’m going to cover in this demo is how you can easily test Section Access through the impersonation. What this example is showing you is Section Access is turned off, and that's simply by commenting out the Section Access declaration. And we'll see more of that as we go through it. What I’ve done is: I’ve created an ‘omit group’ and a secondary table to just list all the fields that I would need to omit for different users. So, basically it's a one to many, right? So, instead of having in the example I showed you before: one user and one omit field; this just gives me a secondary table to have that one-to-many relationship. So, if I have three fields that I have three rows in this table per user, as opposed to three rows in this table; now I could do it either way. I could have multiple rows per user for each field in this authorization table that I set up; and this would be my only Section Access table. But I just set it up as a second table just to make it easier to see, right? So, this is this would be one-to-one per user, and this would be one-to-many for all the omit fields. Okay, and now when you talk about turning Section Access on and off; is that like a setting in the QMC, or how is that done? That's actually done in the load script. So, you can see here I’ve added a couple screen captures of that Section Access declaration. Any script that falls below the Section Access declaration and before the Section Application declaration is all interpreted as data reduction and Section Access security. So, effectively all I need to do is: comment that declaration. You'll see those two tables are here. With it turned on, those two tables are now gone. This is what the user would see if they were in a published application with Section Access turned on. What I’m left with is this floating island that I use for some user-experience things related to what the users can and cannot see. So, I understand you've got a demo set up for us. What version of Qlik Sense are you going to be demoing? I’m actually going to be demonstrating Qlik Sense SaaS. Okay, there are some similarities. There are a couple differences that I’ll cover throughout this, but everything is moving towards the cloud, so I figured why not show that the latest and greatest that that Qlik has to offer. Yeah. What we're looking at now is just something that that we have set up it's called ‘sPortal.’ You could actually get it on Github. It's just a way to impersonate different user Access. Three roles that I’m going to demonstrate would be: like, myself; calling myself the CIO. I have Access to all the data. I develop things, I’m a tinkerer, I see the whole entire application, all the data, all the visualizations. The other person I’ll log into this with is ‘Claire,’ who's a marketing lead in the North. And she has visibility into a couple different territories and sales data. And then lastly, we'll log in as ‘Lisa Grisham,’ who is a sales director. And she has visibility into individual rep data in the South territory. All right. So, here we have the Qlik Sense Hub. And what I’m going to do first is: get into a published copy where the security is already applied, right? So, here I have my published copy which is indicated by this red. This is published to the Everyone space, versus this one which is in my Personal space where I’m doing the development. And that's where I can do my role impersonation. So, we're going to start here. Okay. Now yeah, can you walk us through this app a little bit? So, I’ve got a few sheets to help demonstrate the security. I’ve got a Sales Op tab. I’ve got a tab to show the associations and the reduction. I really want to emphasize that the security in Qlik Sense, the Section Access Data Reduction is - it works the same way as the associative model. You're logging in, Sense is clicking on a value behind the scenes, and then it reduces that data set to everything that's related to that value. Can you tell us what the colors indicate? So, the OMIT groups that I had designated are encompassing the fields that would be hidden based on your access rights and your entitlements table. Remember that OMIT keyword? The Contact Info group is going to omit Phone and SSN for the users that are in the Contact Info group. If you're in the Office Group, you're going to lose visibility into Office City. And if you're in the Salary OMIT group, then you lose visibility into the Salary field. All right. So, I’m going to jump into this Sales Ops tab. And as a user that can see everything, I see the total sales for the organization so that's $422,000. Note the logo here. That's going to change when I when I come in and do my testing with Section Access turned off, so I can do some impersonation. And then over here, you're going to notice this is a scatter table that lists all of my sales reps. Right. So, there's a number of different sales reps here. Down below, we have ‘revenue by territory and customers.’ So, these are all my territories. And then I can actually drill through to the different territories. And that's important, because when you see the data reduction, you're going to see how this this table is affected. In one instance, you're going to see it reduce down to two territories. And another, you're not going to see any territories. It's automatically going to be reduced to the customer level data, because a person only has visibility in one territory. So, this is if we're logged in as an ADMIN, right? That's right. Yeah, somebody has visibility to the total data set. Okay. And then, just as we were talking about the color-coded columns, these are the three that I’m actually applying: it's the Salary, Office City and Phone. That is the main thing here. And I just want to come into this Associations Tab. For those who aren't that familiar with Qlik Sense in the associative model: when I click on a value, all the associated values are in white, and anything that's not associated is grey. So, that's the secret sauce. So, security literally works the same way. So, if I login as a user, and all I can see is New England territory, I’m only going to see data for the city of Newton. I’m not going to see any of these cities. These are going to evaporate, and I would say “before my eyes,” but it would be actually before I even get to see anything. So, obvious that's security, right? I’m not going to see anything related to Reps that aren't in white. All these reps are not in my view, they disappear. But as somebody that can see the whole data set, I can see all these associations across all my data. So, can we test this with one of the other users? I’m going to go in as Claire. And Claire is a marketing lead in the North, so Claire has visibility into two territories. Okay. And now Claire sees this, right? So, same application but the difference is: with no selections applied, I’m only seeing $212,000 in sales. Right, that's gone down, okay. That's right. So, the data reduction is working. I still see all the visualizations; however, this visualization has changed. You might recall, in the previous application, we were looking at a scatter chart. Now we're seeing ‘Sales by Product and Territory-Distribution,’ and that's because she's a marketing manager. And this, products are more relevant to her than a scatter chart of sales reps. So, this demonstrates using Section Access to really drive what visualizations a user can see, right? So, you can do that at the object-level. You can do it at the field-level. You can also do it at the sheet-level. Notice now the territories have been reduced. We had a number of territories at the seven or eight or something like that. Now, we've been reduced to two territories: Canada and New England. Now, I can still drill through, right? And I can still get to the customer level, but I just don't see all the territories that I wouldn't have visibility into. And I see that Salary field is missing as well. Exactly right! So, now that green field is gone, because I shouldn't have visibility into Salary Data as a marketing manager. And so, that's it. That's actually data reduction and object-level security at work through Section Access. So, now they go into associations again. That green Salary column is gone, but on the right-hand side of the screen, we can also see that the reps have been reduced. To really emphasize that, let me jump back into my other example as the administrator. Here's all the City data, here's all the Territory data, right? Flipping back over here, you can see that those data is reduced. And I’m not seeing the gray. Right. Because I don't have visibility into it. So, that's what the data reduction is doing. It actually is removing that data from the application. When you have Section Access turned on, does it actually affect the performance of opening the app? Yeah. So, it could. If you have a very large application, and the reduced data set is much smaller than that, then the perception of the user is going to be: it's taking a, it could be taking a while to open. Effectively you're opening the application if it's large, and then you're reducing it; versus if you had a smaller application from the get-go, that opening time would be somewhat quicker. But and I would say in most cases it's negligible. To be clear, that's just the first time the app is opened after reload, right? Not every time? That's right. And who's the third user we're going to test? Yep, let's do it. So, the other user now that I want to get in as is Lisa. And Lisa is a sales director. So, she only has visibility into one territory. So when Lisa gets in, now we see a different picture, right? We only see $51,000 in sales. And instead of seeing that distribution chart that we were just looking at, we're still seeing, we're seeing the scatter that we saw originally. But now it's reduced to three reps instead of the whole sales force. Yeah. And down below, we're looking at ‘Revenue by a Territory and Customer,’ but it's only the Southeast territory. So, it's automatically drilled down to what territory Lisa can see, and just the customer review. So, that you're seeing what's relevant to Lisa. And over here, we no longer have visibility into the personal information of the employees. You can see that now: you have different levels of access for different users, and different columns are being hidden. Great. So, now I’ll go over to the associations to show the difference in what Lisa can see versus Claire. Okay. All right. And now you can see a much smaller set of data. You can see much fewer rows, and we're only looking at the Southeast territory now and the city of Raleigh. The reduction is doing exactly what it should be doing. Associations are still working, right? So, I can still see that, but it's only at the level that she has visibility into. So, how are you able to manipulate the app to show and hide the different charts and fields? So, let me go back out to my hub here, and go into my unpublished copy that's in my personal space. So, now we're actually opening this with Section Access turned off, and that's that that one Section Access declaration commented out. Okay. Instead of that Qlik logo that was in the corner here, I’ve got this this actual drop-down box. So, this is one object that's actually using the show and hides. Hiding the text box with the image in it and it's showing me a list box. And now I have actual visibility into user-level information. So, if I want to impersonate Lisa or Claire or myself, I can. All I need to do is click on these. This is working just like the associative model works. Right. So, those tables are still in the data model, but now they're no longer Section Access tables; they're just a part of the regular data model? That's exactly right. They're just regular old tables, right. So, you can see as I click it, I’m seeing the visibility that I would see as if I logged in and everything was being reduced. So, how do you configure those show and hide settings? Let me actually get into edit mode. I’ve got some sheets that I’ve duplicated here, and now I can actually edit the sheet and see what's actually driving those show conditions. So, I’ll start with the table here. This is the probably the easiest thing to look at first. So, we've got the Salary field, Office City and Phone. So, the one thing to keep in mind is when you're using Section Access with column-level security, if a column is omitted and it's in a visualization: the visualization won't render. It'll actually fail. You actually have to put in a little protection for that if you're going to omit the field for users that should see that visualization. So, one option is to hide the column or swap it with a different column, or you use a container with different visualizations in it; and I guess the third option would be: show them one sheet versus another. So, I’m going to start here at the field-level in a table, and we'll just look at the Salary expression. Here is your Show Conditions: ‘Show Column If,’ and what I’m actually checking is that OMIT group, the field that I created, and if the OMIT group is equal to Salary, then that means don't show Salary, otherwise show it (-1) is true. Okay. So, that's how you hide it or show it based on the profile? That's exactly right. So, just to reinforce that, if I look at Office; same evaluation, except we're looking at the Office OMIT group instead of Salary, right? So, that way when the Office field is eliminated, you won't get an error that the visualization can't render, because the field doesn't exist. You're telling it to disable the field knowing that it's already not there. That's a great tip for hiding columns with a show condition. I did, I wasn't even familiar with that concept actually. Yeah. It's actually, it's one of those things that, it's a best kept secret, I guess; which shouldn't be a secret. Exactly. So, and then, let's talk about it at the visualization-level. So, in this case, I wanted to drive the marketing manager Claire into a completely different visualization. Data reduction aside, I just wanted to see something else. So, instead of the scatter chart, we use a show/hide container. If those of you familiar with QlikView, it's very similar to that. Here's a Container Object, and the content would be the charts that are in it. So, it's actually the Scatter Chart and the Distribution Plot that we use to show different visualizations based on the access, right? So, in this case the distribution plot. What I’m checking in to see is: if the COUNT of Territories is = 2, then we know that's our marketing manager, and so she should see the distribution chart instead of the scatter plot. Cool. So, it's a container that has multiple visualizations in it, and you just show it based whichever specific visualization based on the profile? You got it. Right. That's super cool. And then just to tie that off here. Then I clicked on the sheet, and then the last one we have is the show condition for that. So, I’m not using that in this example, but this is something newer that was added so you have that ability. Again, to show different sheets based on whatever conditions you want to create, doesn't have to be security. It could be anything, and the same applies for the other things. I’m just using security as a way to drive show/hide conditions. I love this concept of turning off Section Access and being able to cycle through the different users to help troubleshoot. Can you walk us through how the script is set up? Yes. Yes, perfectly. So, the data load editor is what we have to get into, right here. And then I have one tab set up for Section Access. And so, like I was saying earlier, if I want to show my security tables in a data model, all I have to do is comment (//) that Section Access decoration, and now my authorization table and my OMIT fields table become part of the data model so that I can actually select them and impersonate who I want to impersonate without actually having to login as that user, right? And so, you can see this is it. These are my two security tables. I only technically need one, but like I was saying earlier, I created this OMIT group field to then show that one-to-many relationship between a user and the multiple fields that they might need omitted. So, Contact Info is an example: if you're in the Contact Info OMIT group, then you're not going to see Extension, Phone Number and Social Security. The keywords that I was talking about access that's either ADMIN or USER. If you're an ADMIN, you're going to have visibility into all the data. If you're a USER, then the data reduction is going to be applied based on whatever you're using as a reduction field. So, these are both keywords: ACCESS and USER ID. REDUCTION is not a keyword. It's just whatever name you designate for your security key which is going to link to your Application Section. So, this part right here, this Declaration; it’s where your underlying data model fields will begin to load. Okay. This link between REDUCTION in the Section Access script and REDUCTION in the data model is really where the actual reduction is happening. If I’m in a REDUCTION of 1, then I’m only going to see data related to Office ID 1. If I’m in 4, then I’m going to see Office ID 1 and 2. If I’m ALL, I’m gonna see 1-5. Right. That makes sense. Now, the one thing I want to point out here: ALL is something that I just use to help with the testing. Technically if you are an ADMIN user, you're going to have visibility on all the data. But using this this method of turning off Section Access and being able to test everything; if I didn't have this in here, then then the ADMIN actually wouldn't see everything. So, for testing purposes, you might want to have an IF condition to add these duplicate set of keys for the ADMIN; versus in production, you might want to just use the ADMIN. Does that make sense? I understand, yeah. You're just recreating the ADMIN rights for data association for testing purposes, but otherwise you wouldn't need it. Yeah, that's it. That's exactly right. So, that's everything that's behind it. There's different layers of granularity. How would you manage a more complex reduction? For example: not just Office Location; what if it's Product and Department? Yeah. So, that's a good question. Well in that case you, would just you would set up a Composite Key. And that actually helps segue into one of the slides that I that I wanted to show. Okay. Many of you may have already been working with the load script for a while. So, you might be familiar with star schemas and snowflake schemas and whatnot. I think most often when somebody's building a data model, they just bring in the tables as-is; and they don't create an explicit fact table or a link table, and that's fine. But when you're using security and you have these different levels of granularity; you would have to create a Composite Key for all those values. So, let's just use Department and Product as an example; you would have a field that would have Department (maybe separated by a pipe) and then the Product. And then you would link that into your Link Table. So here you have your multiple fact tables, and so that Link Table is going to operate as a junction box for all your access rights into those facts. So, that way, you're not having to duplicate facts. You're just managing all that through your security table, and so every value that should have visibility into: fact one or fact two, you're managing that here, right? In the same token, you can do that with a Combined Fact Table. This is all precipitated on having a security field that has all the values in your Composite Security Key that you would need to determine what facts you have visibility into and what dimensions you have visibility into. So, that way you're not managing a tangled mess of facts and aggregation. And I’ve seen a lot of different users try to manage this through really complex formulas, and the easiest thing is to create the Link Table when you have complex security. Okay, that makes Sense. Is there a way so you can set up the same Section Access tables between multiple documents? So, you're sharing the same access across multiple apps? Yeah, actually. So, you could use what's called an Include Script. So, you could actually store the script for Section Access in a file, in a separate file a text file or a qvs file, and reference that Include statement. Effectively, it's reading the code in as if it was part of the load script, but it's just reading it from a separate file. And any of you that have any web development experience would be familiar with using include files; but it simply is - it's the script that you would use for your Section Access, but you stored it in a separate file. And then, you're just picking that file up and it will interpret that as part of native script. Just so everyone is aware, we'll include a link with the recording of this session on Qlik community to the same demo app that Mike was using today. So, you can reference that if you'd like. Mike, what are some key takeaways from today? Yep, so perfect. One thing I promised earlier was: I wanted to cover some of the keywords. So, if you have some experience with Section Access with QlikView, you're familiar with NTNAME. In SaaS, you're using USERID. So, that's one of the bigger distinctions. And there's also a GROUP keyword that can be used in SaaS. There's a couple other things that I’m not going to get into in this, but we'll share this and along with some other resources that'll really get into some of the details. If you're, let's say, if you're coming from Qlik view and you're coming into Qlik Sense, and what changes you might need to consider when you're implementing Section Access in Sense. So, we'll include a lot of links and helpful information and tips and tricks that the Troy’ll package up and put out there for you guys to consume. And I think that should cover most of the things I wanted to touch on. Yeah, we'll definitely include links to all the documentation where you can find more information and resources. Then it's time for Q and A. Please submit your questions in the Q&A panel on the left side of your On24 console. Mike - which question would you like to address first? So, the question is: in Qlik view Section Access settings, there's the strict exclusion setting I’ve never understood what the check box does. Can you explain? So, what that does is it implies that: if there's no values that match the fields you're reducing on, then you actually get to see everything (with strict exclusion unchecked). So, in QlikView you would always want to check that to ensure that somebody who didn't have matching values wouldn't then end up having visibility in all your data. Now in in Qlik Sense, there is no strict exclusion setting, because it's strict exclusion by default, right? So, for those of you that are actually migrating from QlikView to Qlik Sense, that's just one thing to be aware of when you're actually creating your Section Access or when you're doing the migration. Okay, next question. All right, let's see, okay. So, I’m using Section Access to limit the Access level the data for each department. Each department chief should only see the data of their department, but for some sheets I want them to see all the data. Is there a way to do that? So, the sheets themselves don't have anything to do with the data. The data reduction is going to eliminate the data that they can't see. So that's something that you're going to have to determine in your data model: where the security should be applied and what value should be reduced. If the question is: I don't want them to see some sheets, then there's show conditions on the sheets. But if it's around the data, and you truly want to secure the data, then it should be reduced out. Otherwise, you just, you would have to manage it differently. Okay. Let's see. So, is it possible to restrict action to certain fields in a table to all users? I’m not quite sure when we say “restrict action” what we're looking for, or if it's just visibility then. And if we're trying to restrict it from all users, then I just wouldn't load it in the data model. that would be the simplest. If there's for some reason it needed to be part of the data model, then I would use, I would apply that field to the omit keyword, the preserver that is in your entitlements table Okay. All right, let's see. So, is there a way to have global Section Access and Section application table that can be applied to multiple documents without having it in the script of each app? Now that was something we covered when we were talking about “include.” So if you want to reference Include Script with Qlik Sense, you can you can find plenty of help on the community on that or in the help. And let's see what else we have. So, is there a way to build the Section Access authentication based on their SaaS login, so it automatically limits the data based on who they are without having a separate password for each app? Yes, that's exactly what we're doing with Section Access. So, we are we're taking that user login based on the subject id, and that's their user; and then they are being reduced based on the entitlements that you've already set up, so you don't actually have to have a separate password for each app. Okay, next question. When I develop the app using Qlik Sense locally everything seems to work, but after importing the app on the production server, only the Admin has Access. I could probably help, but I don't know. I think I need a little more context to answer that. So, there might be some difference in the way, it sounds like there's difference in the, differences in the way that the user is being identified. So, that while it's working locally it's interpreting that USERID is something different, that would be my guess. In Qlik view, I can use the loop and reduce within publisher instead of Section Access with large QVWs. Is there something similar in Qlik Sense, so that the data in the app is limited instead of limiting Access to the data? There are techniques to do a loop and reduce. There isn't something specifically the same as loop and reduce in the Qlik Sense scheduler, but there are there are ways to manage that. You can do it through the API is one option. I know that there's a couple other examples out there if you look up “loop reduce for Qlik Sense” you'll see some alternate ways to accomplish that, but I also say the, I mean if it's a concern around the data size, we also have different techniques like “On-Demand App Generation” and also “Dynamic Views.” And what on-demand app generation is just level, is you have an aggregate app, you make some selections in it. It passes those selections to another app which reloads on the fly. And dynamic views is similar, except that you're working with one application, and then certain tables or certain objects are then are refreshing based on the selections in real time, so those are a couple different ways where you wouldn't have to necessarily apply loop and reduce. Kind of a different way of thinking. These are great tips. Yeah, thanks. So, let's see what else we have. So, Section Access reducing data on fields of numbers. Does that work or should the format be text in all reduction fields? It doesn't really matter what the format is, if you have two values that are the same on both ends, then it's automatically going to reduce based on that. So, if it's numbers in one value and text and another value, and a combination of numbers in text, they'll all work if you have a match. Okay, next question. Okay, so what are some best practices to reduce access based on multiple field values in the same record line? Composite Keys ultimately. So, you're gonna when we're talking about the data model, and using a Link Table as your junction box, or a Combined Fact Table, that's how you're gonna have to accomplish that. You're gonna create a Composite Key that ultimately strings together other every value that you want to be a part of that key, and that that is your security right there. Okay. Can you apply Section Access using dates? Yep, just another field. It could be a date by itself or it could be a combination of dates, but yeah; I guess that's the, I’d be curious to know a little bit more about how you would apply just a date as security, and what the use case is, but that's, but at the end of the day, it's just another field. What are the limitations on granting Section Access for a specific sheets in Qlik? So, that was the Show Condition that I had previously demonstrated in the demo. So, you'll have the ability to show hide the three places. Just to recap, you have it in the object, you have it at the field-level, and you also have it at the sheet-level. And let's see, what else? There's also: is this available for Enterprise on-prem? Yes, all our products, you can apply Section Access, except for local copies of Qlik Sense which aren't part of part of the Enterprise deployment. Okay, the next one: how can I efficiently restrict access over multiple values, and where records are not present one table but are in another? This is something I’ve encountered before. You're trying to create security on fields that exist in different tables, and so that goes back to that example I gave previously, where you want to use a Link Table. The Link Table solves a lot of problems, because again it is like a junction box. So, you can shove everything you need to in that security field to help drive what's in your fact table. In this case, it sounds like you have ultimately what would be a Combined Fact Table. You have facts coming from multiple tables. Okay Mike, we have time for one last question. Okay and let's see. So, is there a standard way to allow a user to see his own detailed data and only summary data for users? Okay or for other users, I think that's a good question. Because this comes up a lot what you want to do in that case is actually, typically we say just bring everything at the detail level. Because we would then aggregate but if you're applying security, and you need somebody to see the aggregate information then it's not going to work. But what you could do is: actually add the summary data in as separate records, separate transactions, and you could put that in its own table or you could add it as part of your fact table if you're using a fact table Great! Thank you so much Mike! Again, thank you for everybody for your time. I appreciate this you know. I love being able to share the things that I’ve learned along the way. Section Access, I know when I first started working with it, just seemed like a black box and there wasn't a lot of information around some of the best practices, and all the things that you could do with it back in 2007 when I started working with it. So, happy to share the things I’ve picked up along the way and hopefully you've gained something from this, and you'll be able to be successful in implementing security in your organization. And so with that, again thank you, and you'll see me on the community. Okay, great! Thank you everyone. We hope you enjoyed this session. Thank you to Mike for presenting. We appreciate getting experts like Mike to share with us. Here's our legal disclaimer. And thank you once again. Have a great rest of your day you.
Q: When getting permission for our Sharepoint tenant site - you mention you allready got it approved. Was this done inside of Sharepoint itself or your Office 365 Azure AD portal ?, it seems other connector seem to suggest needing to do this to approve and allow third party apps to send requests. Is this correct?
A: There is nothing to be done from the Azure portal. If it's the first time using the connector, it's mandatory to use the Azure tenant admin when authenticating. This will prompt and authorization approval for Qlik's sharepoint app (the reason why appears as a third party app it's because is not developed by microsoft).
Q: What is the purpose of the "Access Token"?
A: The access token will grant access to connector
Q: Is there a connector to Azure?
A: Yes, both in SaaS and on-premise (december 2020)
Q: When you say 'Azure Tenant,' is that the same as Auzre AD admin portal to approve the app?
Q: We are using a proxy to connect to outside enterprise connections (Sharepoint Online).But it's possible to activate proxy only for some connections types ?
Hello everyone and welcome to the March edition of Support Techspert Thursdays. I’m Troy Raney and I’ll be your host for today's session. Today's presentation is Best Practices for Using Qlik Web Connectors with Francisco Fernandez. Francisco, why don't you tell us a little bit about yourself?
Hello, my name is Francisco and I’m a Technical Support Engineer here at Qlik. I’ve been working at Qlik since 2019 and my main focus are Qlik Web Connectors and Qlik Sense integrations, such as APIs and mashups.
Okay, and what are we going to be going through today?
We are going to go through installation tips; how to use SharePoint Connector; difference between SaaS and on-premise connectors; and a little bit of troubleshooting.
Okay, and for those of us who are kind of new to Qlik Web Connectors, what are Qlik Web Connectors and how do they work?
The Qlik Web Connectors package is a lightweight web service that helps you fetch data from many web-based data sources, and also social media. It contains 42 different connectors with tons of functionalities. Also, it can be used in a separate server, so it's a completely isolated package that doesn't depend on Qlik Sense or QlikView; and that's the main reason it can be used with both.
So Qlik Web Connectors works with both Qlik Sense and QlikView?
Yes. Here in the example architecture. You can see that the Qlik Web Connectors is being run in a different server. That's why it doesn't have a dependency.
Okay, that's cool. Now why should someone start using these connectors? How are they useful?
So, first of all, because it's free of charge with a Qlik Sense license.
That’s always nice.
It's free. It's simple to install and configure. Auto-generate the scripts. It can be used as a Windows service or run as an application. As I previously mentioned, it can be used both with Qlik Sense and QlikView.
Okay, I think the initial question for a lot of people is: how to install it and how to get started?
Yep, you can download the Web Connector from our Support page. As a demo, I already downloaded the package. This is what you get from the download page. I already uncompressed it, so we are ready to go.
Okay, and what version of Web Connectors is this? Is this the latest version?
Yeah, this is the latest version. This is December 2020.
Francisco, what are we looking at here?
We are looking at the connector. It's basically a library of DLLs, as you can see here. Each DLL is a connector. And we also have some folders that contains the connector configuration. Scroll down. We can see the deploy.config file, which contains all the connector configuration. Scroll further, we can see here: application executable and the application service executable.
Yeah, you mentioned there's two different ways of running connectors: one as a service or as an executable application.
Yeah, if we want to use it as an application; we only need to double Qlik this window will pop up and we are ready to go. The connector is already working.
If we want to run it as a service, we have to go here to batch files. And we have two batch files here. The first one contains the installer service, and the second one contains the uninstall service.
Can I ask you, what are the advantages of running it as a service versus just clicking on the application?
Running the Web Connectors as a service is especially useful for server admins. It allows them to automate tasks, such as restart all the services at once using a script.
Okay, can you show us the details of running the service installer?
Yeah, if we open it using the Notepad++, we can see here that it contains a path. This path should be where our Web Connector is located. In our case, it is in the desktop. So, we should change the path here. Save it and install after that. And it will be available in the services tab. If we want to start using the connector, we have to check the configuration in the deploy.config file. Notepad++ and edit it. Here we have an XML configuration file which contains lots of tags, as you can see.
Yes, these are kind of difficult to read with the human eye sometimes if you're not used to editing XML files. What settings here are important that we should take a look at?
First of all, the port where we want to run the connector. This is by default: 5555. You can change it to whatever port you want to use. Secondly, one of the most important tags is: the Allow Remote Access, which basically allows access to the Web Connector from remote machines.
Okay, so if like I have Qlik Sense running on a totally different machine, this is the major setting that allows that connection?
Yeah. Otherwise, you will see an error message saying that you have to enable it.
We have to change it to True. After that, we should look at the File System Allow Path. This tag is quite important, especially for those connectors who download files, because the connector needs access to the file system. You can either allow full access to any path by using the wildcard (*) or you can specify which paths you want to use for the downloads. You have to use the semi-colon (;) to enumerate more than one path.
I see that's a little confusing having the examples right above the setting which is initially by default: blank.
Yeah, usually people tend to mistake this with actual configuration parameters, but the parameter starts after the arrow (->). This is basically not being used by the connector.
Lastly, we have Default Allow IP Addresses. This is also very important if you are using the Web Connector in a multi-node environment because it will allow all the nodes access to the Web Connector. You have two options: use any to allow all the nodes including the new ones if you want to add more nodes. Or you have the other option is to specify the IP Addresses of those nodes. Usually, I tend to prefer to use any. We have some other advanced features.
Yeah, so I see a lot of settings there about security and SSL. And if someone wanted to take advantage of all those security settings, would they go to the Help page to find all that information?
Yeah, they can find it in the Help page in the section called Advanced Qlik Web Connectors Configuration.
How do we access the console here? Could you show us that again?
Now, we are going to run the Web Connector as an application for demonstration purposes. The land page is this one. In case that you are running it as a service, it's the same page. This is the main page. Here you can find all the connectors available.
That's a long list. Would you mind walking us through the steps to set up a connection?
Yeah, we can start by using SharePoint which is here. First of all, what we have to do is authenticate to our SharePoint site using the Can Authenticate table and pressing Parameters.
We can see that we have here the base URL. In this case, we are using Qlik’s SharePoint site. In your case, you will have to use your company SharePoint site. We also have Prompt and Consent level. If you were already using the SharePoint connector, you will have to use None. But if this is the first time that you're using the connector, you should use Consent and using an Azure tenant admin, consent the application. Since we are already allowed access to the application, we are going to use None and continue with the authentication process, which generates this authentication code. Copy to clipboard. Go back to Authentication Code here and paste it and then save. We can run to see if we are authenticated. The Web Connector is allowed to retrieve data.
Okay so we've authenticated ourselves from the connector to our company SharePoint site?
Okay, what's next?
Now we can start by trying to retrieve a list. I already set up a list for the demonstration. It's in here.
Okay, so this is our SharePoint site, and you've got a list there I see?
Yeah. Here in the URL, we can see that we have all this. So, the first part is the site Sub Path.
So here is our site and site Sub Path. So, it usually is always Sites followed by the name of the group that we want to use. After that, we usually find what's the table that we need to use. In our case, since we want to retrieve a list as you can see, we should use the table ‘List.’ But if we want to retrieve document let's say, you we will have to go to the Documents as you can see here. I already put the Site and the Subsite path. And using list. We can run the table, and we can see all the lists that are available.
Okay so, this is a preview of everything it reads from that Subsite?
Yeah, it's reading metadata. So, it's not only reading our list, but everything that gets pulled by the connector.
We already know the name of the list that we want to retrieve. And this is the identifier that we have to use.
Okay, do you need to copy that?
Yeah, we are going to copy this. If you want to retrieve data from a list, it's very important to use one of these tables, because lists are not files; are actually metadata. If you want to use the Get File table, it will fail. So, for example, as you can see: everything is correctly set up and we are going to paste. Bear in mind that this might contain spaces, so you have to delete them, and run the table. And as you can see, we have a Not Found. This might be also a 403 - Not Authorized.
That didn't work because the command was Get File, but you actually pasted the ID of a List, so it doesn't match?
Instead, we are going to use the correct table, which is: List Items from ID. So, the Connector stores what was your last search, so basically that's why we didn't have to paste it again. If we run the table again, we can see that we are able to retrieve the list.
Okay, so it previews the actual list right there.
And if we go to the list, we can see that the information matches.
Okay, how do we get this data in Qlik Sense?
Here, we can see that we have all the instructions to create a connection in Qlik Sense.
Francisco, would you demonstrate for us creating this connection now that you've prepared everything in Web Connectors?
Just following the instructions, we have to copy the URL, and we have to create a web file connection in Qlik Sense. So, we go to the Data Load Editor. Create New Connection. Scroll down until we see Web File. And we have to paste the URL that we copied, and username.
Any name, right?
Any name and create the connection. Also, here in instructions it's mentioned that you have to create a variable with a connection string that you are using. I already done that before, so everything is correctly set up now.
And we only have to copy the script to the clipboard and paste it and make a reload. As you can see, the reload was successful and we are able to retrieve the list.
Francisco, that all seemed to go very smoothly. Just following the instructions, copying, and pasting from Qlik Connectors to Qlik Sense; what should you do when things go wrong?
I prepared an example actually.
We're going to do the same; but in this case, I purposely made an empty list with no data in it. As you can see here, we are going to make the same procedure as we did before.
We are going to look for Empty List. We are going to copy the ID and follow the same procedure as we did before. Delete this, paste. As we mentioned before, we have some spaces. We have to delete them and run the table. The connector is basically saying that there is no data to pull. So usually when this happens, we have to verify in SharePoint that there is actually data in what we are trying to retrieve. There is no data in here, so the connector cannot achieve anything.
How would it be different, Francisco if you're trying to load a file from SharePoint instead of a list?
So, we have here this folder that contains this file. We have to go to List Files. And since we already know the folder: Save Inputs and run the table. We can see here that we have our Excel file and we just need to copy the unique ID. Go back. We have two options: either the Get File or Download File.
We're going to use Get File. Paste the unique ID. Run the table. As you can see, we are able to generate a string.
In case that we want to use it in Qlik Sense, we just need a new connection using this URL.
Would that be a web file connection?
Yeah, it's a web file connection in this case.
What happens if admins encounter an error while trying to set up that connection?
One of the common problems that we are seeing from Support is that sometimes Qlik Sense is failing to reload the data with a 500 error.
In order to troubleshoot that problem, we usually go to the localhost. In this case, we already are here. Try to open the URL. So, as you can see, we are able to download the data. If we open this, we will see the Excel file and all the data that contains. So just copying this URL and going to the node that is failing, pasting in the browser should be the same result as the in the localhost.
That data file that you downloaded, is that the actual XLS file?
Yeah, it doesn't have an extension because it's ready to be used with Qlik Sense.
Okay, that's a great step to test the access to the data. What troubleshooting steps would you take if it doesn't have access?
What we need to do is: go to the home page. Go to Settings. Double check if the allowed IP addresses matches with the deploy.config file. In here, we have ‘Any,’ and also, we have ‘Any’ here. It's not likely that we are going to face this problem, but in case that you have ‘Any’ here and ‘localhost’ here, you will have to change it and then press Save. If you change any parameter here and press Save, there is no need to restart services.
Okay, but it's very important that the settings in the main page here need to match what's in the deploy.config file?
Yeah. Also, if you're using more than one administrator, all the administrators have to have the same allowed IP addresses.
Okay, so that's a very important setting there.
And where would you find the log files if you needed to investigate further?
Go into the home page. Here in the Log Files tab, you can see all the logs. If we open one, for example this one. We can see that it's an XML file. So, for example, when we tried to retrieve the Empty List, we receive ‘No Rows Return’ error message. We can look for the exact same error here in the logs. We can see the error message.
What’s the difference between Qlik Web Connectors for Qlik Sense on Windows versus Web Connectors for Qlik Sense Enterprise SaaS?
The Qlik Web Connectors is standalone. It’s a package that you have to install in the server. As a kindly reminder: always use the latest version. It contains 42 connectors. On the other hand, the Qlik Web Connectors for SaaS it's integrated in Qlik Sense. It's always using the latest version. It contains 33 connectors. It's still growing. Soon it will be available the SharePoint File Connector, so I’m sure that many customers will be happy knowing this.
Yeah definitely. So just to clarify, that whole installation process, configuration that we saw; that's not really necessary in SaaS because it's already built in?
Yeah. Qlik is taking care of that.
To review, what are some basic troubleshooting steps when things go wrong?
First of all, always double check in if you are using the latest version. Then you can check the logs for any error messages.
And lastly, you can check the API limitations of the connector. You can check that in the About button under the Useful Links page.
Okay, because basically the Web Connectors are using the APIs of each individual web source and like Google or Facebook their APIs; and they may change?
Yeah, that's correct.
Where are some resources where admins can find more information?
They can find more information in the help page, and also in the community articles that Support creates.
And again, where can we find the Web Connector download that we started with?
You can go to Qlik Support download site. Look for Qlik Connectors and under the Qlik Connector site, just go to the latest version available.
Okay, now it's time for Q and A. Please submit your questions through the Q&A panel on the left side of your on24 console. Francisco, which question would you like to address first?
So, first we have: do we need to upgrade to the latest version of Qlik Sense to use them?
No, but we strongly suggest to upgrade to the latest version of Qlik Web Connectors.
Moving on to the second one, are all the connectors free along with the Qlik Sense license? Some of the connectors on our site show as premium.
Ah yeah, if you are using a previous version, let say from March 2020 or maybe May 2019 it will appear that they are still premium, but if you upgrade to December 2020 you will see that all appear as free. Also, it’s not only Qlik Sense. If you have a QlikView license and you use the December 2020 version, you will see that the connectors are also free.
Uh, we’ve had trouble connecting to the SharePoint site. What administrative privileges need to be granted to what user?
Probably this question might be related to the when you first start using the SharePoint connector, you have to use an Azure tenant admin to authenticate to the connector; so an application will be created in the tenant site. This might solve the problem that you’re facing.
And we have another question. Does the account running Qlik Sense need to be an administrator on the SharePoint site?
Uh, no, there is no need.
Uh, are all the 42 connectors free to use including SharePoint 365 connector?
Well, I think I already answered this one, yeah. All of them are free to use.
I see some questions that we already answered regarding the license. If you are still seeing the connector is not free to use, you basically need to upgrade to the latest version of the connector.
Does the version of Sense or QlikView matter?
Uh, no. They are not related. As we mentioned during the presentation, the connector is a separate package. It is isolated.
For example, this one is a good one. In a multi-node setup, do I need to install of each node or only on the central node?
You can install the Web Connector only on one node. There is no need to install the Web Connector on each node. This is not like the ODBC package. And you can even install the Web Connector on a different server, as long as it has connectivity to the – to the Qlik Sense Server.
What is the difference between QS Standard and Legacy mode when obtaining the load script?
This is a change regarding to Qlik Sense actually. It’s not related to the Web Connector. The Legacy Mode has some differences when loading the connection strings. You can go to the QMC, after you go to Engines and you can enable or disable the Legacy mode.
We are using a Proxy to connect to object enterprise connections, uh like SharePoint, but it’s possible to activate the Proxy only for some connection types. Example: ActivateProxy for SharePoint online.
Unfortunately, this is a limitation that the connector has. You can only setup one proxy and you cannot select which connectors are using that proxy.
Uh, can Web Connectors be used with Qlik Sense Desktop?
Yeah, they can used in Qlik Sense Desktop. And you can also try them in your Desktop if you want to.
Uh, how can we fetch multiple files from a folder in a SharePoint site?
So unfortunately, this is a current limitation that the connector has. I strongly suggest creating a feature request in the community to raise awareness within the product management.
I saw there’s a couple questions asking about the difference between the REST connector and Web Connectors. You want to address that?
Yeah. Basically, the REST Connector has the same capabilities as the SharePoint connector for example when retrieving data. But the thing is that using the SharePoint Connector is much easier than using the REST connector because all the authentication part is handled by the – by the SharePoint Connector, so it is much easier to use.
A question just came in. Are Web Connectors free for QlikView users as well?
Yes, as long as you are using the latest version.
Is there a connector to Azure?
Yes, we have once connector to Azure Block Storage.
Okay Francisco, we’ve got time for one last question.
Those web connectors have limitation on large volume of data?
Not specifically, since each connector uses a different API. There might exist API limitations, but not the connector itself.
Well thank you very much Francisco.
You're welcome and thanks for everyone for listening. I hope this has helped many people and good luck with your web connections. Feel free to post any questions you might have in the Qlik community. I’m there quite often, and I have written several of the articles on Web Connectors.
Okay great! Thank you everyone! We hope you enjoyed this session. Thank you to Francisco for presenting. We appreciate getting experts like Francisco to share with us. Here is our legal disclaimer. Thank you once again and have a great rest of your day.