Skip to main content
Announcements
Qlik Introduces a New Era of Visualization! READ ALL ABOUT IT

STT - Upgrading Qlik Replicate Best Practices

cancel
Showing results for 
Search instead for 
Did you mean: 
Troy_Raney
Digital Support
Digital Support

STT - Upgrading Qlik Replicate Best Practices

Last Update:

Feb 23, 2024 2:18:39 AM

Updated By:

Troy_Raney

Created date:

Feb 23, 2024 2:18:39 AM

Troy_Raney_0-1708672627733.gif

 

Environment

  • Qlik Replicate

 

Transcript

Hello everyone and welcome to the February edition of Techspert Talks. I’m Troy Raney and I'll be your host for today's session. Today's Techspert Talks session is Upgrading Qlik Replicate Best Practices with her own Alan Wang. Alan why don't you tell us a little bit about yourself?
Hello. I’m Alan, and I've been with Qlik for 3 years and I work in the Qlik Data Integration department; so we work with tools such as Qlik Replicate, Qlik Enterprise Manager and Qlik Compose.
Great! Today, we're going to be trying to understand the requirements when it comes time to upgrading Qlik Replicate; Alan's going to take us through a demonstration of that whole process; and we're going to be discussing and looking at troubleshooting tips and resources. Now Alan, if someone wants to upgrade their Qlik Replicate implementation, how do we get started when it's time to upgrade?
So, the very first thing we want to go through is your platform environment. For example, you want to make sure that you have all the Windows software requirements, such .NET; Visual C++ and TLS. If you have any patches to your Windows or Linux environment, you want to make sure that none of these software requirements have changed during that time .
Okay. And when you're running the installer wizard, does it check for these requirements automatically?
So, you would not specifically see any errors that will pop up during the installation process. The installation process will go fine; but as you start up the Replicate Services, the services will not be able to start up if these prerequisits are not met.
Okay. Once you've checked those pre-reqs, what other requirements are there?
You'll also want to check the platform compatibility list; and you want to make sure that any upgrades still stay within our support.
Okay. So, all of these platforms Windows and Linux are supported operating systems for Qlik Replicate?
Correct, and you'll notice a little note here telling you if there are any additional versions that may not be supported by older servers. In this case, Windows 2022 is only supported on Replicate May 2022 patch number 2 only.
That's an interesting detail. Okay.
You do want to always always check the exact Replicate version that you're using as these requirements will change as you move from version to version. One thing specifically in the May 2023 version, you will notice that Linux 7.x is now depriciated as the dependencies have changed on the platform.
Okay. That's good to be aware for the Linux customers. What about the databases or the endpoints that are being used?
Absolutely. You also want to make sure that the databases and the drivers are compatible you can move to supported endpoints where you can see a very big list of supported endpoints .
Okay. This case, we'll move to SQL Server which falls under the Relational Databases.
All right. So, what are we looking for when we scroll through this list?
Each database type is listed as the endpoint, and the supported versions of those database versions are also listed here. And you'll notice a little check mark for these two columns. What it will tell you if it's supported for Windows and Linux.
Okay. So, that's what we're looking for: to find the endpoints we're connecting to and make sure that it is supported? Okay.
Absolutely.
What about, are there any drivers that need to be upgraded as well?
Drivers will also need to be upgraded if the versions have changed. In this case, we want to go to the Managing and Adding Endpoint section for Source or Target. SQL server in our case. So, under SQL Server, you will find the prerequisites for all the endpoints. Some of these endpoints will just need a driver upgrade; some of them may have extra steps. You do want to look through each endpoint specifically to understand what prerequisites are listed for that endpoint. The driver upgrade is more simple on the Windows end, and a little bit more complex for the Linux end. But steps will be listed for you to follow.
Okay. So, how do you choose the version of Qlik Replicate to up upgrade to?
So, the version is recommended to be the latest 2 versions. You may want to check all the product suites that you use from the Qlik end, if they're still compatible with the products that you are using.
All right, because a lot of customers don't just run Qlik Replicate; they run it in conjunction with Qlik Compose or even Enterprise Manager. So, you got to make sure those are in line and compatible, right? So, how could they check that?
So, from the Qlik Enterprise Manager user guide, under installation and setup, there is a compatibility list with the other 3 products.
All right. So, if planning on upgrading Qlik Replicate, you might also need to upgrade these other products as well?
That is correct.
All right. Make sure you select the version you're upgrading to on the left?
Yes. That will tell you exactly what versioning for the other product suites are compatible.
Perfect. So, what is the actual upgrade process to follow once all the pre-reqs are addressed?
Here you will find 2 methods that you can use for the upgrade process. One will be the one that is recommended, the one where you back up your Data Directory; and we use that same Data Directory. Method 2 is the export of the server settings; but do keep in mind: even though this will save a lot of space during the upgrade, you will lose all your logs save points and any information that comes in the file format. Method 1 is preferred as it will keep the server exactly the same as before.
All right. So, the first method will keep everything exactly the same; and Method 2 will remove all log files and run history and kind of starting fresh and saving disk space?
Correct. So, if you are low on space, instead of using method 2; I recommend you use the automated log cleanup feature. You want to go to the server settings on the top left where it's Tasks, you will want to move to Server, and then from Server all the way down the Logging.
Okay.
And Log File Management. And from here you can decide how many days of logs you would like to keep. So, once you check the Automatic Cleanup and specify the amount of days, it will do a search for any logs that are older than X amount of days and then it will purge them automatically for you.
That's cool. So, how do we actually get the installer files for the upgrade?
So, the installation files can be found from The Qlik Community website under Support > Product Downloads. That will take you to a page where you can filter out exactly what products you need. In our case, we are part of Data Integration for Qlik Replicate. The 2nd filter is a Release filter.
What selection should we make to find the version that we're looking for?
In our case, I would recommend the 2nd option where it's just the latest patch with all the releases. Qlik Replicate. Let's choose 2022-11.
Okay. We still have a long list, how do you identify which file you want to download?
You'll see that they have different extensions. In our case here, the x64 zip file is the Windows version of the installation file and the tar file is the Linux installation.
Okay. Great. So, that is our zip file, all right.
The user guide will tell you to stop the product Services, although it can be automatically done from the installer itself.
Yeah let's see those Qlik Replicate services.
From there, you can turn it off.
Oh, it's actually called the Attunity Replicate server interesting; and what version are you upgrading from?
I am upgrading from May 2022 to November 2022 .
Okay. All right. So, we stopped our Replicate Services, now we run the installer?
That is correct. Make sure you Run it as Admin and your upgrade will automatically start.
So, it recognizes you already have Replicate installed, calling it an upgrade?
That is correct.
Okay. Next. This is going to keep all your current settings and tasks; it recognize those in the data folder and just upgrade the services to the new version?
That is correct. So, it will use the existing data folder that you left behind previously; although we still want to back-up that data folder as a way to roll back to a previous version if needed.
All right it's finished. Do we need to start the services again?
So, services will also automatically restart whenever the upgrade is finished.
Oh, it's changed name?
Yes. So, this installation changed the service name as it's been a newer version and as you can see both services are now running. We can check by accessing the URL just to make sure.
All right. So, here we're back at your Replicate console; everything looks happy, and I see up in the upper right that we're running November 2022. So, that was a successful upgrade?
Correct, your task will automatically stop and start from the upgrade as well. So, you should not have to manually change anything.
Awesome. Okay. Well, everything has run very smoothly with this upgrade. If someone did encounter an issue, what might that look like and how could we troubleshoot?
So, one isue that we do see customers running into: when you try to access the Replicate URL, you'll get a 404 HTTP error.
Okay. So, if someone runs into this what would be a best practice troubleshooting process?
As you can see, the UI Service would not start, even if we attempt to start this. So, to troubleshoot this, we have logs specifically for each of these processes. Under Data > Logs. In our case, since the Replicate UI Service is the one that it's not starting, we will look for the Replicate UI log; it's named RepUiCtl.
Okay.
And the error that we do see here is: global repository is out of date. The repository folder was not able to upgrade correctly. If you scroll down the line here, you will find the exact command to upgrade that repository correctly.
Okay. So, we need to go into a command prompt from that folder and run this command? Is that what it's telling us to do?
Yes. We would have to run this from the Replicate Bin directory, but we will need to first rebuild this command. So, it correctly points to the location that we want to upgrade we'll replace the repository path with the one that we need. The repository path is the Global Repository file which is also found in the Replicate Data Directory. You'll find a file named Globalrepo.sqlite. So, this is the repository file that it is referring to. So, we will want the full path; the file name as well.
Right .
Okay.
The one thing you need to take note of is if your file path includes any spaces, you will need to enquote the path in double quotes.
That's a great tip.
As spaces will break command line from reading the path properly.
Okay.
A second note is that if your data folder is installed in a separate directory from the base installation, you will also need to specify the path for your data folder. For example, if installed it in the D drive you'll need to add the command “-d” to specify the Data Directory, and then the path of new location of the data folder.
All right.
In our case, do not need this extra part of the command. Once you have this command, you can start up a command promp session as an admin; navigate to the Replicate Bin directory, and then run the same command that we have rebuilt previously.
And this command will upgrade the repository folder?
Correct.
Okay. So, after running this command the repository will be upgraded resolving that issue?
Correct.
Okay.
If it does fail, you will want to contact Support to change the global repository file.
All right, good to see what issues are actually possible. Is there anything else you've seen customers run into that we might also be able to demonstrate how you troubleshoot those?
Yes, another issue is where files are not extracted properly. Symptoms for this issue is also very similar where you would not be able to start up the Replicate UI Service. All right. So, it' be a similar issue people would get the 404 error?
Correct.
What can people do about that?
So, this one is a pretty simple fix. Aall you have to do is go back to the Webroot folder, and extract it manually.
Okay. So, that's the folder that didn't expand properly when we unzipped all the other files?
That is correct. Once the web root folder has been copied in here, we can restart the services; and that should resolve the error.
Great! It's nice to see that console up again. All right, now it's time for Q&A. Please submit your questions to the Q&A panel on the left side of your On24 console. There’s already a few coming in, so let's just take him from the top. All right Alan, first question: we're replicating to Fabric High latency AI. What are the issues after replication to Fabric?
So, replicating to Fabric High Latency AI is part of the Qlik Cloud product. I am not aware of any other issues at this time. If you do encounter any issues, we recommend that you open a support case, so we can dive into the issue further.
Yeah, that's a great idea. And you can find support on Qlik Community and just, there's a sub-section for contacting Support. So, if you do have any issues, that's a good place to go. All right, next question: what to do when upgrading the server operating system? So, if someone's upgrading their Windows in the background, what would be the process involved for that or what should they be aware of?
So, you should always turn off the Replicate services and make sure that all tasks have stopped, the tasks will stop when you turn off the services. When you're upgrading the server operating system, you also want to check the user guide to make sure that your patch upgrade doesn't change any of the existing software requirements.
Okay.
We want want to make sure that the .NET Framework still stays within that 4.8. There are times when there are certain patches that will change the versioning of specific software versions and that may interfere with the requirements needed. One last thing to check is also the platform requirements as well. You want to make sure that the patch or server upgrade that you're moving to is still included in the list of supported platforms for the version of Replicate that you're using.
Great. Thank you for highlighting that again. And again, I'll make sure to include those links along with the recording. All right, moving on to the next question: are there any files or folders that you don't really need to keep or would be better in an archive? I think we talked about that a little bit when you're mentioning the two different versions for upgrading.
When you are upgrading, we don't recommend that you manually delete files and folders from the data folder. What we do recommend is using the Replicate Automatic Cleaning settings to clear up any existing files that you do not want to keep. So, under the Server tab, Logging and Log File Management; you will see Automatic Cleanup Settings which you can delete logs older than X days.
Right, okay. So, it's good to go through this process of log file management before you do that upgrade if you're trying to limit the number of folders that you're keeping?
Right. They want to limit the amount of days worth of logs that they want to keep. By default, it's not even on. So, they may have logs from a year or two ago. If they just keep this down to like 2 weeks, that will clear up a lot of space compared to ones that don't use a l cleanup at all.
Yeah that's a great tip. Okay. Great! Next question: is it necessary to keep Log Stream and Data folder in two different drives? On one of the Qlik calls, we heard that keeping Log Stream and Data folder in different drives will give a performance boost. Are you familiar with that?
So, it is preferred that you keep Log Stream and Data folder in separate drives. There's always resource contention when you put one folder inside another folder as a subfolder. So, there's resource contention between the Replicate server accessing all the files within that Data folder, and then the task itself needing to go through that process to access the Log Stream. So, for any instances of using Log Stream, we do recommend keeping the Log Stream folder and the Data folder in separate drives to allow it a more efficient performance .
Okay. That's something I haven't heard of before but it's good to know. So, if people are using Log Stream, it is actually better to have them on separate drives; Log Atream and the Data folder?
That is correct.
Okay, interesting. All right, next question: how to enable High availability on Linux OS?
So, for a Linux operating system, there are a few instances of Linux clusters that are recommended. Go to setting up Replicate in a cluster environment, you'll find recommendations of what sort of clusters that are recommended. The process was setting up a Linux cluster can be found under Setting Up Qlik Replicate on Linux. You will need to set up Qlik Replicate on all the nodes of the cluster, and then use a shared SAN drive for your Data Directory .
Okay. It's nice to see where those instructions are documented. I'll be sure to include the link to this as well. Great, thank you. Okay, moving on to the next question: what if you want to move the location of the installed files during an upgrade? We didn't really get an option for that since it just used where the folders were already located.
Yes. So, it is not possible to move the location of the Data Directory during the installation of an upgrade, but you can change the Data Directory after the upgrade is completed. For more information on changing the Data Directory location, please head to the setting up Qlik Replicate page and move down to the Changing Directory Location part of the user guide.
So, it gives directions here, step-by-step instructions on how to go through that process. Okay. Actually editing the registry a bit, wow. Okay. Well it’s nice to know that is possible and the steps are documented pretty clearly here. Great, all right. So, that would be something you do after the upgrade instead of during?
Yes, that will be after the upgrade.
Next question: how do we handle tables with no primary keys.
So, tables without primary keys will have an impact on the task overall. It will slow down the searches for any Update and Delete operations. The best way to handle tables without Primary Keys is to isolate them in their own task; that way the slowdown will only impact the small group of tables, rather than all the tables in the task itself.
Okay. That's a great tip. All right, next question: if we're only upgrading our Windows Server first, is there a way to keep the current implementation of Replicate on the machine? So, I guess that was similar to the previous question when you're upgrading the server operating system.
Right. The Replicate installation and the Replicate configuration and settings itself will not change from the server upgrade. So, it is possible just to stop all the Replicate Services, keep the server down, and upgrade the Windows Server itself. Once the Windows server is upgraded, you can restart Replicate and everything is still the same.
Great! The process is just: leave it alone, and turn it off, and turn it back on.
Pretty much.
That's nice; that's good to know. Okay, next question: will the same license Keys work on the latest versions or is the version that any way tied to the license Keys? License keys are not tied to versions in in any way. So, you can use the same license key as long as it's not expired on any version of referate that's good to know .
Okay. Great, all right. We have time for a few more questions: what about SAP? Are there special applications that need upgrading for those SAP endpoints?
SAP endpoints will require transports and the SAP extract files. You do want to go back to the SAP endpoints to verify the exact transfer version that you need. If it has changed, then you will need to upgrade the transports for SAP. If it has not changed, then you do not need to upgrade. For the exact versions, please refer to the user guide under the SAP client. You will see the transport versions refered to as the installation codes.
Right, were some of those files that what we saw on the download page?
Yes, for the transport installation files, please refer back to the Qlik Community page where the Product Downloads can be found for Qlik Replicate and all the SAP related installation files.
All right, next question: which version of Replicate is more stable May 2023 or November 2023?
In terms of stability, both May 2023 or November 2023 is still the same. The only differences between these two versions is the fact that November 2023 may have new features.
Okay.
Customers may prefer to use a slightly older version as the new features may still have undiscovered bugs and other instances where it may require some more fixing.
That's fair. We have time for one last question: how can I find the best upgrade path if we're upgrading from a very old version like 5.0.1. That is old.
In terms of best practices, we always want to make a jump from just a maximum of 2 major release jumps.
Okay.
For example, a May to November is one jump. So, one year apart will be two jumps in the case of a version of 5.0.1 we recommend you open a support case. So, we can find out all the version jumps that are needed, and provide a more accurate release path for you to upgrade from.
Yeah, I think that might be a decision where they have to make, decide if it's worth upgrading or just doing a fresh install.
Yeah, fresh installs can work, but they would have to do a lot of testing, because that can cause you know something from that from that point onwards. The amount of the unknown.
That's true. Well Alan, thank you so much for this demonstration and going through all these details with us. I think it'll help a lot of people in the future, especially when it comes time to look at upgrading their Qlik Replicate instance.
Thank you for joining our Qlik Replicate upgrade session today. It was a pleasure working with everyone. If you need more information, you can always refer to Help. Qlik.com or our Qlik Community forums where everyone can interact.
Great! Thank you everyone. We hope you enjoyed this session; and thank you to Alan for presenting. We always appreciate getting experts like Alan to share with us. Here's our legal disclaimer. And thank you once again. Have a great rest of your day.

 

Contributors
Version history
Last update:
‎2024-02-23 02:18 AM
Updated by: