Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
vicio
Contributor III
Contributor III

Qlik Sense on AWS Spot Request Instances

I'm currently using Qlik Sense 3.2 SR4 on Windows R2016 Instances: On-Demand Instances and Reserved Instance.

Has anybody experienced Qlik Sense on Spot Request Instances?

I have a Test site License and it could be great to use for development on a cheap and powerful instance.

Is possible to preserve the data on an EBS volume and restart (the day after or after a forced shutdown)

without losing any progress on Data?

I intend to try anyway in the next days, but perhaps someone has already walked this path and share some hints.

Labels (1)
  • SaaS

4 Replies
vicio
Contributor III
Contributor III
Author

Hi neo.lee‌,

I added your reply here because I created this more specific thread before. Hope you don't mind.

wrote:

To utilize spot instance, you can call EC2 spot instance termination notice API and insert a pending terminate state to do an application snapshot and store it into S3, you then can repeat step 1 by bootstrapping all required dependencies to another spot instances.All of these be automated.


vicio
Contributor III
Contributor III
Author

I have some solution from AWS Support:

My question:

I'm using Reserved Instances for my productions servers: no problems.

I'm using On-Demand Instances for my staging/test/development servers: no problem.

I'm considering to use Spot Request Instances.

In my case, Operating System, software and data are on EBS volume, that I would like to preserve over the start/stop process.

I'm using Window R2016 Base with an m4.4xlarge EC2.

I would like to start an m4.16xlarge Spot Request Instance in the morning and use it

for 4-5 hours or more. Stop at Lunch Break and restart for other 4-5 hours.

I would like to be sure that the data are saved on EBS, and I can restart with my existing EBS (OnDemand, Spot Request)?



AWS Support reply:

To run the instances as per your requirement, you can make use of ‘Reserve for Duration’ option. Also, make sure that you uncheck the ‘Delete’ option on the root volume while requesting a Spot instance. This will make sure that even though the instance is terminated, the volume will be stored and your data will not be lost.

Spot instance do not shut down the instances but completely terminate them. However, since we have the volume of the instance, we can attach it as a root volume to a new instance (OnDemand).

If you would like to continue with the Spot Instance, with the same volume of the terminated instance attached to the new instance, you can:

1. Create a new spot request instance. Make sure to uncheck "Delete on Termination" for the root device, so that the volume stays behind in the next step. Make sure to note the architecture and the kernel ID that your instance is using (very important!)

2. Now, RDP into your new instance and make a file in the filesystem. After making some changes to the filesystem, go ahead and log out of the RDP connection and terminate the instance.

3. Login to your EC2 web console and find the new volume that was being used for the instance we just terminated. Right-click the volume and select "Create Image". Follow the wizard, making certain to select the same architecture and kernel ID that we noted earlier.

4. Now, start the spot request wizard using your new image. Follow the wizard, again making certain to uncheck "Delete on Termination". Additionally, and this is the easy step to miss, make sure to expand the collapsed section titled 'Advanced Options' and set the correct kernel ID again.

Following the above steps, you will have a new instance at the same point that your old instance was at when it was terminated.

Conclusion:

I tested manually and it works smoothly.

At first sight, I thought was a little bit light on security, but I balanced with a high-frequency backup of the apps that is useful for my developers to revert to previous versions of the apps.


My yearly OnDemand quote for m4.16xlarge (188ECU, 256RAM) for 10hours per working day: 12,000$. With Spot Request on zone 1a of my region -> 6,500$, zone 1b -> 3,000$ (75% saving).

Awesome AWS. This is intended for Test Site Qlik Sense server.

Next step:

Study an automatic procedure that my IT guys can setup with just AWS console (no API) and execute the bid and the restarts with scheduled events of Cloud Watch.

neo.lee, what do you think about the AWS Support solution?


Thanks

Vicio


Anonymous
Not applicable

Hi Vicio,

Would you happen to know of any support documentation or could offer some advice on deploying a Qlik Sense Multi-Node site BUT across several AWS instances with one licence key?

I am required to implement this to ensure failover should an AWS instance go down, and your post is the closest I have came to any indication this is possible.

Many Thanks in Advance of any Info!!!

vicio
Contributor III
Contributor III
Author

Hi carm1988‌,

I'm working on it. I'm using already spot request instances, shared persistence, shared file server.

I still have some issues, so I'm not ready yet for provide an advice now. 

But AWS Multi-Node is definitively my focus in the next 2 weeks.

I also involved other resources in this project.

If you are interested to join cases, forces, solutions in a friendly, or professional way, please contact me directly.

Vicio

(Your request is 2 months old, and if you already did it, I could be interested to shortcut my development process)