3 Replies Latest reply: May 30, 2017 12:01 PM by Carmelle Morrison RSS

    Qlik Sense on AWS Spot Request Instances

    Vicio Giglio

      I'm currently using Qlik Sense 3.2 SR4 on Windows R2016 Instances: On-Demand Instances and Reserved Instance.

       

      Has anybody experienced Qlik Sense on Spot Request Instances?

      I have a Test site License and it could be great to use for development on a cheap and powerful instance.

       

      Is possible to preserve the data on an EBS volume and restart (the day after or after a forced shutdown)

      without losing any progress on Data?

       

      I intend to try anyway in the next days, but perhaps someone has already walked this path and share some hints.

        • Re: Qlik Sense on AWS Spot Request Instances
          Vicio Giglio

          Hi neo.lee,

           

          I added your reply here because I created this more specific thread before. Hope you don't mind.

          wrote:

          To utilize spot instance, you can call EC2 spot instance termination notice API and insert a pending terminate state to do an application snapshot and store it into S3, you then can repeat step 1 by bootstrapping all required dependencies to another spot instances.All of these be automated.


          • Re: Qlik Sense on AWS Spot Request Instances
            Vicio Giglio

            I have some solution from AWS Support:

             

            My question:

             

            I'm using Reserved Instances for my productions servers: no problems.

            I'm using On-Demand Instances for my staging/test/development servers: no problem.

            I'm considering to use Spot Request Instances.

            In my case, Operating System, software and data are on EBS volume, that I would like to preserve over the start/stop process.

            I'm using Window R2016 Base with an m4.4xlarge EC2.

            I would like to start an m4.16xlarge Spot Request Instance in the morning and use it

            for 4-5 hours or more. Stop at Lunch Break and restart for other 4-5 hours.

            I would like to be sure that the data are saved on EBS, and I can restart with my existing EBS (OnDemand, Spot Request)?



            AWS Support reply:

             

            To run the instances as per your requirement, you can make use of ‘Reserve for Duration’ option. Also, make sure that you uncheck the ‘Delete’ option on the root volume while requesting a Spot instance. This will make sure that even though the instance is terminated, the volume will be stored and your data will not be lost.

             

            Spot instance do not shut down the instances but completely terminate them. However, since we have the volume of the instance, we can attach it as a root volume to a new instance (OnDemand).

             

            If you would like to continue with the Spot Instance, with the same volume of the terminated instance attached to the new instance, you can:

             

            1. Create a new spot request instance. Make sure to uncheck "Delete on Termination" for the root device, so that the volume stays behind in the next step. Make sure to note the architecture and the kernel ID that your instance is using (very important!)

            2. Now, RDP into your new instance and make a file in the filesystem. After making some changes to the filesystem, go ahead and log out of the RDP connection and terminate the instance.

            3. Login to your EC2 web console and find the new volume that was being used for the instance we just terminated. Right-click the volume and select "Create Image". Follow the wizard, making certain to select the same architecture and kernel ID that we noted earlier.

            4. Now, start the spot request wizard using your new image. Follow the wizard, again making certain to uncheck "Delete on Termination". Additionally, and this is the easy step to miss, make sure to expand the collapsed section titled 'Advanced Options' and set the correct kernel ID again.

             

            Following the above steps, you will have a new instance at the same point that your old instance was at when it was terminated.

             

            Conclusion:

            I tested manually and it works smoothly.

            At first sight, I thought was a little bit light on security, but I balanced with a high-frequency backup of the apps that is useful for my developers to revert to previous versions of the apps.


            My yearly OnDemand quote for m4.16xlarge (188ECU, 256RAM) for 10hours per working day: 12,000$. With Spot Request on zone 1a of my region -> 6,500$, zone 1b -> 3,000$ (75% saving).

            Awesome AWS. This is intended for Test Site Qlik Sense server.

             

            Next step:

            Study an automatic procedure that my IT guys can setup with just AWS console (no API) and execute the bid and the restarts with scheduled events of Cloud Watch.

             

            neo.lee, what do you think about the AWS Support solution?


            Thanks

             

            Vicio

             

             

             


              • Re: Qlik Sense on AWS Spot Request Instances
                Carmelle Morrison

                Hi Vicio,

                 

                Would you happen to know of any support documentation or could offer some advice on deploying a Qlik Sense Multi-Node site BUT across several AWS instances with one licence key?

                 

                I am required to implement this to ensure failover should an AWS instance go down, and your post is the closest I have came to any indication this is possible.

                 

                Many Thanks in Advance of any Info!!!