To utilize spot instance, you can call EC2 spot instance termination notice API and insert a pending terminate state to do an application snapshot and store it into S3, you then can repeat step 1 by bootstrapping all required dependencies to another spot instances.All of these be automated.
I'm using Reserved Instances for my productions servers: no problems.
I'm using On-Demand Instances for my staging/test/development servers: no problem.
I'm considering to use Spot Request Instances.
In my case, Operating System, software and data are on EBS volume, that I would like to preserve over the start/stop process.
I'm using Window R2016 Base with an m4.4xlarge EC2.
I would like to start an m4.16xlarge Spot Request Instance in the morning and use it
for 4-5 hours or more. Stop at Lunch Break and restart for other 4-5 hours.
I would like to be sure that the data are saved on EBS, and I can restart with my existing EBS (OnDemand, Spot Request)?
AWS Support reply:
To run the instances as per your requirement, you can make use of ‘Reserve for Duration’ option. Also, make sure that you uncheck the ‘Delete’ option on the root volume while requesting a Spot instance. This will make sure that even though the instance is terminated, the volume will be stored and your data will not be lost.
Spot instance do not shut down the instances but completely terminate them. However, since we have the volume of the instance, we can attach it as a root volume to a new instance (OnDemand).
If you would like to continue with the Spot Instance, with the same volume of the terminated instance attached to the new instance, you can:
1. Create a new spot request instance. Make sure to uncheck "Delete on Termination" for the root device, so that the volume stays behind in the next step. Make sure to note the architecture and the kernel ID that your instance is using (very important!)
2. Now, RDP into your new instance and make a file in the filesystem. After making some changes to the filesystem, go ahead and log out of the RDP connection and terminate the instance.
3. Login to your EC2 web console and find the new volume that was being used for the instance we just terminated. Right-click the volume and select "Create Image". Follow the wizard, making certain to select the same architecture and kernel ID that we noted earlier.
4. Now, start the spot request wizard using your new image. Follow the wizard, again making certain to uncheck "Delete on Termination". Additionally, and this is the easy step to miss, make sure to expand the collapsed section titled 'Advanced Options' and set the correct kernel ID again.
Following the above steps, you will have a new instance at the same point that your old instance was at when it was terminated.
I tested manually and it works smoothly.
At first sight, I thought was a little bit light on security, but I balanced with a high-frequency backup of the apps that is useful for my developers to revert to previous versions of the apps.
My yearly OnDemand quote for m4.16xlarge (188ECU, 256RAM) for 10hours per working day: 12,000$. With Spot Request on zone 1a of my region -> 6,500$, zone 1b -> 3,000$ (75% saving).
Awesome AWS. This is intended for Test Site Qlik Sense server.
Study an automatic procedure that my IT guys can setup with just AWS console (no API) and execute the bid and the restarts with scheduled events of Cloud Watch.
neo.lee, what do you think about the AWS Support solution?