Cloud Squeeze – AWS Cost Optimization & Right Sizing with AI

Amazon’s T2 Unlimited – Who should use it, when, how and the why?

Published on March 6, 2018
Share this on

Amazon recently introduced a feature that can be turned on in real time for a T2 instance family EC2 instance type, called T2 unlimited. In this article, let’s answer these three questions:

  • Does it make sense to use a T class instance type?
  • How does the T class work and what is “burstable”?
  • When does it make sense to enable T2 unlimited?

Does it make sense to use a T class instance type?

The T2 class EC2 is unique in that it has a baseline level of CPU performance and the ability to burst above it for brief periods of time. To compare with alternatives the C class and the M class are two EC2 classes, with some differentiation of features. Let’s introduce a thought, and I will explain further as we progress in this article:

“A workload that can be performed by a C or an M type instance may be processable by a T class – if it is burstable.”

Since T class inception, with a vague characteristic of “burstable”, most have stayed away from use in production like environments. The CPU can get throttled if CPU credits expire. T2 unlimited option is a solution to this problem, but one has to have a clear understanding of how T2 standard and T2 unlimited work. Let’s look at the price per vCPU hour in relationship to GB memory across these 3 EC2 classes (for various C, M and T classes).

For the below image, I am using current pricing for us-east-1 (North Virginia) region. Notice the green – T class comes up lower than C/M class for the number of vCPUs except for two C class blue dots (at 8GB, 16GB) for the Linux platforms.

compare AWS C, M, T class pricing

The two exception for the Linux platform is the c5.xlarge (4 vCPU, 8 GB) and c5.2xlarge (8 vCPUs, 16 GB) with a cost per CPU hour at $0.0425. Unfortunately, the c5 category of instance type isn’t available in all regions (for the mix of Unix/Linux and desired software configuration of an existing workload). This C5 type is an excellent example of lowered compute costs. Outside of these 2 exceptions, if your workload is “burstable”, for a fixed amount of GB memory, the T2 class has a lower cost than its close relatives – C and M.

“The C5 category of instance types can provide a better price performance than some T class instance types, if your workload can use it and if they are available in your region.”

For the Windows-based platform, there are no anomalies, T class wins across the board and has a lower cost per vCPU – across GB of memory.

AWS T2 pricing compared

How does the T class work and what is “burstable”?

This bursting attribute is fine as long as there are enough available CPU credits, otherwise performance is reduced to the baseline, and there is a potential of CPU throttling.

How AWS T2 standard works
As your workload consumes CPU, you are either in CPU credit building zone or CPU credit consumption zone. Four concepts to grasp here are:

  • CPU credit: 1 CPU credit is equal to 1 vCPU running at 100% for 1 minute, or 1 CPU operating at 50% for two minutes, and so forth computed in milliseconds.
  • Baseline performance: The CPU performance level at which you are neither using up a CPU credit nor earning any CPU credits.
  • Launch credits: 30 x the number of vCPU of the T class instance credits given at launch or reboot.
  • Maximum credit limit: A maximum limit of credits for an instance type that once accumulated over are forever-lost.

When your instance is operating under the baseline performance, you are building up credits unless you have reached your maximum credit limit, at which point they vanish. When you exceed your baseline CPU utilization, you first use up all the launch credits, and after all the launch credits are exhausted, you use your earned credits. All these calculations are in milliseconds. An alternate depiction of the above with added details is here. CPU throttling issue can occur when launch credits and earned credits are exhausted, i.e., your “CPU credit balance” is 0.

What is the baseline performance for the T class instances?

They vary. The number of vCPUs, memory and the baseline CPU as measured across all processors looks like:

AWS T2 class thresholds

Note above, there is an assumption on a workloads ability to equally distribute, compute-bound tasks across multiple processors, which is rarely the case except for some RDBMS and well-developed applications. However, this characteristic can be analyzed by looking at a workload’s ability to handle multiprocessing well.

“The baseline CPU performance across T2 standard class range from 5%-30%”

Most people don’t have a way to assess if a workload is burstable, from Cloudwatch data (without some statistical analysis). Many workloads, web server and some database workload patterns can qualify as “burstable”, which requires some analysis of past Cloudwatch data and assessing the quartiles in which your CPU consumption reside for a small period. A box plot type of view is one way to evaluate this:

CPU usage over time with quartiles

Here is an example over a period of days, a workload that may or may not be burstable:
CPU as boxplot for utilization analysis

Visualizing these peaks on an hourly basis shows, it is hard to analyze if a load is or isn’t burstable:
CPU utilization analysis

“Unless you have a workload where you use its memory and CPU well above baseline levels, you may have a burstable workload.”

When you enable T2 unlimited option, the potential of a throttled CPU performance from 0 CPU credits disappears. However, it comes at a slightly higher cost. Before we look at how T2 unlimited works, let’s compare how new T2 unlimited rates compare to other alternatives.

When T2 unlimited operates at the point where CPU credits are exhausted your new price is:

$0.05 per vCPU hour for Linux
$0.096 per vCPU hour for Windows

“With this fixed price based on vCPUs for T2 unlimited, the rationale to enable this on a t2.nano, or a t2.micro relative to a t2.small, that are all 1vCPUs with increasing memory is something one needs to assess!”

In the below picture I have included all the T class rates (when not operating at T2 unlimited) to compare this set with other C and M class instance types:

Linux
Price comparison of EC2 instance types

Windows:
Windows EC2 pricing compared

Pricing is region specific, and the above pricing is current as of this writing. AWS frequently updates its pricing, mostly lower as newer generation families and virtualized types are introduced. As to what the color coding represents for the current data:

  • Green cells at the top of the chart (includes T class at the price credits aren’t exhausted)
  • Yellow cells represent the price at which T class operate when credits are exhausted.
  • Orange represents the price of other M, C class servers, which far exceed T2 class servers with exhausted credits.

“T2 unlimited with credits exhausted operate at $0.05 per vCPU hour for Linux and $0.096 per vCPU hour for Windows.”

When does it make sense to enable T2 unlimited?

Now on how T2 unlimited works – the best feature of it is the ability to be turned on in real time without a reboot, unlike changing the workload from a T class to a C or M class system which requires a reboot. T2 unlimited introduces a new Cloudwatch monitor for “CPU Surplus Credit Balance,” and the launch credits from T2 standard get removed altogether, replaced for a full cup of days credits, similar to a cash advance on a paycheck.

At the point where your CPU credits get exhausted, AWS introduces an unlimited pipe, a valve turned on-demand, with these new rates for T class servers:

$0.05 per vCPU hour for Linux
$0.096 per vCPU hour for Windows

T2 unlimited - how it works

“T2 unlimited when running with borrowed AWS credits is at about the pricing of a C4 or M4 class server.”

The critical difference is a swapping of launch credits earned at boot up (30 times the number of vCPUs) to a borrowed advanced daily limit of credit.

We have added a contribution to AWS serverless repository that enables you to turn on T2 unlimited based on an SNS threshold, thereby switching from T2 standard to T2 unlimited after launch credits have been exhausted. You can choose these thresholds, and the serverless code turns a T2 standard to a T2 unlimited at the limit you select.

Note how we have measured price at the CPU hour across memory ranges, in units of hours so far.

“Serverless technology gives a workload the ability to scale vCPUs from 0-10,000+ with memory variations measured in GB-seconds.”

AWS provides 1M free invocations of serverless code each month or 400,000 GB-SECONDS. This fantastic capability to burst 10,000 vCPUs for few seconds, along with AI enable us to assess workload patterns and determine the best class instances, along with availability, price in few minutes irrespective of the number of workloads that need to be analyzed.

This SNS initiated serverless code is part of AWS’s serverless repository that you can use for free, that switches a T2 standard to T2 unlimited in one invocation when it occurs. Chances are unless you have many thousands of T2 instances or other serverless code, you should be well inside the free tier.

Note that T class operates at much lower cost when T2 unlimited feature is not in operation (green cells, in the context of the credit balance). Some of the newer C5, M5 EC2 types operate at better rates than T2 unlimited. T2 unlimited is priced at about the M4, C4 pricing. Assuming you have some variability in CPU utilization chances, are the T2 unlimited will provide a lower cost than the ones in yellow. If you have one of the instance types in deep orange color and there is a comparable T class that can perform that workload, the T2 unlimited, if turned on indefinitely, will play better than holding on to one of these instance types. If you are in a long-term reservation commitment on one of these classes – perhaps your AWS rep can help you find a way out!

With current spot, instance pricing one can occasionally get a P2 (GPU class server, with much higher memory and CPU) at T2 prices. It is like going to a car rental company where you usually get a Honda, and they offer you a stretched limousine at 1/10 the cost, with a clause that may require you to exit abruptly. How well can your workload use the capacity of the instance type is a better question than picking up an instance type that has so much excess capacity for you to grow over the years! This article on cloud capacity planning requires a new thinking addresses how capacity planning for the cloud needs a new kind of thinking.

“Choose between carrying excess capacity your workload cannot use or let your workload determine the best cloud resource type.”

Considering something that is cheaper, though your workload cannot use it, is thinking that has crept into IT from the days of former capacity planning where it would take years to provision capacity. For the economics of all this, review a workload from two dimensions – memory & CPUs. Every workload needs some amount of memory and CPU, and other components – storage, network, etc. Storage capacity (disks) in this scenario is somewhat standardized with EBS, and each of these EC2 types has some minor differentiation in networking capabilities. A workload running n a C class, M class can be shifted to a T class with a stop/start, usually with a couple of minutes of downtime. T2 unlimited is a real-time setting change, that changes the rates a bit higher, yet at par with C4/M4 rates. The newer generation C5, M5 if available in your region can give your workload better rates, assuming your workload can use the memory at its discretion.

“Let your workload determine the right cloud resource types.”

In summary, T class and unlimited when enabled as necessary provide a better solution to many workload types based on burst ability of a workload. Understand the true characteristics of your workload to determine which instance type gives you the best overall value, rather than picking up an instance type that has excess capacity for you to grow into!

“With cloud-based environments you can change instance families or types in minutes.”

Select the instance class that best suits your workload and consider T2 class with its unlimited feature enabled on demand with this free SNS triggered serverless code or T2 unlimited turned on if concerned of CPU throttling possibility impacting your applications! Unless your region has a C5 class instance type and your workload can use the CPU, memory configuration chances are there is a T2 that may be a better fit with or without T2 unlimited, depending on your compute patterns.

If you liked the rationale and this explanation of T2 standards vs. T2 unlimited, chances are you will like – How to transform capacity planning thought processes for your enterprise, you will find this 15-page depth white paper valuable.

To download this whitepaper please click the image above.

 

AWS Marketplace Cost Optimization

This tool gives you cost optimization advice based on your monthly AWS spend. We typically find 25-55% cost savings opportunities in your AWS account with a 5% minimum guarantee

 

Take a three-minute check-up, a seven-day no obligation free trial and visualize your specific AWS cost savings opportunities – guaranteed.

whitepaper - cloud capacity planning

CLOUD CAPACITY PLANNING

Receive this white paper and updates to transform the way you think about capacity planning