6 ways to control and optimize your cloud costs
Most companies have migrated some or all of their digital applications and platforms to the cloud. Multiple environments are provisioned in different cloud regions. Every workload that runs in the cloud incurs a combination of compute, storage and networking costs. We all know that without proper control, expenses will most likely grow unexpectedly. This article describes six ways to control and optimize your cloud costs.
1. Set budgets and review costs
After setting up a cloud account, the first thing is to create your budget with planned expenses. Budgets allow you to monitor spending overtime, forecast costs using historical data, and, most importantly, alert you when your expenses reach preconfigured limits.
Budget setup is relatively straightforward, and the functionality is very similar between AWS and Azure.
Billing details distributed by cloud vendors may explain more about your spending. You can use this information to understand which services incur the highest costs and which are redundant.
If you are developing and operating multiple cloud products, we recommend you set up separate billing accounts/subscriptions with the corresponding budget for each product. This approach will provide a better understanding of the total costs over the product’s lifecycle.
Best practice is to use resource tagging to better understand the costs for different cloud services used in your project. Costs can be grouped by these tags and provide more detailed information. If it comes to resource tagging, following your cloud providers’ standard naming conventions is usually best for scale and transparency.
2. Take advantage of discounts and savings plans
Some cloud vendors offer discounts for development and testing environments. For example, Azure discounts active Visual Studio subscribers with “up to 57% for a typical web app dev/test environment running SQL Database and application service”.
Another good option is using saving plans for your computing resources – lower prices in exchange for a specific one-year or three-year usage commitment. Sometimes these saving plans are available not only for virtual machines but also for cloud services. At the time of writing (April 2023), Amazon has a particularly attractive savings plan for SageMaker.
3. Use reserved and spot instances
Reserved instances (AWS) and Reserved Virtual Machine Instances are prepaid computing resources. They are particularly advantageous for predictable and stable workloads.
To purchase reserved instances, you must provide information about the type of instances, their size and location (region and availability zone), and usage duration (one year or three years). The final price might also be affected by the payment type you select (paying in full upfront, partial payment upfront, or nothing upfront). By using reserved instances, you can save up to 75% of your costs compared to on-demand pay-as-you-go options.
Side by side comparison of AWS and Azure reserved instances:
| AWS standard | AWS convertible | Azure | |
|---|---|---|---|
| Term | 1 or 3 years | 1 or 3 years | 1 or 3 years |
| Payment options | Upfront, partial or no upfront | Upfront, partial or no upfront | Upfront only |
| Reserving instance sharing | Yes | Yes | Yes |
| Exchange options | Availability Zone, instance size and networking type only | Full flexibility | Full flexibility |
| Cancellation | No | No | No |
| Priority access to spare capacity | Only when scoped to an availability zone | Only when scoped to an availability zone | Yes |
| Auto scaling support | Yes | Yes | Yes |
| Maximum discount* | 75% | 54% | 72% |
Even bigger savings can be made by using spot instances. Spot instances are available for last-minute purchases on cloud vendor auctions; those are usually leftover resources. Spot instances (compared to reserved instances) can’t be used for critical workloads as the cloud vendor can terminate them at any time. The best scenario for using spot instances is a batch job or non-critical/non-time-consuming job, which ends immediately and resumes when resources are available.
4. Identify and remove unused resources
The easiest way to optimize cloud costs is by identifying and removing unused or underutilized resources. The most common scenarios for such resources include:
- Developers have provisioned temporary compute instances and forget to remove them
- Compute instances have been removed without removing attached storage
- Network interfaces or security groups without attached Virtual Machines (VM)
It is always a good idea to review your resources’ usage information. For example, if your VM usage is only 20%, you can most likely reserve/buy a smaller VM, which will be more cost-effective. For a cluster that is just 20%, the better approach would be to use auto-scaling, which automatically adjusts the number of resources in your cluster based on your needs according to CPU or RAM metrics.
A typical scenario concerns cloud resources that are not in use 24/7. This is common when the entire development team works in the same time zone. In this case, you can provision cloud environment resources for a specific number of hours (for example, 8 hours) and de-provision after that. This approach may reduce your costs two or three times.
Cloud vendors and third-party companies provide different tools to monitor your resource usage and detect unused resources (for example, Azure Cost Management). The tools detect usage anomalies and propose more cost-effective resource types fitting your needs based on historical usage data.
5. Use appropriate storage options and data lifecycle policies
Cloud storage lets you move data from your on-premises physical devices to highly accessible and scalable cloud servers. Those servers’ data can be accessed via a user interface or a specific storage API. Cloud storage is classified into three main categories: object storage, file storage and block storage. When you move data to the cloud, the storage and quality properties (performance, resilience, accessibility, security, etc.) come with the option you configure.
Cloud storage has different tiers, and the price for each is very different. It’s much more economical to store data as a backup or for legal commitments – and thus never or rarely accessed – in the “cold” tier, which is dramatically cheaper than the “hot” tiers used for frequently accessed data. For example, 100TB in Azure Data Lake Storage (Gen2) in the “cold” tier with a 1-year reservation is 17 times (!) cheaper than in the “hot” tier. But you need to consider the following restrictions:
- There will be minimum contract lengths for these tiers
- Retrieving data from cold tiers might be expensive and can take a few hours
6. Calculate data transfer costs
Cloud vendors charge different fees for moving data to and from the cloud. Data transfer into the cloud provider’s network (ingress) fees are relatively cheap. But note that ingress fees can be incurred when transferring data between cloud regions, even within the same provider. Therefore, if you are planning data migration – for example, from your on-premises data storage to your public cloud – it is advisable to access and evaluate your cloud vendor’s data transfer fees.
Expect data transfer from the cloud (egress) to be more costly than ingress and, in our experience, rarely budgeted for. The “rule of thumb” we use is 5-20 cents per GB for moving data from AWS or Microsoft Azure to other data centers or on-premises locations.
If your application is transferring large data volumes from on-premises to the cloud, it can be worth considering migrating the whole application onto the cloud infrastructure.
If you are unsure
If you are unsure, and think that you may be missing something, arranging a cost optimization assessment with your cloud vendor or trusted partner specialists is always possible. During this assessment, cloud vendor experts and your team will review your architecture, analyze historical resource usage and come up with recommendations on how to reduce or optimize your costs. Experts will also explain which tools you can use to control and optimize your cloud costs regularly without assistance.
Please note that if you are a large premium client, then there’s a good chance the assessment service is already included in your cloud vendor contract at no additional costs.
If you’d like to learn more about cloud cost control and optimization, other cloud-related topics or have any doubts about the best practices, get in touch with our team! You can also visit our Insights page, where our team explores and shares expertise on various tech topics such as why it is Time to put MVP to rest, thoughts on Explaining DevOps to executives, and the 2 key metrics delivery managers should care about first.
Additional Insights
The DevOps shortcut to AI
Explaining DevOps to executives
