With Cloud becoming more pervasive, many organisations will find a significant and unexpected cost challenge arising from cloud platform fees.
This should be expected - Imagine what kind of bill you will receive if you left your 'sensible' child in a sweet shop or arcade for a day to do as he wants... Why should it be any different when you leave your team with a platform that provides 'unlimited' resources?
Behaviours will change when the operating environment changes
At Midships, we know this from experience (both with kids and our own team). The team's behaviour changed from one of continuously conserving / optimising disk space to one where they stopped all housekeeping as they no longer needed to worry about running out of disk space.
Being a small team, we were able to take control and optimise it early on.
This article will discuss the behaviours that we have observed as a result of using a Cloud platform and how these contribute to cost. It will also identify practical actions that should be taken to minimise them and enable you to get your cloud spend back under control.
Behaviours
When organisations move to the cloud, we have observed the following factors that seem to drive a change in organisational behaviour:
Unlimited Resources - Cloud platforms unshackle organisations from their previous physical resource constraints (CPU, Memory, Storage, Rack Space). This immediately drives different processes when additional virtual resources are required (as you no longer need to face the scrutiny and delays often associated with purchasing physical assets). This coupled with the low perceived cost means that less thought goes into what is being deployed and the necessary housekeeping that follows.
Speed of Delivery - The pressure and expectation to deliver quickly and overcome blockers has increased with agile. As a result, more individuals are empowered to provision resources (themselves or by requesting a central team). Whilst this is great for the speed of delivery, it does create sprawl with many resources under utilised or not used / required at all.
Always On - 24x7 and Online has increased the focus on availability and performance. New peaks are now being created which can be more significant than what was seen before and as a result you need to ensure you have the capacity to manage these. The general philosophy of when in doubt, provision more is often followed, leading to wasted resources.
What can you do to control spend without inhibiting delivery & service?
Ensure each provisioned resource has an owner & hold them to account
When provisioning any resource, ensure ownership is assigned to an individual who is accountable for:
Justifying the resource requirement & projecting the associated cost;
Undertaking housekeeping activities including deprovisioning, cleaning up data and switching off resources when not being used (e.g. at night)
Managing resource utilisation to ensure that resources are not being under utilised by more than x%
Management should use data to hold resource owners to account in the same way as they use data to drive delivery.
Gain Visibility
Use tools to gain visibility of your cloud use and bills as well as to empower resource owners (see above).
At Midships, we are developing a multi cloud portal (currently in beta) which will enable our customers to obtain a single view of all their cloud usage across all the cloud platforms by resources, actual utilisation etc.
Use tools like ours to understand: what has been deployed; where and why in order to better manage sprawl; actual resource utilisation to identify under utilisation; setup budget alerts; and, monitor usage.
The aim is to use these data points to effectively and proactively take tangible & decisive action to drive down costs without inhibiting delivery.
Leverage automation with machine learning & data mining to optimise / delete unused resources
Use machine learning & data mining to proactively identify:
Where deployed services are not being accessed (due to zero user traffic)
Usage patterns, which can then be used to automate scaling & switch on/off of services
Identify over provisioning
At Midships we use automation to minimise under & over provisioning by the minute or less where possible. We have the expertise to automate the (de-)provisioning of resources such as CPU, Memory and Network Bandwidth (virtual circuits).
Strategically use SaaS
In our experience there are some services where you should consider using a SaaS as opposed to running yourself. In these cases, the cost of operation and maintaining the service is usually far higher than the SaaS equivalent. Some good examples include: DevOps Tooling such as CI/CD server, code repos, security scanning tools like anchore; SMS gateway etc. Many of these SaaS services will be able to support your data sovereignty and security requirements and are worth considering.
Without getting control of your cloud and continuously optimising it, you could find that you have inadvertently opened Pandora's box.
Midships works with organisations to increase control and optimise cloud spend. To learn more, please reach out to ajit@midships.io