If you are looking for an automated way to manage quotas over a large number of projects, we are excited to introduce a Quota Monitoring Solution from Google Cloud Professional Services.
By default, Google Cloud employs resource quotas to restrict how much of a particular shared Google Cloud resource you can use. Each quota represents a specific countable resource, such as API calls to a particular service or the number of compute cores used concurrently by your project.
Quotas are enforced for a variety of reasons, including:
- To protect the community of Google Cloud users by preventing unforeseen spikes in usage and overloaded services.
- To help you manage resources. For example, you can set your own limits on service usage while developing and testing your applications to avoid unexpected bills from using expensive resources.
Quota Monitoring Solution solution benefits anyone who manages quotas across projects, folders, or organizations. It offers an easy and centralized way to view and monitor the quota usage in a central dashboard and to use default alerting capabilities across all quotas.
Specifically, the solution provides:
- Automated aggregation of quotas across all projects in given organizations or folders and a recurring scan at a defined frequency (e.g. hourly, daily) for new projects to automatically capture their quotas.
- A dashboard that provides visibility into recent resource usage against the individual quotas across all projects.
- Preconfigured alerting through email or other communication channels (e.g., email, SMS, Pub/Sub, etc.) when a resource reaches a certain threshold of its quota.
The solution is easily deployable through Terraform so that you can adopt it into your project with minimal time investment.
Outside of the Quota Monitoring Solution, there are additional ways of viewing your quota information, such as using the Google Cloud Console or using the gcloud command-line tool. You can also manually define Alerting Policies in Cloud Monitoring to send out notifications when a resource reaches a certain threshold of its quota. For example, you can define an alerting policy that triggers when the CPU usage of Compute Engine VM instances goes above 75% of the quota in any region.
In case your project needs more of a particular resource than your quota allows, you can request a quota limit increase for the majority of quotas directly in the Google Cloud Console. In the vast majority of cases, quota increase requests are evaluated and processed automatically. However, depending on the nature of your request, a small number of quota increase requests needs to be handled by human reviewers. They typically process your request within 2-3 business days, so it is important to plan ahead.
Ineffective quota management can lead to many different problems. For example, the lack of sufficient quota can prevent consuming additional resources, which could be needed for auto-scaling events, or for performing a GKE cluster upgrade. This can cause outages or service degradation, which could impact your customers’ experience and potentially impact your business revenues.
Please note: Many services also have limits that are unrelated to the quota system. These are fixed constraints, such as maximum file sizes or database schema limitations, which cannot be increased or decreased. You can find out about these on the relevant service’s Quotas and limits page (for example, Cloud Storage quotas and limits).
1. Technical Architecture
The diagram below shows the Quota Monitoring Solution architecture flow you can deploy in minutes using the deployment guide and accompanying terraform scripts.