What is orchestration?
Orchestration often refers to the automated configuration, coordination, and management of computer systems and services.
In the context of service-oriented architectures, orchestration can range from simply executing a single service at a specific time and day, to a more sophisticated approach of automating and monitoring multiple services over longer periods of time, with the ability to react and handle failures as they crop up.
In the data engineering context, orchestration is central to coordinating the services and workflows that prepare, ingest, and transform data. It can go beyond data processing and also involve a workflow to train a machine learning (ML) model from the data.
There is no shortage of orchestration tools in Google Cloud. In this blog post, we will explore service and data orchestration tools and help you choose what’s best for your use case.
Orchestration in Google Cloud
Google Cloud Platform offers a number of tools and services for orchestration:
- Cloud Scheduler for schedule driven single-service orchestration
- Workflows for complex multi-service orchestration
- Cloud Composer for orchestration of your data workloads