Create
yaml
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"
Create clusters in Google Cloud Dataproc.
Examples
Creates a cluster in Google Cloud Dataproc.
yaml
id: "create"
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
workerMachineType: n1-standard-2
workers: 2
bucket: YOUR_BUCKET_NAME
Creates a cluster in Google Cloud Dataproc with specific disc size.
yaml
id: "create"
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
masterDiskSizeGB: 500
workerMachineType: n1-standard-2
workerDiskSizeGB: 200
workers: 2
bucket: YOUR_BUCKET_NAME
Properties
clusterName
- Type: string
- Dynamic: ✔️
- Required: ✔️
- Min length:
1
The cluster name.
region
- Type: string
- Dynamic: ✔️
- Required: ✔️
- Min length:
1
The region.
bucket
- Type: string
- Dynamic: ✔️
- Required: ❌
The GCS bucket name.
imageVersion
- Type: string
- Dynamic: ✔️
- Required: ❌
The Dataproc image URI.
The Compute Engine image resource used for cluster instances.
masterDiskSizeGB
- Type: integer
- Dynamic: ❌
- Required: ❌
The disk size in GB for each master node.
masterMachineType
- Type: string
- Dynamic: ✔️
- Required: ❌
The master machine type.
projectId
- Type: string
- Dynamic: ✔️
- Required: ❌
The GCP project ID.
scopes
- Type: array
- SubType: string
- Dynamic: ✔️
- Required: ❌
- Default:
[https://www.googleapis.com/auth/cloud-platform]
The GCP scopes to be used.
serviceAccount
- Type: string
- Dynamic: ✔️
- Required: ❌
The GCP service account key.
workerDiskSizeGB
- Type: integer
- Dynamic: ❌
- Required: ❌
The disk size in GB for each worker node.
workerMachineType
- Type: string
- Dynamic: ✔️
- Required: ❌
The worker machine type.
workers
- Type: integer
- Dynamic: ❌
- Required: ❌
The number of workers.
zone
- Type: string
- Dynamic: ✔️
- Required: ❌
The zone.
Outputs
clusterName
- Type: string
- Required: ❌
The cluster name.
created
- Type: boolean
- Required: ❌
- Default:
false
Whether cluster was created successfully.
Definitions
Was this page helpful?