Dataproc on GKE overviewStay organized with collectionsSave and categorize content based on your preferences.
Dataproc on GKE allows you to execute Big Data applications using the
DataprocjobsAPI on GKE clusters.
Use the Google Cloud console, Google Cloud CLI or the Dataproc API
(HTTP request or Cloud Client Libraries) tocreate a Dataproc on GKE virtual cluster,
then submit a Spark, PySpark, SparkR, or Spark-SQL job to the Dataproc
service.
Dataproc on GKE deploys Dataprocvirtualclusters on
a GKE cluster. UnlikeDataproc on Compute Engine clusters,
Dataproc on GKE virtual clusters do not include separate
master and worker VMs. Instead, when you create a Dataproc on GKE virtual cluster,
Dataproc on GKE creates node pools within a GKE cluster. Dataproc on GKE
jobs are run as pods on these node pools. The node pools and
scheduling of pods on the node pools are managed by GKE.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-11-11 UTC."],[],[]]
