Identity and Access Management (IAM) lets you control access to your project's resources. This document focuses on the IAM permissions relevant to Serverless for Apache Spark and the IAM roles that grant those permissions.
Dataproc permissions for Serverless for Apache Spark
Dataproc permissions allow users and service accounts
,
to perform actions on Serverless for Apache Spark
resources. For example, the dataproc.batches.create
permission lets you create batch workloads in a project.
You don't directly give users permissions; instead, you grant them IAM roles, which have one or more permissions bundled within them. You can grant predefined roles that contain a list of permissions, or you can create and grant custom roles that contain one or more permissions that you include in the custom role.
The following tables list the basic permissions necessary to call
Dataproc APIs (methods) that create or access Serverless for Apache Spark
resources. The tables are organized according to the APIs associated with each
Serverless for Apache Spark resource, which include batches
, sessions
, sessionTemplates
, and operations
.
Examples:
-
dataproc.batches.createallows the creation of batches in the containing project. -
dataproc.sessions.createallows the creation of interactive sessions in the containing project.
Batch permissions
| Method | Required Permission(s) |
|---|---|
| projects.locations.batches.create | dataproc.batches.create 1 |
| projects.locations.batches.delete | dataproc.batches.delete |
| projects.locations.batches.get | dataproc.batches.get |
| projects.locations.batches.list | dataproc.batches.list |
1
dataproc.batches.create
also requires dataproc.batches.get
and dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Session permissions
| Method | Required Permission(s) |
|---|---|
| projects.locations.sessions.create | dataproc.sessions.create 1 |
| projects.locations.sessions.delete | dataproc.sessions.delete |
| projects.locations.sessions.get | dataproc.sessions.get |
| projects.locations.sessions.list | dataproc.sessions.list |
| projects.locations.sessions.terminate | dataproc.sessions.terminate |
1
dataproc.sessions.create
also requires dataproc.sessions.get
and dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Session template permissions
| Method | Required Permission(s) |
|---|---|
| projects.locations.sessionTemplates.create | dataproc.sessionTemplates.create 1 |
| projects.locations.sessionTemplates.delete | dataproc.sessionTemplates.delete |
| projects.locations.sessionTemplates.get | dataproc.sessionTemplates.get |
| projects.locations.sessionTemplates.list | dataproc.sessionTemplates.list |
| projects.locations.sessionTemplates.update | dataproc.sessionTemplates.update |
1
dataproc.sessionTemplates.create
also requires dataproc.sessionTemplates.get
and dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Operations permissions
| Method | Required Permission(s) |
|---|---|
| projects.regions.operations.get | dataproc.operations.get |
| projects.regions.operations.list | dataproc.operations.list |
| projects.regions.operations.cancel 1 | dataproc.operations.cancel |
| projects.regions.operations.delete | dataproc.operations.delete |
| projects.regions.operations.getIamPolicy | dataproc.operations.getIamPolicy |
| projects.regions.operations.setIamPolicy | dataproc.operations.setIamPolicy |
1
To cancel batch operations, dataproc.operations.cancel
also requires dataproc.batches.cancel
permission.
Serverless for Apache Spark 3.0+ runtime permissions
The following permissions apply to Serverless for Apache Spark 3.0
and
later runtimes.
Workloads permissions
| Method | Required Permission(s) |
|---|---|
| dataprocrm.v1.dataprocrm.projects.locations.workloads.create | dataprocrm.workloads.create |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.cancel | dataprocrm.workloads.cancel |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.delete | dataprocrm.workloads.delete |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.get | dataprocrm.workloads.get |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.list | dataprocrm.workloads.list |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.use | dataprocrm.workloads.use |
NodePools permissions
| Method | Required Permission(s) |
|---|---|
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.create | dataprocrm.nodePools.create |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.delete | dataprocrm.nodePools.delete |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.resize | dataprocrm.nodePools.resize |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.deleteNodes | dataprocrm.nodePools.deleteNodes |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.update | dataprocrm.nodePools.update |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.get | dataprocrm.nodePools.get |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.list | dataprocrm.nodePools.list |
Nodes permissions
| Method | Required Permission(s) |
|---|---|
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.create | dataprocrm.nodes.create |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.delete | dataprocrm.nodes.delete |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.update | dataprocrm.nodes.update |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.heartbeat | dataprocrm.nodes.heartbeat |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.get | dataprocrm.nodes.get |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.list | dataprocrm.nodes.list |
| dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.mintOAuthToken | dataprocrm.nodes.mintOAuthToken |
Operations permissions
| Method | Required Permission(s) |
|---|---|
| dataprocrm.v1.dataprocrm.projects.locations.operations.get | dataprocrm.operations.get |
| dataprocrm.v1.dataprocrm.projects.locations.operations.list | dataprocrm.operations.list |
Serverless for Apache Spark role requirements
The following table lists roles that contain the permissions required to manage batch workloads and sessions. The requirements can vary depending on the batch or session runtime version and whether the batch or session is running with service account or end-user credentials (EUC).
3.0
- Dataproc Editorrole to run batches or sessions or the Dataproc Viewerrole to get and list batches and sessions.
- Service Account Userrole on the Compute Engine default service account or a custom service account.
3.0
+- Dataproc Serverless Editorrole to submit batches and sessions or the Dataproc Serverless Viewerrole to get and list batches and sessions.
- If you select Service Account credentials
instead of accepting the default
3.0+ runtime End User Credentials (EUC), grant users the Service Account Userrole on a user-specified custom service account, and grant the user-specified custom service account the Dataproc Serverless Noderole.
Notes:
-
When submitting a batch workload or creating an interactive session with the a
3.0+runtime and end user credentials (the 3.0+ default), dataplane system operations are executed by the Dataproc Resource Manager Node Service Agent. For more information, see 3.0+ runtime service agent service account . -
For backward compatibility, the legacy Dataproc Editorand Dataproc Viewerroles can be granted with
3.0+runtimes instead of the Dataproc Serverless Editorand Dataproc Serverless Viewerroles. Also, the Dataproc Workerrole can be granted instead of the Dataproc Serverless Noderole. -
If a service account has been granted the project Editorrole, it contains the permissions included in the Dataproc Workerrole.
-
For more information: see Serverless for Apache Spark service accounts .
Do you need to grant roles?
Depending on your organization policy, a required role may already have been granted.
Check roles granted to users
To see if a user has been granted a role, follow the instructions in Manage access to projects, folders, and organizations > View current access .
Check roles granted to service accounts
To see if the a service account has been granted a role, see View and manage IAM service account roles .
Check roles granted on a service account
To see if a user has been granted a role on a service account, follow the instructions in Manage access to service accounts > View current access .
Lookup Dataproc roles and permissions
You can use the following sections to lookup Dataproc roles and permissions.
Dataproc Administrator
( roles/
)
Full control of Dataproc resources.
Dataproc Editor
( roles/
)
Provides the permissions necessary for viewing the resources required to manage Dataproc, including machine types, networks, projects, and zones.
Lowest-level resources where you can grant this role:
- Cluster
Dataproc Hub Agent
( roles/
)
Allows management of Dataproc resources. Intended for service accounts running Dataproc Hub instances.
Dataproc Serverless Editor
( roles/
)
Permissions needed to run serverless sessions and batches as a user
Dataproc Serverless Node.
( roles/
)
Node access to Dataproc Serverless sessions and batches. Intended for service accounts.
Dataproc Serverless Viewer
( roles/
)
Permissions needed to view serverless sessions and batches
Dataproc Service Agent
( roles/
)
Gives Dataproc Service Account access to service accounts, compute resources, storage resources, and kubernetes resources. Includes access to service accounts.
Dataproc Viewer
( roles/
)
Provides read-only access to Dataproc resources.
Lowest-level resources where you can grant this role:
- Cluster
Dataproc Worker
( roles/
)
Provides worker access to Dataproc resources. Intended for service accounts.
Project roles
You can also set permissions at the project level by using IAM Projectroles. The following table summarizes the permissions associated with IAM project roles:
| Project Role | Permissions |
|---|---|
| Project Viewer | All project permissions for read-only actions that preserve state (get, list) |
| Project Editor | All Project Viewer permissions plus all project permissions for actions that modify state (create, delete, update, use, cancel, stop, start) |
| Project Owner | All Project Editor permissions plus permissions to manage access control for the project (get/set IamPolicy) and to set up project billing |
What's next
- Learn how to Manage access to projects, folders, and organizations .

