Announced May 2025: Dataproc Serverless is now Google Cloud Serverless for Apache Spark

    Google Cloud Serverless for Apache Spark

    Focus on your code, not your infrastructure

    Run your Apache Spark jobs easier on a customizable zero-ops platform, smarter with Gemini assistance, and faster with the performance of Lightning Engine.

    Apache Spark is a trademark of The  Apache Software Foundation .


    Features

    Industry-leading performance

    Supercharge your jobs with Lightning Engine, our next-generation vectorized engine. Get over 4.3x faster performance and lower TCO for your serverless Spark workloads, automatically.

    Learn about Lightning Engine

    Zero-Ops with intelligent autoscaling

    Eliminate cluster management with intelligent autoscaling. Resources scale up, and down automatically to perfectly match your job's needs, ensuring maximum performance, and cost-efficiency without paying for idle time.

    Learn about autoscaling

    AI-powered development

    Accelerate your entire workflow. Write and debug PySpark, Scala, and Java code with Gemini Code Assist in BigQuery Studio and launch GPU-accelerated environments with pre-configured ML Runtimes.

    Explore Gemini assistance

    Unified Spark and SQL experience

    Eliminate context switching. Develop and run your workloads in a single environment like BigQuery Studio, seamlessly blending powerful SQL with the flexibility of PySpark in the same notebook.


    Read about PySpark in BigQuery Studio

    Two tiers of performance

    Two tiers of performance
    Tiers to match your specific needs, from standard batch processing to the most demanding, performance-critical jobs.
    Tier
    Best for

    Standard

    Ideal for cost-effective batch processing, data transformations, and general-purpose Spark jobs.

    • General purpose Spark ETL
    • Scheduled data pipelines
    • Cost-sensitive batch jobs

    Premium

    For the most demanding workloads, offering maximum performance with Lightning Engine, AI/ML acceleration, and interactive capabilities.

    • Performance-critical jobs powered by Lightning Engine for 4.3x boost
    • Interactive data science and analysis
    • GPU-accelerated AI and ML
    • Complex, large-scale data processing

    Compare tiers of Serverless Spark on Google in more detail .

    Two tiers of performance

    Tiers to match your specific needs, from standard batch processing to the most demanding, performance-critical jobs.

    Standard

    Best for

    Ideal for cost-effective batch processing, data transformations, and general-purpose Spark jobs.

    • General purpose Spark ETL
    • Scheduled data pipelines
    • Cost-sensitive batch jobs

    Premium

    Best for

    For the most demanding workloads, offering maximum performance with Lightning Engine, AI/ML acceleration, and interactive capabilities.

    • Performance-critical jobs powered by Lightning Engine for 4.3x boost
    • Interactive data science and analysis
    • GPU-accelerated AI and ML
    • Complex, large-scale data processing

    Compare tiers of Serverless Spark on Google in more detail .

    How It Works

    Develop your Apache Spark application in your favorite tools, including BigQuery Studio notebooks. Submit your serverless Spark job with a single command, and let Google handle the rest—no clusters to create, configure, or manage.


    Common Uses

    Interactive Data Science

    Empower data scientists to explore data and rapidly iterate on Spark ML models. Unify SQL and Spark in a single BigQuery Studio notebook, moving seamlessly from data exploration with SQL to model building with PySpark without ever managing infrastructure.

  • Use the JupyterLab extension to develop serverless Spark workloads

    • Interactive Data Science

    Empower data scientists to explore data and rapidly iterate on Spark ML models. Unify SQL and Spark in a single BigQuery Studio notebook, moving seamlessly from data exploration with SQL to model building with PySpark without ever managing infrastructure.

  • Use the JupyterLab extension to develop serverless Spark workloads

    • Automated ETL Pipelines

     Build robust, event-driven Spark ETL pipelines that automatically scale on demand. Pay only for what you use, making it perfect for spiky or unpredictable workloads.

      Automated ETL Pipelines

       Build robust, event-driven Spark ETL pipelines that automatically scale on demand. Pay only for what you use, making it perfect for spiky or unpredictable workloads.

        AI/ML at scale

        Accelerate large-scale model training and batch inference with serverless Spark. Attach NVIDIA GPUs with pre-configured libraries with a single command.

          AI/ML at scale

          Accelerate large-scale model training and batch inference with serverless Spark. Attach NVIDIA GPUs with pre-configured libraries with a single command.

            Pricing

            Transparent, value-driven pricing
            Serverless for Apache Spark pricing is based on per-second usage of compute (DCUs), GPUs, and shuffle storage.
            Services and usage
            Subscription type
            Price (USD)

            Data Compute Unit (DCU)

            Standard

            Starting at

            $0.06

            per hour

            Premium

            Starting at

            $0.089

            per hour

            Shuffle storage

            Standard

            Starting at

            $0.04

            per GB/month

            Premium

            Starting at

            $0.1

            per GB/month

            Accelerator pricing

            a100 40 GB

            Starting at

            $3.52069

            per hour

            a100 80 GB

            Starting at

            $4.713696

            per hour

            L4

            Starting at

            $0.672048

            per hour

            View pricing details for Google Cloud Serverless for Apache Spark.

            Transparent, value-driven pricing

            Serverless for Apache Spark pricing is based on per-second usage of compute (DCUs), GPUs, and shuffle storage.

            Data Compute Unit (DCU)

            Subscription type

            Standard

            Price (USD)

            Starting at

            $0.06

            per hour

            Premium

            Subscription type

            Starting at

            $0.089

            per hour

            Shuffle storage

            Subscription type

            Standard

            Price (USD)

            Starting at

            $0.04

            per GB/month

            Premium

            Subscription type

            Starting at

            $0.1

            per GB/month

            Accelerator pricing

            Subscription type

            a100 40 GB

            Price (USD)

            Starting at

            $3.52069

            per hour

            a100 80 GB

            Subscription type

            Starting at

            $4.713696

            per hour

            L4

            Subscription type

            Starting at

            $0.672048

            per hour

            View pricing details for Google Cloud Serverless for Apache Spark.

            Pricing calculator

            Calculate your monthly costs by region.

            Custom quote

            Connect with our sales team to get a custom quote for your organization.

            Get started today

            Tutorial for getting started

            Have a large project?

            Product documnetation

            Use BigQuery connector with Serverless for Apache Spark

            Use GPUs with Serverless for Apache Spark

            Business Case

             Build your business case for Google Cloud Serverless for Apache Spark

            The economic benefits of Google Cloud Dataproc and Serverless Spark versus alternative solutions

            See how Serverless for Apache Spark delivers significant TCO savings and business value compared to on-prem and other cloud solutions.

            In the report:

            Discover how Dataproc and Serverless for Apache Spark can deliver 18% to 60% cost savings compared to other cloud-based Spark alternatives.

            Explore how Google Cloud Serverless for Apache Spark can provide 21% to 55% better price-performance than other serverless Spark offerings.

            Learn how Dataproc and Google Cloud Serverless for Apache Spark simplify Spark deployments and help reduce operational complexity.

            FAQ

            When should I choose Serverless for Apache Spark versus Dataproc?

            Choose Serverless for Apache Spark when you want to focus on your code and eliminate all infrastructure management. It's ideal for new Spark pipelines, interactive analysis, and jobs with unpredictable demand where speed and simplicity are the priority.

            See our decision guide .

            The Premium tier is designed for AI/ML and comes with pre-configured ML Runtimes that have common libraries like PyTorch, XGBoost, and scikit-learn built-in. This eliminates complex setup and allows you to get started with your data science workloads in minutes.

            Learn about GPU workloads and runtimes .

            For maximum performance, you can select the Premium tier, which is powered by Lightning Engine. Pricing is based on a "pay-for-what-you-use" model, where you are billed per second only for the duration of your job's execution. This is highly cost-effective as it eliminates the cost of idle clusters.

            View detailed pricing .
            Google Cloud
            send_spark
              Docs Support
              Console
              Sign in
              • Accelerate your digital transformation
              • Whether your business is early in its journey or well on its way to digital transformation, Google Cloud can help solve your toughest challenges.
              • Featured Products
              • Business Intelligence
              • Hybrid and Multicloud
              • Industry Specific
              • Media Services
              • Mixed Reality
              • Productivity and Collaboration
              • Save money with our transparent approach to pricing
              • Google Cloud's pay-as-you-go pricing offers automatic savings based on monthly usage and discounted rates for prepaid resources. Contact us today to get a quote.
              Google Cloud
              Create a Mobile Website
              View Site in Mobile | Classic
              Share by: