Supported storage services for Cluster DirectorStay organized with collectionsSave and categorize content based on your preferences.
This document provides a conceptual overview of the Google Cloud storage
services supported for Cluster Director.
High-performance storage is a critical component for large-scale artificial
intelligence (AI) and high performance computing (HPC) workloads. The storage
services that you use with Cluster Director handle everything from
preparing and loading training data to saving model checkpoints and managing
shared user environments.
Shared file system requirements for Slurm
When you deploy a cluster with Slurm as the orchestrator, a shared file system
is required for the/homedirectory on all controller and login nodes. To meet
this requirement, you can configure Cluster Director to create a new
Filestore or Google Cloud Managed Lustre instance, or use an existing
instance. This configuration provides a shared space for user files and
configurations.
Supported storage services
In addition to a mandatory Filestore or
Managed Lustre instance for the/homedirectory, you can attach
other storage solutions to your cluster to meet the specific needs of your
workloads. Cluster Director supports the following storage services:
Overview: Filestore is a fully
managed, high-performance NFS file storage service. It provides a familiar
file system interface and is suitable for a wide range of use cases. A
Filestore instance is the default option for home
directories in Cluster Director clusters.
Overview: Managed Lustre is a
high-performance, fully managed parallel file system optimized for AI and
HPC applications. With its ultra-low latency and full POSIX support, it's
ideal for migrating on-premises AI workloads to Google Cloud.
Overview: Cloud Storage is a highly
scalable, durable, and cost-effective object store. Through integration
with Cloud Storage FUSE, Cloud Storage buckets can be mounted as local
file systems, making it suitable for storing vast datasets for training
and model checkpoints.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-04-10 UTC."],[],[]]
