Stay organized with collectionsSave and categorize content based on your preferences.
Before you create a file system transfer job, you'll need to enable required
APIs and install Docker.
If you receive errors while performing first-time setup, confirm that the user
you logged in with has permissions required to perform the set up steps. In many
cases, these permissions are not available to all users, and you may need to
contact a project administrator for assistance.
System requirements
To use Storage Transfer Service for file system transfers, you need:
TCP port 443 (HTTPS) open for outbound connections.
The transfer agent machine requirements are listed below:
ADocker-supported 64-bit Linux
server or virtual machine that can access the data you plan to transfer.Docker Community Edition,
supports CentOs, Debian, Fedora, and Ubuntu operating systems.
At least 8 GB of memory per container and at least four CPUs to be most
effective.
Any system-level or user-level constraints on the number of allowed
process IDs, such as configured withulimit -u, must account for the
number of agents you plan to run. Each agent process can create up to many
hundreds of threads. We recommend removing process ID limits when running
transfer agents.
Customers usingPodmanto create agent containers
should specify a--pids-limitvalue high enough to accommodate Storage Transfer Service agent
threads. The default--pids-limitvalue is 2048, which may cause issues
when running more than one agent. We recommend setting--pids-limit=-1to allow unlimited process IDs per container.
Scaling restrictions on jobs and agents
Storage Transfer Service has the following scale restrictions on transfer jobs and
agents:
To allow the Docker container to authenticate with yourgclouddefault
credentials, create a Docker volume containing a file with your application
default credentials by running the following command:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[],[],null,["# Requirements for file system transfers\n\nBefore you create a file system transfer job, you'll need to enable required\nAPIs and install Docker.\n\nIf you receive errors while performing first-time setup, confirm that the user\nyou logged in with has permissions required to perform the set up steps. In many\ncases, these permissions are not available to all users, and you may need to\ncontact a project administrator for assistance.\n\nSystem requirements\n-------------------\n\nTo use Storage Transfer Service for file system transfers, you need:\n\n- A POSIX-compliant source and/or destination.\n\n- A Cloud Storage bucket without a\n [retention policy](/storage/docs/bucket-lock). To transfer to a bucket with\n a retention policy, see\n [Transfer to a bucket with a retention policy](/storage-transfer/docs/on-prem-agent-details#retention_policy).\n\n- TCP port 443 (HTTPS) open for outbound connections.\n\nThe transfer agent machine requirements are listed below:\n\n- A [Docker](https://docker.com)-supported 64-bit Linux\n server or virtual machine that can access the data you plan to transfer.\n [Docker Community Edition](https://docs.docker.com/install/linux/docker-ce/binaries/),\n supports CentOs, Debian, Fedora, and Ubuntu operating systems.\n\n To use other Linux operating systems, see\n [Docker Enterprise](https://docs.docker.com/ee/supported-platforms/).\n- At least 8 GB of memory per container and at least four CPUs to be most\n effective.\n\n- Any system-level or user-level constraints on the number of allowed\n process IDs, such as configured with `ulimit -u`, must account for the\n number of agents you plan to run. Each agent process can create up to many\n hundreds of threads. We recommend removing process ID limits when running\n transfer agents.\n\n Customers using [Podman](https://podman.io/) to create agent containers\n should specify a\n `--pids-limit` value high enough to accommodate Storage Transfer Service agent\n threads. The default `--pids-limit` value is 2048, which may cause issues\n when running more than one agent. We recommend setting `--pids-limit=-1`\n to allow unlimited process IDs per container.\n\nScaling restrictions on jobs and agents\n---------------------------------------\n\nStorage Transfer Service has the following scale restrictions on transfer jobs and\nagents:\n\n- Fewer than one billion files per job\n- Bandwidth cap must be over 1MBps\n- Up to 100 agents per agent pool\n- Up to 800 agent pools per project\n\nEnable APIs\n-----------\n\n1.\n\n\n Enable the Google Storage Transfer API.\n\n\n [Enable the API](https://console.cloud.google.com/flows/enableapi?apiid=storagetransfer)\n\nInstall Docker\n--------------\n\nIf you're using Google Cloud CLI or Docker to\n[install and run transfer agents](/storage-transfer/docs/managing-on-prem-agents#install-agent),\ninstall Docker Engine on a physical or virtual Linux machine.\n\nIf you're using Podman to install and run transfer agents, skip directly to\nthe\n[Podman-specific installation instructions](/storage-transfer/docs/managing-on-prem-agents#podman). \n\n curl -fsSL https://get.docker.com -o get-docker.sh\n\n sudo sh get-docker.sh\n\n sudo systemctl enable docker --now\n\nIf you encounter an installation error, see\n[Troubleshooting](https://docs.docker.com/install/linux/linux-postinstall/#troubleshooting).\n\nAllow Docker to authenticate\n----------------------------\n\nTo allow the Docker container to authenticate with your `gcloud` default\ncredentials, create a Docker volume containing a file with your application\ndefault credentials by running the following command: \n\n sudo docker run -ti --name gcloud-config google/cloud-sdk gcloud auth application-default login\n\nWhat's next?\n------------\n\n- [Configure Google Cloud permissions](/storage-transfer/docs/file-system-permissions)\n- [Create an agent pool](/storage-transfer/docs/on-prem-agent-pools)\n- [Install transfer agents](/storage-transfer/docs/managing-on-prem-agents)\n- [Create a transfer job](/storage-transfer/docs/create-transfers)\n- [Maximize transfer agent performance](/storage-transfer/docs/on-prem-agent-best-practices)"]]
