Manages Hive table metadata. As a default, uses the localmariadb(image versions < 1.5) ormysql(image versions 1.5+) database
on the master node as the Hive table metadata store.
Using the default database is not recommended because these databases
are tied to the cluster's lifecycle. Instead, use either of the following as
the Hive metastore database (in recommendation order):
A quorum of journal nodes maintains an edit log of HDFS namespace modifications.
If a failover occurs, the Standby NameNode reads the edit log
and takes control from the Active NameNode.
Manages Hive table metadata. As a default, uses the localmariadb(image versions < 1.5) ormysql(image versions 1.5+) database
on the master node as the Hive table metadata store.
Using the default database is not recommended because these databases
are tied to the cluster's lifecycle. Instead, use either of the following as
the Hive metastore database (in recommendation order):
ZKFC is theZKFailoverControllerprocess, which runs
with the HDFS NameNode. It monitors the health of the NameNode, and manages leader
election via ZooKeeper in the event of a failover.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[[["\u003cp\u003eThis document details the services that run on Dataproc cluster nodes, categorized by node type and cluster configuration (standard or high availability).\u003c/p\u003e\n"],["\u003cp\u003eAll Dataproc cluster nodes run the \u003ccode\u003egoogle-dataproc-agent\u003c/code\u003e for job management and \u003ccode\u003egoogle-fluentd\u003c/code\u003e for log collection and pushing to Logging.\u003c/p\u003e\n"],["\u003cp\u003eStandard Dataproc clusters run services like \u003ccode\u003ehadoop-hdfs-namenode\u003c/code\u003e, \u003ccode\u003ehadoop-yarn-resourcemanager\u003c/code\u003e, \u003ccode\u003ehive-metastore\u003c/code\u003e, and \u003ccode\u003espark-history-server\u003c/code\u003e on all nodes, while \u003ccode\u003ehadoop-yarn-nodemanager\u003c/code\u003e runs on all worker nodes, and \u003ccode\u003ehadoop-hdfs-datanode\u003c/code\u003e runs only on primary workers.\u003c/p\u003e\n"],["\u003cp\u003eIn high availability (HA) Dataproc clusters, services are distributed across master nodes, with all masters running \u003ccode\u003ehadoop-hdfs-journalnode\u003c/code\u003e, \u003ccode\u003ehadoop-yarn-resourcemanager\u003c/code\u003e, and \u003ccode\u003ezookeeper-server\u003c/code\u003e, while specific services like \u003ccode\u003ehadoop-hdfs-namenode\u003c/code\u003e run on only master 0 and 1.\u003c/p\u003e\n"],["\u003cp\u003eThe default database for Hive metastore is either \u003ccode\u003emariadb\u003c/code\u003e for Dataproc image versions below 1.5, or \u003ccode\u003emysql\u003c/code\u003e for versions 1.5 and above, however, it is recommended to use Dataproc Metastore or Cloud SQL.\u003c/p\u003e\n"]]],[],null,[]]
