Manage and monitor AlloyDB Omni

Single-server

By default, to retrieve AlloyDB Omni logs, run:

   
docker  
logs  
 CONTAINER_NAME 
 

   
podman  
logs  
 CONTAINER_NAME 
 

Kubernetes

Find your database cluster log files

postgresql.audit and postgresql.log files are located on the file system of the database pod. postgresql.audit is only present if you enabled pgaudit .

To access these files, follow these steps:

1. Define an environment variable containing the name of the database pod.

     export 
     
    DB_POD 
    = 
    ` 
   kubectl  
   get  
   pod  
   -l  
   alloydbomni.internal.dbadmin.goog/dbcluster = 
    DB_CLUSTER_NAME 
   ,alloydbomni.internal.dbadmin.goog/task-type = 
   database  
   -o  
    jsonpath 
    = 
    '{.items[0].metadata.name}' 
    ` 
    
   

   Replace DB_CLUSTER_NAME with the name of your database cluster. It's the same database cluster name you declared when you created it .

2. Run a shell on the database pod as root.

    kubectl  
    exec 
     
    ${ 
    DB_POD 
    } 
     
   -it  
   --  
   /bin/bash 
   

3. Find the log files in the /obs/diagnostic/ directory:

   • /obs/diagnostic/postgresql.audit
   • /obs/diagnostic/postgresql.log

List monitoring services

 kubectl  
get  
svc  
-n  
 NAMESPACE 
  
 | 
  
grep  
monitoring 

 al-1060-dbc-monitoring-system  
ClusterIP  
 10 
.0.15.227  
<none>  
 9187 
/TCP  
7d20h
al-3de6-dbc-monitoring-system  
ClusterIP  
 10 
.0.5.205  
<none>  
 9187 
/TCP  
7d19h
al-4bc0-dbc-monitoring-system  
ClusterIP  
 10 
.0.15.92  
<none>  
 9187 
/TCP  
7d19h 

 kubectl  
get  
svc  
-n  
 NAMESPACE 
  
 | 
  
grep  
monitoring 

 al-2953-dbcluster-foo7-monitoring-db  
ClusterIP  
 10 
.36.3.243  
<none>  
 9187 
/TCP  
44m
al-2953-dbcluster-foo7-monitoring-system  
ClusterIP  
 10 
.36.7.72  
<none>  
 9187 
/TCP  
44m 

View Prometheus metrics from the command line

The port 9187 is named as metricsalloydbomni for all monitoring services.

1. Set up port forwarding from your local environment to the monitoring service.

    kubectl  
   port-forward  
   service/ MONITORING_SERVICE 
     
   -n  
    NAMESPACE 
     
    MONITORING_METRICS_PORT 
   :metricsalloydbomni 
   

   Replace the following:

   • MONITORING_SERVICE : The name of the monitoring service that you want to forward—for example, al-1060-dbc-monitoring-system .

   • NAMESPACE : The namespace where your cluster belongs.

   • MONITORING_METRICS_PORT : A local available TCP port.

   The following response shows that the services are being forwarded.

    Forwarding  
   from  
    127 
   .0.0.1:9187  
   ->  
    9187 
   Forwarding  
   from  
    [ 
   ::1 ] 
   :9187  
   ->  
    9187 
    
   

2. While the previous command runs, you can access monitoring metrics through HTTP on the port that you specified. For example, you can use curl to see all of the metrics as plain text:

    curl  
   http://localhost: MONITORING_METRICS_PORT 
   /metrics 
   

View metrics using the Prometheus API

The alloydbomni.internal.dbadmin.goog/task-type label key and the metricsalloydbomni port is available as a default for all monitoring services in AlloyDB Omni. You can use them together with a single serviceMonitor custom resource to select all the services for all namespaces in your database cluster.

For more information about using the Prometheus API, see the Prometheus Operator documentation .

The following is an example spec field of the serviceMonitor custom resource that includes the alloydbomni.internal.dbadmin.gdc.goog/task-type label key and metricsalloydbomni port. The serviceMonitor custom resource monitors and collects all the kubernetes services in all namespaces

For more information about the complete ServiceMonitor definition, see the ServiceMonitor custom resource definition .

   
 spec 
 : 
  
 selector 
 : 
  
 matchLabels 
 : 
  
 alloydbomni.internal.dbadmin.goog/task-type 
 : 
  
 monitoring 
  
 namespaceSelector 
 : 
  
 any 
 : 
  
 true 
  
 endpoints 
 : 
  
 - 
  
 port 
 : 
  
 metricsalloydbomni 
 

   
 spec 
 : 
  
 selector 
 : 
  
 matchExpressions 
 : 
  
 - 
  
 key 
 : 
  
 alloydbomni.internal.dbadmin.gdc.goog/task-type 
  
 operator 
 : 
  
 Exists 
  
 values 
 : 
  
 [] 
  
 namespaceSelector 
 : 
  
 any 
 : 
  
 true 
  
 endpoints 
 : 
  
 - 
  
 port 
 : 
  
 metricsalloydbomni 
 

View metrics using Grafana

For a visual representation of metrics in AlloyDB Omni on Kubernetes, use the monitoring dashboard. The monitoring dashboard relies on a basic observability stack that consists of Prometheus and Grafana. To configure the monitoring dashboard to collect metrics from AlloyDB Omni, follow these steps:

1. To download the Grafana dashboard, use the wget command:

    wget  
   https://raw.githubusercontent.com/GoogleCloudPlatform/alloydb-omni-samples/refs/heads/main/monitoring-dashboards/grafana/alloydbomni_dashboard.yaml 
   

2. You must download and install the grafana-operator before deploying Grafana in Kubernetes. For detailed instructions, see Installation .

3. Add the monitoring.dashboard/product=alloydb-omni label to the Grafana instance where you install the dashboard:

    kubectl  
   label  
   grafana/ GRAFANA_INSTANCE_NAME 
     
   monitoring.dashboard/product = 
   alloydb-omni  
   -n  
    NAMESPACE 
    
   

   Replace the following:

   • GRAFANA_INSTANCE_NAME : the name of the Grafana instance where you put the dashboard.
   • NAMESPACE : the namespace where you deployed the Grafana operator.

4. To apply the Grafana dashboard configuration to your AlloyDB Omni cluster on Kubernetes, use the following command:

    kubectl  
   apply  
   -f  
   alloydbomni_dashboard.yaml  
   -n  
    NAMESPACE 
    
   

   See Grafana operator documentation for information about using the Grafana operator.

5. To configure Grafana to use Prometheus as a data source, see Datasources .

6. To verify Grafana is configured correctly, do one of the following:

   • View the collection of Grafana panels on the AlloyDB Omni dashboard.

   • Retrieve information about the Grafana dashboard in a Kubernetes cluster:

      kubectl  
     get  
     grafanadashboard  
     alloydb-omni-dashboard  
     -n  
      NAMESPACE 
       
     -o  
      jsonpath 
      = 
      '{.status.conditions[?(@.type=="DashboardSynchronized")].status}' 
      
     

     If the command returns True , the alloydb-omni-dashboard is successfully deployed to the Grafana instance.

Single-server

To uninstall AlloyDB Omni, stop and delete the AlloyDB Omni container using the following command:

   
docker  
container  
stop  
 CONTAINER_NAME 
  
docker  
container  
rm  
 CONTAINER_NAME 
 

   
podman  
container  
stop  
 CONTAINER_NAME 
  
podman  
container  
rm  
 CONTAINER_NAME 
 

   
podman  
container  
stop  
 CONTAINER_NAME 
  
podman  
container  
rm  
 CONTAINER_NAME 
 

You can move, archive, or delete an external data directory depending on whether and how you want to preserve your data after uninstalling AlloyDB Omni.

Kubernetes

Delete your database cluster

To delete your database cluster, set isDeleted to true in its manifest. You can accomplish this with the following command.

 kubectl  
patch  
dbclusters.alloydbomni.dbadmin.goog  
 DB_CLUSTER_NAME 
  
-p  
 '{"spec":{"isDeleted":true}}' 
  
--type = 
merge 

Replace DB_CLUSTER_NAME with the name of your database cluster. It's the same database cluster name you declared when you created it .

Uninstall the AlloyDB Omni operator

To uninstall the AlloyDB Omni Kubernetes operator from your Kubernetes cluster, take the following steps:

1. Delete all of your database clusters:

     for 
     
   ns  
    in 
     
    $( 
   kubectl  
   get  
   dbclusters.alloydbomni.dbadmin.goog  
   --all-namespaces  
   -o = 
    jsonpath 
    = 
    '{range .items[*]}{.metadata.namespace}{"\n"}{end}' 
    ) 
    ; 
     
    do 
    for 
     
   cr  
    in 
     
    $( 
   kubectl  
   get  
   dbclusters.alloydbomni.dbadmin.goog  
   -n  
    $ns 
     
   -o = 
    jsonpath 
    = 
    '{range .items[*]}{.metadata.name}{"\n"}{end}' 
    ) 
    ; 
     
    do 
   kubectl  
   patch  
   dbclusters.alloydbomni.dbadmin.goog  
    $cr 
     
   -n  
    $ns 
     
   --type = 
   merge  
   -p  
    '{"spec":{"isDeleted":true}}' 
    done 
    done 
    
   

2. Wait for the AlloyDB Omni Kubernetes operator to delete all of your database clusters. Use the following command to check whether any database resources remain:

    kubectl  
   get  
   dbclusters.alloydbomni.dbadmin.goog  
   --all-namespaces 
   

3. Delete other resources that the AlloyDB Omni Kubernetes operator created:

    kubectl  
   delete  
   failovers.alloydbomni.dbadmin.goog  
   --all  
   --all-namespaces 
    kubectl  
   delete  
   restores.alloydbomni.dbadmin.goog  
   --all  
   --all-namespaces 
    kubectl  
   delete  
   switchovers.alloydbomni.dbadmin.goog  
   --all  
   --all-namespaces 
   

4. Uninstall the AlloyDB Omni Kubernetes operator:

    helm  
   uninstall  
   alloydbomni-operator  
   --namespace  
   alloydb-omni-system 
   

5. Clean up secrets, custom resource descriptions, and namespaces related to the AlloyDB Omni Kubernetes operator:

    kubectl  
   delete  
   certificate  
   -n  
   alloydb-omni-system  
   --all 
    kubectl  
   get  
   secrets  
   --all-namespaces  
   -o  
   custom-columns = 
   NAMESPACE:.metadata.namespace,NAME:.metadata.name,ANNOTATION:.metadata.annotations.cert-manager \. 
   io/issuer-name  
    | 
     
   grep  
   -E  
    'alloydbomni|dbs-al' 
     
    | 
     
   awk  
    '{print $1 " " $2}' 
     
    | 
     
   xargs  
   -n  
    2 
     
   kubectl  
   delete  
   secret  
   -n 
    kubectl  
   delete  
   crd  
   -l  
   alloydb-omni = 
    true 
    
    kubectl  
   delete  
   ns  
   alloydb-omni-system