Stackdriver Logging
The Stackdriver Logging service collects and stores logs from applications and services on the Google Cloud Platform, giving you fine-grained, programmatic control over your projects' logs. You can use the Stackdriver Logging API to:
- Read and filter log entries
- Export your log entries to Cloud Storage, BigQuery, or Cloud Pub/Sub
- Create logs-based metrics for use in Cloud Monitoring
- Write log entries
For general information about Stackdriver Logging, read Stackdriver Logging Documentation .
The goal of google-cloud is to provide an API that is comfortable to Rubyists. Your authentication credentials are detected automatically in Google Cloud Platform (GCP), including Google Compute Engine (GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud Functions (GCF) and Cloud Run. In other environments you can configure authentication easily, either directly in your code or via environment variables. Read more about the options for connecting in the Authentication Guide .
Listing log entries
Stackdriver Logging gathers log entries from many services, including Google App Engine and Google Compute Engine. (See the List of Log Types .) In addition, you can write your own log entries to the service.
Project#entries returns the Entry records belonging to your project:
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entries = logging . entries entries . each do | e | puts "[ #{ e . timestamp } ] #{ e . log_name } #{ e . payload . inspect } " end
You can narrow the results to a single log using an advanced logs
filter
. A log is a
named collection of entries. Logs can be produced by Google Cloud Platform
services, by third-party services, or by your applications. For example, the log compute.googleapis.com/activity_log
is produced by Google Compute Engine. Logs
are simply referenced by name in google-cloud. There is no Log
type in
google-cloud or Log
resource in the Stackdriver Logging API.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entries = logging . entries filter : "logName:syslog" entries . each do | e | puts "[ #{ e . timestamp } ] #{ e . payload . inspect } " end
You can also order the log entries by timestamp
.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entries = logging . entries order : "timestamp desc" entries . each do | e | puts "[ #{ e . timestamp } ] #{ e . log_name } " end
Exporting log entries
Stackdriver Logging lets you export log entries to destinations including Google Cloud Storage buckets (for long term log storage), Google BigQuery datasets (for log analysis), and Google Pub/Sub (for streaming to other applications).
Creating sinks
A Sink is an object that lets you to specify a set of log entries to export.
In addition to the name of the sink and the export destination, Project#create_sink accepts an advanced logs filter to narrow the collection.
Before creating the sink, ensure that you have granted cloud-logs@google.com
permission to write logs to the destination. See Exporting Logs
(V2)
.
require "google/cloud/storage" require "google/cloud/logging" storage = Google :: Cloud :: Storage . new bucket = storage . create_bucket "my-logs-bucket" # Grant owner permission to Stackdriver Logging service email = "cloud-logs@google.com" bucket . acl . add_owner "group- #{ email } " logging = Google :: Cloud :: Logging . new sink = logging . create_sink "my-sink" , "storage.googleapis.com/ #{ bucket . id } "
When you create a sink, only new log entries are exported. Stackdriver Logging does not send previously-ingested log entries to the sink's destination.
Listing sinks
You can also list the sinks belonging to your project with Project#sinks .
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new sinks = logging . sinks sinks . each do | s | puts " #{ s . name } : #{ s . filter } -> #{ s . destination } " end
Creating logs-based metrics
You can use log entries in your project as the basis for Google Cloud Monitoring metrics. These metrics can then be used to produce Cloud Monitoring reports and alerts.
Creating metrics
A metric is a measured value that can be used to assess a system. Use Project#create_metric to configure a Metric based on a collection of log entries matching an advanced logs filter .
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new metric = logging . create_metric "errors" , "severity>=ERROR"
Listing metrics
You can also list the metrics belonging to your project with Project#metrics .
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new metrics = logging . metrics metrics . each do | m | puts " #{ m . name } : #{ m . filter } " end
Writing log entries
An Google::Cloud::Logging::Entry is composed of metadata and a payload. The payload is traditionally a message string, but in Stackdriver Logging it can also be a JSON or protocol buffer object. A single log can have entries with different payload types. In addition to the payload, your argument(s) to Project#write_entries must also contain a log name and a resource.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entry = logging . entry entry . payload = "Job started." entry . log_name = "my_app_log" entry . resource . type = "gae_app" entry . resource . labels [ :module_id ] = "1" entry . resource . labels [ :version_id ] = "20150925t173233" logging . write_entries entry
To write a JSON payload to the log, simply pass a hash argument:
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entry = logging . entry entry . payload = { "stats" = > { "a" = > 8 , "b" = > 12 . 5 } } entry . log_name = "my_app_log" entry . resource . type = "gae_app" entry . resource . labels [ :module_id ] = "1" entry . resource . labels [ :version_id ] = "20150925t173233" logging . write_entries entry
If you write a collection of log entries, you can provide the log name, resource, and/or labels hash to be used for all of the entries, and omit these values from the individual entries.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new entry1 = logging . entry entry1 . payload = "Job started." entry2 = logging . entry entry2 . payload = "Job completed." labels = { job_size : "large" , job_code : "red" } resource = logging . resource "gae_app" , "module_id" = > "1" , "version_id" = > "20150925t173233" logging . write_entries [ entry1 , entry2 ] , log_name : "my_app_log" , resource : resource , labels : labels
Normally, writing log entries is done synchronously; the call to Project#write_entries will block until it has either completed transmitting the data or encountered an error. To "fire and forget" without blocking, use AsyncWriter ; it spins up a background thread that writes log entries in batches. Calls to AsyncWriter#write_entries simply add entries to its work queue and return immediately.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new async = logging . async_writer entry1 = logging . entry entry1 . payload = "Job started." entry2 = logging . entry entry2 . payload = "Job completed." labels = { job_size : "large" , job_code : "red" } resource = logging . resource "gae_app" , "module_id" = > "1" , "version_id" = > "20150925t173233" async . write_entries [ entry1 , entry2 ] , log_name : "my_app_log" , resource : resource , labels : labels , partial_success : true
Creating a Ruby Logger implementation
If your environment requires a logger instance that is API-compatible with Ruby's standard library Logger , you can use Project#logger to create one.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new resource = logging . resource "gae_app" , module_id : "1" , version_id : "20150925t173233" logger = logging . logger "my_app_log" , resource , env : :production logger . info "Job started."
By default, the logger instance writes log entries asynchronously in a background thread using an AsyncWriter . If you want to customize or disable asynchronous writing, you may call the Logger constructor directly.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new resource = logging . resource "gae_app" , module_id : "1" , version_id : "20150925t173233" logger = Google :: Cloud :: Logging :: Logger . new logging , "my_app_log" , resource , { env : :production } logger . info "Log entry written synchronously."
Configuring timeout
You can configure the request timeout
value in seconds.
require "google/cloud/logging" logging = Google :: Cloud :: Logging . new timeout : 120
Additional information
Stackdriver Logging can be configured to be used in Rack applications or to use gRPC's logging. To learn more, see the Instrumentation Guide and Logging guide .
