Package dataflow is an auto-generated package for the Dataflow API.
Manages Google Cloud Dataflow projects on Google Cloud Platform.
NOTE : This package is in beta . It is not stable , and may be subject to changes .
General documentation
For information that is relevant for all client libraries please reference https://pkg.go.dev/cloud.google.com/go#pkg-overview . Some information on this page includes:
- Authentication and Authorization
- Timeouts and Cancellation
- Testing against Client Libraries
- Debugging Client Libraries
- Inspecting errors
Example usage
To get started with this package, create a client.
// go get cloud.google.com/go/dataflow/apiv1beta3@latest ctx := context . Background () // This snippet has been automatically generated and should be regarded as a code template only. // It will require modifications to work: // - It may require correct/in-range values for request initialization. // - It may require specifying regional endpoints when creating the service client as shown in: // https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options c , err := dataflow . NewSnapshotsV1Beta3Client ( ctx ) if err != nil { // TODO: Handle error. } defer c . Close ()
The client will use your default application credentials. Clients should be reused instead of created as needed. The methods of Client are safe for concurrent use by multiple goroutines. The returned client must be Closed when it is done being used.
Using the Client
The following is an example of making an API call with the newly created client, mentioned above.
req := & dataflowpb . DeleteSnapshotRequest { // TODO: Fill request struct fields. // See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#DeleteSnapshotRequest. } resp , err := c . DeleteSnapshot ( ctx , req ) if err != nil { // TODO: Handle error. } // TODO: Use resp. _ = resp
Use of Context
The ctx passed to NewSnapshotsV1Beta3Client is used for authentication requests and for creating the underlying connection, but is not used for subsequent calls. Individual methods on the client use the ctx given to them.
To close the open connection, use the Close() method.
Functions
func DefaultAuthScopes
func
DefaultAuthScopes
()
[]
string
DefaultAuthScopes reports the default set of authentication scopes to use with this package.
FlexTemplatesCallOptions
type
FlexTemplatesCallOptions
struct
{
LaunchFlexTemplate
[]
gax
.
CallOption
}
FlexTemplatesCallOptions contains the retry settings for each method of FlexTemplatesClient.
FlexTemplatesClient
type
FlexTemplatesClient
struct
{
// The call options for this service.
CallOptions
*
FlexTemplatesCallOptions
// contains filtered or unexported fields
}
FlexTemplatesClient is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
Provides a service for Flex templates. This feature is not ready yet.
func NewFlexTemplatesClient
func
NewFlexTemplatesClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
FlexTemplatesClient
,
error
)
NewFlexTemplatesClient creates a new flex templates service client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
Provides a service for Flex templates. This feature is not ready yet.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewFlexTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewFlexTemplatesRESTClient
func
NewFlexTemplatesRESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
FlexTemplatesClient
,
error
)
NewFlexTemplatesRESTClient creates a new flex templates service rest client.
Provides a service for Flex templates. This feature is not ready yet.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewFlexTemplatesRESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*FlexTemplatesClient) Close
func
(
c
*
FlexTemplatesClient
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*FlexTemplatesClient) Connection (deprecated)
func
(
c
*
FlexTemplatesClient
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*FlexTemplatesClient) LaunchFlexTemplate
func
(
c
*
FlexTemplatesClient
)
LaunchFlexTemplate
(
ctx
context
.
Context
,
req
*
dataflowpb
.
LaunchFlexTemplateRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
LaunchFlexTemplateResponse
,
error
)
LaunchFlexTemplate launch a job with a FlexTemplate.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewFlexTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
LaunchFlexTemplateRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#LaunchFlexTemplateRequest.
}
resp
,
err
:=
c
.
LaunchFlexTemplate
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
JobIterator
type
JobIterator
struct
{
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response
interface
{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch
func
(
pageSize
int
,
pageToken
string
)
(
results
[]
*
dataflowpb
.
Job
,
nextPageToken
string
,
err
error
)
// contains filtered or unexported fields
}
JobIterator manages a stream of *dataflowpb.Job.
func (*JobIterator) All
func
(
it
*
JobIterator
)
All
()
iter
.
Seq2
[
*
dataflowpb
.
Job
,
error
]
All returns an iterator. If an error is returned by the iterator, the iterator will stop after that iteration.
func (*JobIterator) Next
func
(
it
*
JobIterator
)
Next
()
(
*
dataflowpb
.
Job
,
error
)
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.
func (*JobIterator) PageInfo
func
(
it
*
JobIterator
)
PageInfo
()
*
iterator
.
PageInfo
PageInfo supports pagination. See the [google.golang.org/api/iterator] package for details.
JobMessageIterator
type
JobMessageIterator
struct
{
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response
interface
{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch
func
(
pageSize
int
,
pageToken
string
)
(
results
[]
*
dataflowpb
.
JobMessage
,
nextPageToken
string
,
err
error
)
// contains filtered or unexported fields
}
JobMessageIterator manages a stream of *dataflowpb.JobMessage.
func (*JobMessageIterator) All
func
(
it
*
JobMessageIterator
)
All
()
iter
.
Seq2
[
*
dataflowpb
.
JobMessage
,
error
]
All returns an iterator. If an error is returned by the iterator, the iterator will stop after that iteration.
func (*JobMessageIterator) Next
func
(
it
*
JobMessageIterator
)
Next
()
(
*
dataflowpb
.
JobMessage
,
error
)
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.
func (*JobMessageIterator) PageInfo
func
(
it
*
JobMessageIterator
)
PageInfo
()
*
iterator
.
PageInfo
PageInfo supports pagination. See the [google.golang.org/api/iterator] package for details.
JobsV1Beta3CallOptions
type
JobsV1Beta3CallOptions
struct
{
CreateJob
[]
gax
.
CallOption
GetJob
[]
gax
.
CallOption
UpdateJob
[]
gax
.
CallOption
ListJobs
[]
gax
.
CallOption
AggregatedListJobs
[]
gax
.
CallOption
CheckActiveJobs
[]
gax
.
CallOption
SnapshotJob
[]
gax
.
CallOption
}
JobsV1Beta3CallOptions contains the retry settings for each method of JobsV1Beta3Client.
JobsV1Beta3Client
type
JobsV1Beta3Client
struct
{
// The call options for this service.
CallOptions
*
JobsV1Beta3CallOptions
// contains filtered or unexported fields
}
JobsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.
func NewJobsV1Beta3Client
func
NewJobsV1Beta3Client
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
JobsV1Beta3Client
,
error
)
NewJobsV1Beta3Client creates a new jobs v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewJobsV1Beta3RESTClient
func
NewJobsV1Beta3RESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
JobsV1Beta3Client
,
error
)
NewJobsV1Beta3RESTClient creates a new jobs v1 beta3 rest client.
Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3RESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*JobsV1Beta3Client) AggregatedListJobs
func
(
c
*
JobsV1Beta3Client
)
AggregatedListJobs
(
ctx
context
.
Context
,
req
*
dataflowpb
.
ListJobsRequest
,
opts
...
gax
.
CallOption
)
*
JobIterator
AggregatedListJobs list the jobs of a project across all regions.
Examples
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
"google.golang.org/api/iterator"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobsRequest.
}
it
:=
c
.
AggregatedListJobs
(
ctx
,
req
)
for
{
resp
,
err
:=
it
.
Next
()
if
err
==
iterator
.
Done
{
break
}
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
// If you need to access the underlying RPC response,
// you can do so by casting the `Response` as below.
// Otherwise, remove this line. Only populated after
// first call to Next(). Not safe for concurrent access.
_
=
it
.
Response
.(
*
dataflowpb
.
ListJobsResponse
)
}
}
all
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobsRequest.
}
for
resp
,
err
:=
range
c
.
AggregatedListJobs
(
ctx
,
req
).
All
()
{
if
err
!=
nil
{
// TODO: Handle error and break/return/continue. Iteration will stop after any error.
}
// TODO: Use resp.
_
=
resp
}
}
func (*JobsV1Beta3Client) CheckActiveJobs
func
(
c
*
JobsV1Beta3Client
)
CheckActiveJobs
(
ctx
context
.
Context
,
req
*
dataflowpb
.
CheckActiveJobsRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
CheckActiveJobsResponse
,
error
)
CheckActiveJobs check for existence of active jobs in the given project across all regions.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
CheckActiveJobsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#CheckActiveJobsRequest.
}
resp
,
err
:=
c
.
CheckActiveJobs
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*JobsV1Beta3Client) Close
func
(
c
*
JobsV1Beta3Client
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*JobsV1Beta3Client) Connection (deprecated)
func
(
c
*
JobsV1Beta3Client
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*JobsV1Beta3Client) CreateJob
func
(
c
*
JobsV1Beta3Client
)
CreateJob
(
ctx
context
.
Context
,
req
*
dataflowpb
.
CreateJobRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Job
,
error
)
CreateJob creates a Cloud Dataflow job.
To create a job, we recommend using projects.locations.jobs.create with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). Using projects.jobs.create is not recommended, as your job will always start in us-central1.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
CreateJobRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#CreateJobRequest.
}
resp
,
err
:=
c
.
CreateJob
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*JobsV1Beta3Client) GetJob
func
(
c
*
JobsV1Beta3Client
)
GetJob
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetJobRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Job
,
error
)
GetJob gets the state of the specified Cloud Dataflow job.
To get the state of a job, we recommend using projects.locations.jobs.get with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). Using projects.jobs.get is not recommended, as you can only get the state of jobs that are running in us-central1.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetJobRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetJobRequest.
}
resp
,
err
:=
c
.
GetJob
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*JobsV1Beta3Client) ListJobs
func
(
c
*
JobsV1Beta3Client
)
ListJobs
(
ctx
context
.
Context
,
req
*
dataflowpb
.
ListJobsRequest
,
opts
...
gax
.
CallOption
)
*
JobIterator
ListJobs list the jobs of a project.
To list the jobs of a project in a region, we recommend using projects.locations.jobs.list with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). To list the all jobs across all regions, use projects.jobs.aggregated. Using projects.jobs.list is not recommended, as you can only get the list of jobs that are running in us-central1.
Examples
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
"google.golang.org/api/iterator"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobsRequest.
}
it
:=
c
.
ListJobs
(
ctx
,
req
)
for
{
resp
,
err
:=
it
.
Next
()
if
err
==
iterator
.
Done
{
break
}
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
// If you need to access the underlying RPC response,
// you can do so by casting the `Response` as below.
// Otherwise, remove this line. Only populated after
// first call to Next(). Not safe for concurrent access.
_
=
it
.
Response
.(
*
dataflowpb
.
ListJobsResponse
)
}
}
all
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobsRequest.
}
for
resp
,
err
:=
range
c
.
ListJobs
(
ctx
,
req
).
All
()
{
if
err
!=
nil
{
// TODO: Handle error and break/return/continue. Iteration will stop after any error.
}
// TODO: Use resp.
_
=
resp
}
}
func (*JobsV1Beta3Client) SnapshotJob
func
(
c
*
JobsV1Beta3Client
)
SnapshotJob
(
ctx
context
.
Context
,
req
*
dataflowpb
.
SnapshotJobRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Snapshot
,
error
)
SnapshotJob snapshot the state of a streaming job.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
SnapshotJobRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#SnapshotJobRequest.
}
resp
,
err
:=
c
.
SnapshotJob
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*JobsV1Beta3Client) UpdateJob
func
(
c
*
JobsV1Beta3Client
)
UpdateJob
(
ctx
context
.
Context
,
req
*
dataflowpb
.
UpdateJobRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Job
,
error
)
UpdateJob updates the state of an existing Cloud Dataflow job.
To update the state of an existing job, we recommend using projects.locations.jobs.update with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). Using projects.jobs.update is not recommended, as you can only update the state of jobs that are running in us-central1.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewJobsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
UpdateJobRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#UpdateJobRequest.
}
resp
,
err
:=
c
.
UpdateJob
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
MessagesV1Beta3CallOptions
type
MessagesV1Beta3CallOptions
struct
{
ListJobMessages
[]
gax
.
CallOption
}
MessagesV1Beta3CallOptions contains the retry settings for each method of MessagesV1Beta3Client.
MessagesV1Beta3Client
type
MessagesV1Beta3Client
struct
{
// The call options for this service.
CallOptions
*
MessagesV1Beta3CallOptions
// contains filtered or unexported fields
}
MessagesV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.
func NewMessagesV1Beta3Client
func
NewMessagesV1Beta3Client
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
MessagesV1Beta3Client
,
error
)
NewMessagesV1Beta3Client creates a new messages v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMessagesV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewMessagesV1Beta3RESTClient
func
NewMessagesV1Beta3RESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
MessagesV1Beta3Client
,
error
)
NewMessagesV1Beta3RESTClient creates a new messages v1 beta3 rest client.
The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMessagesV1Beta3RESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*MessagesV1Beta3Client) Close
func
(
c
*
MessagesV1Beta3Client
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*MessagesV1Beta3Client) Connection (deprecated)
func
(
c
*
MessagesV1Beta3Client
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*MessagesV1Beta3Client) ListJobMessages
func
(
c
*
MessagesV1Beta3Client
)
ListJobMessages
(
ctx
context
.
Context
,
req
*
dataflowpb
.
ListJobMessagesRequest
,
opts
...
gax
.
CallOption
)
*
JobMessageIterator
ListJobMessages request the job status.
To request the status of a job, we recommend using projects.locations.jobs.messages.list with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). Using projects.jobs.messages.list is not recommended, as you can only request the status of jobs that are running in us-central1.
Examples
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
"google.golang.org/api/iterator"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMessagesV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobMessagesRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobMessagesRequest.
}
it
:=
c
.
ListJobMessages
(
ctx
,
req
)
for
{
resp
,
err
:=
it
.
Next
()
if
err
==
iterator
.
Done
{
break
}
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
// If you need to access the underlying RPC response,
// you can do so by casting the `Response` as below.
// Otherwise, remove this line. Only populated after
// first call to Next(). Not safe for concurrent access.
_
=
it
.
Response
.(
*
dataflowpb
.
ListJobMessagesResponse
)
}
}
all
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMessagesV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListJobMessagesRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListJobMessagesRequest.
}
for
resp
,
err
:=
range
c
.
ListJobMessages
(
ctx
,
req
).
All
()
{
if
err
!=
nil
{
// TODO: Handle error and break/return/continue. Iteration will stop after any error.
}
// TODO: Use resp.
_
=
resp
}
}
MetricsV1Beta3CallOptions
type
MetricsV1Beta3CallOptions
struct
{
GetJobMetrics
[]
gax
.
CallOption
GetJobExecutionDetails
[]
gax
.
CallOption
GetStageExecutionDetails
[]
gax
.
CallOption
}
MetricsV1Beta3CallOptions contains the retry settings for each method of MetricsV1Beta3Client.
MetricsV1Beta3Client
type
MetricsV1Beta3Client
struct
{
// The call options for this service.
CallOptions
*
MetricsV1Beta3CallOptions
// contains filtered or unexported fields
}
MetricsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.
func NewMetricsV1Beta3Client
func
NewMetricsV1Beta3Client
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
MetricsV1Beta3Client
,
error
)
NewMetricsV1Beta3Client creates a new metrics v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewMetricsV1Beta3RESTClient
func
NewMetricsV1Beta3RESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
MetricsV1Beta3Client
,
error
)
NewMetricsV1Beta3RESTClient creates a new metrics v1 beta3 rest client.
The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3RESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*MetricsV1Beta3Client) Close
func
(
c
*
MetricsV1Beta3Client
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*MetricsV1Beta3Client) Connection (deprecated)
func
(
c
*
MetricsV1Beta3Client
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*MetricsV1Beta3Client) GetJobExecutionDetails
func
(
c
*
MetricsV1Beta3Client
)
GetJobExecutionDetails
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetJobExecutionDetailsRequest
,
opts
...
gax
.
CallOption
)
*
StageSummaryIterator
GetJobExecutionDetails request detailed information about the execution status of the job.
EXPERIMENTAL. This API is subject to change or removal without notice.
Examples
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
"google.golang.org/api/iterator"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetJobExecutionDetailsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetJobExecutionDetailsRequest.
}
it
:=
c
.
GetJobExecutionDetails
(
ctx
,
req
)
for
{
resp
,
err
:=
it
.
Next
()
if
err
==
iterator
.
Done
{
break
}
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
// If you need to access the underlying RPC response,
// you can do so by casting the `Response` as below.
// Otherwise, remove this line. Only populated after
// first call to Next(). Not safe for concurrent access.
_
=
it
.
Response
.(
*
dataflowpb
.
JobExecutionDetails
)
}
}
all
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetJobExecutionDetailsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetJobExecutionDetailsRequest.
}
for
resp
,
err
:=
range
c
.
GetJobExecutionDetails
(
ctx
,
req
).
All
()
{
if
err
!=
nil
{
// TODO: Handle error and break/return/continue. Iteration will stop after any error.
}
// TODO: Use resp.
_
=
resp
}
}
func (*MetricsV1Beta3Client) GetJobMetrics
func
(
c
*
MetricsV1Beta3Client
)
GetJobMetrics
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetJobMetricsRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
JobMetrics
,
error
)
GetJobMetrics request the job status.
To request the status of a job, we recommend using projects.locations.jobs.getMetrics with a [regional endpoint] ( https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints )). Using projects.jobs.getMetrics is not recommended, as you can only request the status of jobs that are running in us-central1.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetJobMetricsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetJobMetricsRequest.
}
resp
,
err
:=
c
.
GetJobMetrics
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*MetricsV1Beta3Client) GetStageExecutionDetails
func
(
c
*
MetricsV1Beta3Client
)
GetStageExecutionDetails
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetStageExecutionDetailsRequest
,
opts
...
gax
.
CallOption
)
*
WorkerDetailsIterator
GetStageExecutionDetails request detailed information about the execution status of a stage of the job.
EXPERIMENTAL. This API is subject to change or removal without notice.
Examples
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
"google.golang.org/api/iterator"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetStageExecutionDetailsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetStageExecutionDetailsRequest.
}
it
:=
c
.
GetStageExecutionDetails
(
ctx
,
req
)
for
{
resp
,
err
:=
it
.
Next
()
if
err
==
iterator
.
Done
{
break
}
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
// If you need to access the underlying RPC response,
// you can do so by casting the `Response` as below.
// Otherwise, remove this line. Only populated after
// first call to Next(). Not safe for concurrent access.
_
=
it
.
Response
.(
*
dataflowpb
.
StageExecutionDetails
)
}
}
all
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewMetricsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetStageExecutionDetailsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetStageExecutionDetailsRequest.
}
for
resp
,
err
:=
range
c
.
GetStageExecutionDetails
(
ctx
,
req
).
All
()
{
if
err
!=
nil
{
// TODO: Handle error and break/return/continue. Iteration will stop after any error.
}
// TODO: Use resp.
_
=
resp
}
}
SnapshotsV1Beta3CallOptions
type
SnapshotsV1Beta3CallOptions
struct
{
GetSnapshot
[]
gax
.
CallOption
DeleteSnapshot
[]
gax
.
CallOption
ListSnapshots
[]
gax
.
CallOption
}
SnapshotsV1Beta3CallOptions contains the retry settings for each method of SnapshotsV1Beta3Client.
SnapshotsV1Beta3Client
type
SnapshotsV1Beta3Client
struct
{
// The call options for this service.
CallOptions
*
SnapshotsV1Beta3CallOptions
// contains filtered or unexported fields
}
SnapshotsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
Provides methods to manage snapshots of Google Cloud Dataflow jobs.
func NewSnapshotsV1Beta3Client
func
NewSnapshotsV1Beta3Client
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
SnapshotsV1Beta3Client
,
error
)
NewSnapshotsV1Beta3Client creates a new snapshots v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
Provides methods to manage snapshots of Google Cloud Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewSnapshotsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewSnapshotsV1Beta3RESTClient
func
NewSnapshotsV1Beta3RESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
SnapshotsV1Beta3Client
,
error
)
NewSnapshotsV1Beta3RESTClient creates a new snapshots v1 beta3 rest client.
Provides methods to manage snapshots of Google Cloud Dataflow jobs.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewSnapshotsV1Beta3RESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*SnapshotsV1Beta3Client) Close
func
(
c
*
SnapshotsV1Beta3Client
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*SnapshotsV1Beta3Client) Connection (deprecated)
func
(
c
*
SnapshotsV1Beta3Client
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*SnapshotsV1Beta3Client) DeleteSnapshot
func
(
c
*
SnapshotsV1Beta3Client
)
DeleteSnapshot
(
ctx
context
.
Context
,
req
*
dataflowpb
.
DeleteSnapshotRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
DeleteSnapshotResponse
,
error
)
DeleteSnapshot deletes a snapshot.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewSnapshotsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
DeleteSnapshotRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#DeleteSnapshotRequest.
}
resp
,
err
:=
c
.
DeleteSnapshot
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*SnapshotsV1Beta3Client) GetSnapshot
func
(
c
*
SnapshotsV1Beta3Client
)
GetSnapshot
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetSnapshotRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Snapshot
,
error
)
GetSnapshot gets information about a snapshot.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewSnapshotsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetSnapshotRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetSnapshotRequest.
}
resp
,
err
:=
c
.
GetSnapshot
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*SnapshotsV1Beta3Client) ListSnapshots
func
(
c
*
SnapshotsV1Beta3Client
)
ListSnapshots
(
ctx
context
.
Context
,
req
*
dataflowpb
.
ListSnapshotsRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
ListSnapshotsResponse
,
error
)
ListSnapshots lists snapshots.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewSnapshotsV1Beta3Client
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
ListSnapshotsRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#ListSnapshotsRequest.
}
resp
,
err
:=
c
.
ListSnapshots
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
StageSummaryIterator
type
StageSummaryIterator
struct
{
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response
interface
{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch
func
(
pageSize
int
,
pageToken
string
)
(
results
[]
*
dataflowpb
.
StageSummary
,
nextPageToken
string
,
err
error
)
// contains filtered or unexported fields
}
StageSummaryIterator manages a stream of *dataflowpb.StageSummary.
func (*StageSummaryIterator) All
func
(
it
*
StageSummaryIterator
)
All
()
iter
.
Seq2
[
*
dataflowpb
.
StageSummary
,
error
]
All returns an iterator. If an error is returned by the iterator, the iterator will stop after that iteration.
func (*StageSummaryIterator) Next
func
(
it
*
StageSummaryIterator
)
Next
()
(
*
dataflowpb
.
StageSummary
,
error
)
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.
func (*StageSummaryIterator) PageInfo
func
(
it
*
StageSummaryIterator
)
PageInfo
()
*
iterator
.
PageInfo
PageInfo supports pagination. See the [google.golang.org/api/iterator] package for details.
TemplatesCallOptions
type
TemplatesCallOptions
struct
{
CreateJobFromTemplate
[]
gax
.
CallOption
LaunchTemplate
[]
gax
.
CallOption
GetTemplate
[]
gax
.
CallOption
}
TemplatesCallOptions contains the retry settings for each method of TemplatesClient.
TemplatesClient
type
TemplatesClient
struct
{
// The call options for this service.
CallOptions
*
TemplatesCallOptions
// contains filtered or unexported fields
}
TemplatesClient is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.
Provides a method to create Cloud Dataflow jobs from templates.
func NewTemplatesClient
func
NewTemplatesClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
TemplatesClient
,
error
)
NewTemplatesClient creates a new templates service client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.
Provides a method to create Cloud Dataflow jobs from templates.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func NewTemplatesRESTClient
func
NewTemplatesRESTClient
(
ctx
context
.
Context
,
opts
...
option
.
ClientOption
)
(
*
TemplatesClient
,
error
)
NewTemplatesRESTClient creates a new templates service rest client.
Provides a method to create Cloud Dataflow jobs from templates.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewTemplatesRESTClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
// TODO: Use client.
_
=
c
}
func (*TemplatesClient) Close
func
(
c
*
TemplatesClient
)
Close
()
error
Close closes the connection to the API service. The user should invoke this when the client is no longer required.
func (*TemplatesClient) Connection (deprecated)
func
(
c
*
TemplatesClient
)
Connection
()
*
grpc
.
ClientConn
Connection returns a connection to the API service.
Deprecated: Connections are now pooled so this method does not always return the same resource.
func (*TemplatesClient) CreateJobFromTemplate
func
(
c
*
TemplatesClient
)
CreateJobFromTemplate
(
ctx
context
.
Context
,
req
*
dataflowpb
.
CreateJobFromTemplateRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
Job
,
error
)
CreateJobFromTemplate creates a Cloud Dataflow job from a template.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
CreateJobFromTemplateRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#CreateJobFromTemplateRequest.
}
resp
,
err
:=
c
.
CreateJobFromTemplate
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*TemplatesClient) GetTemplate
func
(
c
*
TemplatesClient
)
GetTemplate
(
ctx
context
.
Context
,
req
*
dataflowpb
.
GetTemplateRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
GetTemplateResponse
,
error
)
GetTemplate get the template associated with a template.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
GetTemplateRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#GetTemplateRequest.
}
resp
,
err
:=
c
.
GetTemplate
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
func (*TemplatesClient) LaunchTemplate
func
(
c
*
TemplatesClient
)
LaunchTemplate
(
ctx
context
.
Context
,
req
*
dataflowpb
.
LaunchTemplateRequest
,
opts
...
gax
.
CallOption
)
(
*
dataflowpb
.
LaunchTemplateResponse
,
error
)
LaunchTemplate launch a template.
Example
package
main
import
(
"context"
dataflow
"cloud.google.com/go/dataflow/apiv1beta3"
dataflowpb
"cloud.google.com/go/dataflow/apiv1beta3/dataflowpb"
)
func
main
()
{
ctx
:=
context
.
Background
()
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in:
// https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
c
,
err
:=
dataflow
.
NewTemplatesClient
(
ctx
)
if
err
!=
nil
{
// TODO: Handle error.
}
defer
c
.
Close
()
req
:=
& dataflowpb
.
LaunchTemplateRequest
{
// TODO: Fill request struct fields.
// See https://pkg.go.dev/cloud.google.com/go/dataflow/apiv1beta3/dataflowpb#LaunchTemplateRequest.
}
resp
,
err
:=
c
.
LaunchTemplate
(
ctx
,
req
)
if
err
!=
nil
{
// TODO: Handle error.
}
// TODO: Use resp.
_
=
resp
}
WorkerDetailsIterator
type
WorkerDetailsIterator
struct
{
// Response is the raw response for the current page.
// It must be cast to the RPC response type.
// Calling Next() or InternalFetch() updates this value.
Response
interface
{}
// InternalFetch is for use by the Google Cloud Libraries only.
// It is not part of the stable interface of this package.
//
// InternalFetch returns results from a single call to the underlying RPC.
// The number of results is no greater than pageSize.
// If there are no more results, nextPageToken is empty and err is nil.
InternalFetch
func
(
pageSize
int
,
pageToken
string
)
(
results
[]
*
dataflowpb
.
WorkerDetails
,
nextPageToken
string
,
err
error
)
// contains filtered or unexported fields
}
WorkerDetailsIterator manages a stream of *dataflowpb.WorkerDetails.
func (*WorkerDetailsIterator) All
func
(
it
*
WorkerDetailsIterator
)
All
()
iter
.
Seq2
[
*
dataflowpb
.
WorkerDetails
,
error
]
All returns an iterator. If an error is returned by the iterator, the iterator will stop after that iteration.
func (*WorkerDetailsIterator) Next
func
(
it
*
WorkerDetailsIterator
)
Next
()
(
*
dataflowpb
.
WorkerDetails
,
error
)
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.
func (*WorkerDetailsIterator) PageInfo
func
(
it
*
WorkerDetailsIterator
)
PageInfo
()
*
iterator
.
PageInfo
PageInfo supports pagination. See the [google.golang.org/api/iterator] package for details.
