View and manage your data across Google Cloud Platform services.

View your Google Compute Engine resources.

View and manage your Google Compute Engine resources.

View your email address.

ApproximateProgress: Obsolete in favor of ApproximateReportedProgress and ApproximateSplitRequest.

ApproximateReportedProgress: A progress measurement of a WorkItem by a worker.

ApproximateSplitRequest: A suggestion by the service to the worker to dynamically split the WorkItem.

AutoscalingEvent: A structured message reporting an autoscaling decision made by the Dataflow service.

AutoscalingSettings: Settings for WorkerPool autoscaling.

BigQueryIODetails: Metadata for a BigQuery connector used by the job.

BigTableIODetails: Metadata for a BigTable connector used by the job.

ComponentSource: Description of an interstitial value between transforms in an execution stage.

ComponentTransform: Description of a transform executed as part of an execution stage.

ComputationTopology: All configuration data for a particular Computation.

ConcatPosition: A position that encapsulates an inner position and an index for the inner position.

CounterMetadata: CounterMetadata includes all static non-name non-value counter attributes.

CounterStructuredName: Identifies a counter within a per-job namespace.

CounterStructuredNameAndMetadata: A single message which encapsulates structured name and metadata for a given counter.

CounterUpdate: An update to a Counter sent from a worker.

CPUTime: Modeled after information exposed by /proc/stat.

CreateJobFromTemplateRequest: A request to create a Cloud Dataflow job from a template.

CustomSourceLocation: Identifies the location of a custom souce.

DataDiskAssignment: Data disk assignment for a given VM instance.

DatastoreIODetails: Metadata for a Datastore connector used by the job.

DerivedSource: Specification of one of the bundles produced as a result of splitting a Source (e.g.

Disk: Describes the data disk used by a workflow job.

DisplayData: Data provided with a pipeline or transform to provide descriptive info.

DistributionUpdate: A metric value representing a distribution.

DynamicSourceSplit: When a task splits using WorkItemStatus.dynamic_source_split, this message describes the two parts of the split relative to the description of the current task's input.

Environment: Describes the environment in which a Dataflow Job runs.

ExecutionStageState: A message describing the state of a particular execution stage.

ExecutionStageSummary: Description of the composing transforms, names/ids, and input/outputs of a stage of execution.

FailedLocation: Indicates which location failed to respond to a request for data.

FileIODetails: Metadata for a File connector used by the job.

FlattenInstruction: An instruction that copies its inputs (zero or more) to its (single) output.

FloatingPointList: A metric value representing a list of floating point numbers.

FloatingPointMean: A representation of a floating point mean metric contribution.

GetDebugConfigRequest: Request to get updated debug configuration for component.

GetDebugConfigResponse: Response to a get debug configuration request.

GetTemplateResponse: The response to a GetTemplate request.

Histogram: Histogram of value counts for a distribution.

InstructionInput: An input of an instruction, as a reference to an output of a producer instruction.

InstructionOutput: An output of an instruction.

IntegerGauge: A metric value representing temporal values of a variable.

IntegerList: A metric value representing a list of integers.

IntegerMean: A representation of an integer mean metric contribution.

Job: Defines a job to be run by the Cloud Dataflow service.

JobExecutionInfo: Additional information about how a Cloud Dataflow job will be executed that isn't contained in the submitted job.

JobExecutionStageInfo: Contains information about how a particular google.dataflow.v1beta3.Step will be executed.

JobMessage: A particular message pertaining to a Dataflow job.

JobMetadata: Metadata available primarily for filtering jobs.

JobMetrics: JobMetrics contains a collection of metrics descibing the detailed progress of a Dataflow job.

KeyRangeDataDiskAssignment: Data disk assignment information for a specific key-range of a sharded computation.

KeyRangeLocation: Location information for a specific key-range of a sharded computation.

LaunchTemplateParameters: Parameters to provide to the template being launched.

LaunchTemplateResponse: Response to the request to launch a template.

LeaseWorkItemRequest: Request to lease WorkItems.

LeaseWorkItemResponse: Response to a request to lease WorkItems.

ListJobMessagesResponse: Response to a request to list job messages.

ListJobsResponse: Response to a request to list Cloud Dataflow jobs.

MapTask: MapTask consists of an ordered set of instructions, each of which describes one particular low-level operation for the worker to perform in order to accomplish the MapTask's WorkItem.

MetricShortId: The metric short id is returned to the user alongside an offset into ReportWorkItemStatusRequest.

MetricStructuredName: Identifies a metric, by describing the source which generated the metric.

MetricUpdate: Describes the state of a metric.

MountedDataDisk: Describes mounted data disk.

MultiOutputInfo: Information about an output of a multi-output DoFn.

NameAndKind: Basic metadata about a counter.

Package: The packages that must be installed in order for a worker to run the steps of the Cloud Dataflow job that will be assigned to its worker pool.

ParallelInstruction: Describes a particular operation comprising a MapTask.

Parameter: Structured data associated with this message.

ParameterMetadata: Metadata for a specific parameter.

ParDoInstruction: An instruction that does a ParDo operation.

PartialGroupByKeyInstruction: An instruction that does a partial group-by-key.

PipelineDescription: A descriptive representation of submitted pipeline as well as the executed form.

Position: Position defines a position within a collection of data.

PubSubIODetails: Metadata for a PubSub connector used by the job.

PubsubLocation: Identifies a pubsub location to use for transferring data into or out of a streaming Dataflow job.

ReadInstruction: An instruction that reads records.

ReportedParallelism: Represents the level of parallelism in a WorkItem's input, reported by the worker.

ReportWorkItemStatusRequest: Request to report the status of WorkItems.

ReportWorkItemStatusResponse: Response from a request to report the status of WorkItems.

ResourceUtilizationReport: Worker metrics exported from workers.

ResourceUtilizationReportResponse: Service-side response to WorkerMessage reporting resource utilization.

RuntimeEnvironment: The environment values to set at runtime.

SdkVersion: The version of the SDK used to run the jobl.

SendDebugCaptureRequest: Request to send encoded debug information.

SendDebugCaptureResponse: Response to a send capture request.

SendWorkerMessagesRequest: A request for sending worker messages to the service.

SendWorkerMessagesResponse: The response to the worker messages.

SeqMapTask: Describes a particular function to invoke.

SeqMapTaskOutputInfo: Information about an output of a SeqMapTask.

ShellTask: A task which consists of a shell command for the worker to execute.

SideInputInfo: Information about a side input of a DoFn or an input of a SeqDoFn.

Sink: A sink that records can be encoded and written to.

Source: A source that records can be read and decoded from.

SourceFork: DEPRECATED in favor of DynamicSourceSplit.

SourceGetMetadataRequest: A request to compute the SourceMetadata of a Source.

SourceGetMetadataResponse: The result of a SourceGetMetadataOperation.

SourceMetadata: Metadata about a Source useful for automatically optimizing and tuning the pipeline, etc.

SourceOperationRequest: A work item that represents the different operations that can be performed on a user-defined Source specification.

SourceOperationResponse: The result of a SourceOperationRequest, specified in ReportWorkItemStatusRequest.source_operation when the work item is completed.

SourceSplitOptions: Hints for splitting a Source into bundles (parts for parallel processing) using SourceSplitRequest.

SourceSplitRequest: Represents the operation to split a high-level Source specification into bundles (parts for parallel processing).

SourceSplitResponse: The response to a SourceSplitRequest.

SourceSplitShard: DEPRECATED in favor of DerivedSource.

SpannerIODetails: Metadata for a Spanner connector used by the job.

SplitInt64: A representation of an int64, n, that is immune to precision loss when encoded in JSON.

StageSource: Description of an input or output of an execution stage.

StateFamilyConfig: State family configuration.

Status: The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs.

Step: Defines a particular step within a Cloud Dataflow job.

StreamingComputationConfig: Configuration information for a single streaming computation.

StreamingComputationRanges: Describes full or partial data disk assignment information of the computation ranges.

StreamingComputationTask: A task which describes what action should be performed for the specified streaming computation ranges.

StreamingConfigTask: A task that carries configuration information for streaming computations.

StreamingSetupTask: A task which initializes part of a streaming Dataflow job.

StreamingSideInputLocation: Identifies the location of a streaming side input.

StreamingStageLocation: Identifies the location of a streaming computation stage, for stage-to-stage communication.

StreamLocation: Describes a stream of data, either as input to be processed or as output of a streaming Dataflow job.

StringList: A metric value representing a list of strings.

StructuredMessage: A rich message format, including a human readable string, a key for identifying the message, and structured data associated with the message for programmatic consumption.

TaskRunnerSettings: Taskrunner configuration settings.

TemplateMetadata: Metadata describing a template.

TopologyConfig: Global topology of the streaming Dataflow job, including all computations and their sharded locations.

TransformSummary: Description of the type, names/ids, and input/outputs for a transform.

WorkerHealthReport: WorkerHealthReport contains information about the health of a worker.

WorkerHealthReportResponse: WorkerHealthReportResponse contains information returned to the worker in response to a health ping.

WorkerLifecycleEvent: A report of an event in a worker's lifecycle.

WorkerMessage: WorkerMessage provides information to the backend about a worker.

WorkerMessageCode: A message code is used to report status and error messages to the service.

WorkerMessageResponse: A worker_message response allows the server to pass information to the sender.

WorkerPool: Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job.

WorkerSettings: Provides data to pass through to the worker harness.

WorkerShutdownNotice: Shutdown notification from workers.

WorkerShutdownNoticeResponse: Service-side response to WorkerMessage issuing shutdown notice.

WorkItem: WorkItem represents basic information about a WorkItem to be executed in the cloud.

WorkItemServiceState: The Dataflow service's idea of the current state of a WorkItem being processed by a worker.

WorkItemStatus: Conveys a worker's progress through the work described by a WorkItem.

WriteInstruction: An instruction that writes records.