PipelineOptions specific to Google Cloud Dataflow.See: Description
| Interface | Description |
|---|---|
| DataflowPipelineDebugOptions |
Internal.
|
| DataflowPipelineOptions |
Options that can be used to configure the
DataflowRunner. |
| DataflowPipelineWorkerPoolOptions |
Options that are used to configure the Dataflow pipeline worker pool.
|
| DataflowProfilingOptions |
Options for controlling profiling of pipeline execution.
|
| DataflowStreamingPipelineOptions |
[Internal] Options for configuring StreamingDataflowWorker.
|
| DataflowWorkerHarnessOptions |
Options that are used exclusively within the Dataflow worker harness.
|
| DataflowWorkerLoggingOptions | Deprecated
This interface will no longer be the source of truth for worker logging configuration
once jobs are executed using a dedicated SDK harness instead of user code being co-located
alongside Dataflow worker code.
|
| Class | Description |
|---|---|
| DataflowPipelineDebugOptions.DataflowClientFactory |
Returns the default Dataflow client built from the passed in PipelineOptions.
|
| DataflowPipelineDebugOptions.StagerFactory |
Creates a
Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass(). |
| DataflowPipelineDebugOptions.UnboundedReaderMaxReadTimeFactory |
Sets Integer value based on old, deprecated field (
DataflowPipelineDebugOptions.getUnboundedReaderMaxReadTimeSec()). |
| DataflowPipelineOptions.StagingLocationFactory |
Returns a default staging location under
GcpOptions.getGcpTempLocation(). |
| DataflowProfilingOptions.DataflowProfilingAgentConfiguration |
Configuration the for profiling agent.
|
| DataflowStreamingPipelineOptions.EnableWindmillServiceDirectPathFactory |
EnableStreamingEngine defaults to false unless one of the experiment is set.
|
| DataflowStreamingPipelineOptions.GlobalConfigRefreshPeriodFactory |
Read global get config request period from system property
'windmill.global_config_refresh_period'.
|
| DataflowStreamingPipelineOptions.HarnessUpdateReportingPeriodFactory |
Read counter reporting period from system property 'windmill.harness_update_reporting_period'.
|
| DataflowStreamingPipelineOptions.LocalWindmillHostportFactory |
Factory for creating local Windmill address.
|
| DataflowStreamingPipelineOptions.MaxStackTraceDepthToReportFactory |
Read 'MaxStackTraceToReport' from system property 'windmill.max_stack_trace_to_report' or
Integer.MAX_VALUE if unspecified.
|
| DataflowStreamingPipelineOptions.PeriodicStatusPageDirectoryFactory |
Read 'PeriodicStatusPageOutputDirector' from system property
'windmill.periodic_status_page_directory' or null if unspecified.
|
| DataflowStreamingPipelineOptions.WindmillServiceStreamingRpcBatchLimitFactory |
Factory for setting value of WindmillServiceStreamingRpcBatchLimit based on environment.
|
| DataflowWorkerLoggingOptions.WorkerLogLevelOverrides |
Defines a log level override for a specific class, package, or name.
|
| DefaultGcpRegionFactory |
Factory for a default value for Google Cloud region according to
https://cloud.google.com/compute/docs/gcloud-compute/#default-properties.
|
| Enum | Description |
|---|---|
| DataflowPipelineOptions.FlexResourceSchedulingGoal |
Set of available Flexible Resource Scheduling goals.
|
| DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType |
Type of autoscaling algorithm to use.
|
| DataflowWorkerLoggingOptions.Level |
The set of log levels that can be used on the Dataflow worker.
|
PipelineOptions specific to Google Cloud Dataflow.