Package org.apache.beam.runners.dataflow.options

Interface DataflowWorkerLoggingOptions

All Superinterfaces:

HasDisplayData, PipelineOptions

All Known Subinterfaces:

DataflowPipelineOptions, DataflowWorkerHarnessOptions, TestDataflowPipelineOptions

public interface DataflowWorkerLoggingOptions
extends PipelineOptions

Options that are used to control logging configuration on the Dataflow worker.

Some options in this interface are no longer the source of truth for worker logging configuration. Consider using the corresponding options within SdkHarnessOptions to ensure compatibility with other runners.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static enum	DataflowWorkerLoggingOptions.Level	The set of log levels that can be used on the Dataflow worker.
static class	DataflowWorkerLoggingOptions.WorkerLogLevelOverrides	Defines a log level override for a specific class, package, or name.

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner, PipelineOptions.JobNameFactory, PipelineOptions.UserAgentFactory

Method Summary

Modifier and Type	Method	Description
DataflowWorkerLoggingOptions.Level	getDefaultWorkerDirectLoggerLevel()	Controls the log level for which messages are uploaded to Cloud Logging.
DataflowWorkerLoggingOptions.Level	getDefaultWorkerLogLevel()	Deprecated. Prefer SdkHarnessOptions.getDefaultSdkHarnessLogLevel() which works across runners.
Integer	getDirectLoggingCooldownSeconds()
Boolean	getDirectLoggingFallbackToDiskOnErrors()
Long	getWorkerDirectLoggerBufferByteLimit()
Long	getWorkerDirectLoggerBufferElementLimit()
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides	getWorkerDirectLogLevelOverrides()	This option controls the direct log levels for specifically named loggers.
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides	getWorkerLogLevelOverrides()	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() which works across runners.
DataflowWorkerLoggingOptions.Level	getWorkerSystemErrMessageLevel()	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.err' logger as this works across runners.
DataflowWorkerLoggingOptions.Level	getWorkerSystemOutMessageLevel()	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.out' logger as this works across runners.
void	setDefaultWorkerDirectLoggerLevel(DataflowWorkerLoggingOptions.Level level)
void	setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level level)	Deprecated. Prefer SdkHarnessOptions.getDefaultSdkHarnessLogLevel() which works across runners.
void	setDirectLoggingCooldownSeconds(Integer value)
void	setDirectLoggingFallbackToDiskOnErrors(Boolean value)
void	setWorkerDirectLoggerBufferByteLimit(Long value)
void	setWorkerDirectLoggerBufferElementLimit(Long value)
void	setWorkerDirectLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides value)
void	setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides value)	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() which works across runners.
void	setWorkerSystemErrMessageLevel(DataflowWorkerLoggingOptions.Level level)	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.err' logger as this works across runners.
void	setWorkerSystemOutMessageLevel(DataflowWorkerLoggingOptions.Level level)	Deprecated. Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.out' logger as this works across runners.

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, getGbek, getJobName, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, getUserAgent, outputRuntimeOptions, revision, setGbek, setJobName, setOptionsId, setRunner, setStableUniqueNames, setTempLocation, setUserAgent

Method Details

getDefaultWorkerLogLevel

@Enum("INFO")
@Deprecated
DataflowWorkerLoggingOptions.Level getDefaultWorkerLogLevel()

Deprecated.

Prefer SdkHarnessOptions.getDefaultSdkHarnessLogLevel() which works across runners.

This option controls the default log level of all loggers without a log level override.

setDefaultWorkerLogLevel

@Deprecated
void setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level level)

Deprecated.

Prefer SdkHarnessOptions.getDefaultSdkHarnessLogLevel() which works across runners.

getDefaultWorkerDirectLoggerLevel

@Enum("OFF")
DataflowWorkerLoggingOptions.Level getDefaultWorkerDirectLoggerLevel()

Controls the log level for which messages are uploaded to Cloud Logging. If a message is configured to be sent to both directly to cloud logging and default disk-based logging it will just be sent to disk-based logging. This allows for configuration such as "--defaultWorkerLogLevel=WARN --defaultWorkerDirectLoggerLevel=INFO where INFO logs will be directly sent to cloud logging and WARN logs and higher will be sent to disk-based logging.

Note that this is just the default and may be overridden for specific classes with --workerDirectLogLevelOverrides.

setDefaultWorkerDirectLoggerLevel

void setDefaultWorkerDirectLoggerLevel(DataflowWorkerLoggingOptions.Level level)

getWorkerDirectLoggerBufferByteLimit

@Long(104857600L)
Long getWorkerDirectLoggerBufferByteLimit()

setWorkerDirectLoggerBufferByteLimit

void setWorkerDirectLoggerBufferByteLimit(Long value)

getWorkerDirectLoggerBufferElementLimit

@Long(1000000L)
Long getWorkerDirectLoggerBufferElementLimit()

setWorkerDirectLoggerBufferElementLimit

void setWorkerDirectLoggerBufferElementLimit(Long value)

getWorkerSystemOutMessageLevel

@Enum("INFO")
@Deprecated
DataflowWorkerLoggingOptions.Level getWorkerSystemOutMessageLevel()

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.out' logger as this works across runners.

Controls the log level given to messages printed to System.out.

Note that the message may be filtered depending on the defaultWorkerLogLevel or if a System.out override is specified via workerLogLevelOverrides.

setWorkerSystemOutMessageLevel

@Deprecated
void setWorkerSystemOutMessageLevel(DataflowWorkerLoggingOptions.Level level)

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.out' logger as this works across runners.

getWorkerSystemErrMessageLevel

@Enum("ERROR")
@Deprecated
DataflowWorkerLoggingOptions.Level getWorkerSystemErrMessageLevel()

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.err' logger as this works across runners.

Controls the log level given to messages printed to System.err.

Note that the message may be filtered depending on the defaultWorkerLogLevel or if a System.err override is specified via workerLogLevelOverrides.

setWorkerSystemErrMessageLevel

@Deprecated
void setWorkerSystemErrMessageLevel(DataflowWorkerLoggingOptions.Level level)

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() to override the 'System.err' logger as this works across runners.

getWorkerLogLevelOverrides

@Deprecated
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides getWorkerLogLevelOverrides()

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() which works across runners.

This option controls the log levels for specifically named loggers.

Later options with equivalent names override earlier options.

See DataflowWorkerLoggingOptions.WorkerLogLevelOverrides for more information on how to configure logging on a per Class, Package, or name basis. If used from the command line, the expected format is {"Name":"Level",...}, further details on DataflowWorkerLoggingOptions.WorkerLogLevelOverrides.from(java.util.Map<java.lang.String, java.lang.String>).

setWorkerLogLevelOverrides

@Deprecated
void setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides value)

Deprecated.

Prefer using SdkHarnessOptions.getSdkHarnessLogLevelOverrides() which works across runners.

getWorkerDirectLogLevelOverrides

DataflowWorkerLoggingOptions.WorkerLogLevelOverrides getWorkerDirectLogLevelOverrides()

This option controls the direct log levels for specifically named loggers. If a message is configured to be sent to both directly to cloud logging and default disk-based logging it will just be sent to disk-based logging. If an override only exists for a logger for direct logging, the --defaultWorkerLogLevel will be used for the non-direct configuration for the logger.

Later options with equivalent names override earlier options.

See DataflowWorkerLoggingOptions.WorkerLogLevelOverrides for more information on how to configure logging on a per Class, Package, or name basis. If used from the command line, the expected format is {"Name":"Level",...}, further details on DataflowWorkerLoggingOptions.WorkerLogLevelOverrides.from(java.util.Map<java.lang.String, java.lang.String>).

setWorkerDirectLogLevelOverrides

void setWorkerDirectLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides value)

getDirectLoggingFallbackToDiskOnErrors

@Boolean(true)
Boolean getDirectLoggingFallbackToDiskOnErrors()

setDirectLoggingFallbackToDiskOnErrors

void setDirectLoggingFallbackToDiskOnErrors(Boolean value)

getDirectLoggingCooldownSeconds

@Integer(10)
Integer getDirectLoggingCooldownSeconds()

setDirectLoggingCooldownSeconds

void setDirectLoggingCooldownSeconds(Integer value)