Package org.apache.beam.sdk.io.iceberg

Class TableMetadataDriver.Builder

java.lang.Object

org.apache.beam.sdk.io.iceberg.TableMetadataDriver.Builder

Enclosing class:

TableMetadataDriver

public abstract static class TableMetadataDriver.Builder
extends Object

Constructor Summary

Constructors

Constructor	Description
Builder()

Method Summary

Modifier and Type	Method	Description
TableMetadataDriver	build()
abstract TableMetadataDriver.Builder	setCatalogConfig(IcebergCatalogConfig catalogConfig)
abstract TableMetadataDriver.Builder	setDynamicDestinations(DynamicDestinations dynamicDestinations)
abstract TableMetadataDriver.Builder	setMaximumCacheSize(@Nullable Integer maximumCacheSize)
abstract TableMetadataDriver.Builder	setPollingBuckets(@Nullable Integer pollingBuckets)	Sets the number of parallel buckets (worker tasks) used to query the Iceberg catalog.
abstract TableMetadataDriver.Builder	setRefreshInterval(@Nullable Duration refreshInterval)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Builder

public Builder()

Method Details

setCatalogConfig

public abstract TableMetadataDriver.Builder setCatalogConfig(IcebergCatalogConfig catalogConfig)

setDynamicDestinations

public abstract TableMetadataDriver.Builder setDynamicDestinations(DynamicDestinations dynamicDestinations)

setMaximumCacheSize

public abstract TableMetadataDriver.Builder setMaximumCacheSize(@Nullable Integer maximumCacheSize)

setRefreshInterval

public abstract TableMetadataDriver.Builder setRefreshInterval(@Nullable Duration refreshInterval)

setPollingBuckets

public abstract TableMetadataDriver.Builder setPollingBuckets(@Nullable Integer pollingBuckets)

Sets the number of parallel buckets (worker tasks) used to query the Iceberg catalog.

Defaults to TableMetadataDriver.DEFAULT_POLLING_BUCKETS (1), which serializes all catalog lookups to avoid overwhelming catalog metastores (e.g. Hive Metastore, REST catalog). For pipelines writing to a large number of distinct dynamic tables (e.g. hundreds of tables per window), consider increasing this value (e.g. 5–10) to parallelize catalog lookups while still bounding load.

build

public TableMetadataDriver build()