Package org.apache.beam.sdk.extensions.sql.meta.provider.datacatalog

Class DataCatalogTableProvider

java.lang.Object

org.apache.beam.sdk.extensions.sql.meta.provider.FullNameTableProvider

org.apache.beam.sdk.extensions.sql.meta.provider.datacatalog.DataCatalogTableProvider

All Implemented Interfaces:

AutoCloseable, CustomTableResolver, TableProvider

public class DataCatalogTableProvider
extends FullNameTableProvider
implements AutoCloseable

Uses DataCatalog to get the source type and schema for a table.

Method Summary

Modifier and Type	Method	Description
BeamSqlTable	buildBeamSqlTable(Table table)	Build a BeamSqlTable using the given table meta info.
void	close()
static DataCatalogTableProvider	create(DataCatalogPipelineOptions options)
static com.google.cloud.datacatalog.v1beta1.DataCatalogClient	createDataCatalogClient(DataCatalogPipelineOptions options)
void	createTable(Table table)	Creates a table.
void	dropTable(String tableName)	Drops a table.
@Nullable Table	getTable(String tableName)	Get a specific table from this provider it is present, or null if it is not present.
Table	getTableByFullName(TableName fullTableName)
Map<String,Table>	getTables()	Get all tables from this provider.
String	getTableType()	Gets the table type this provider handles.
boolean	setSchemaIfNotPresent(String resource, Schema schema)

Methods inherited from class org.apache.beam.sdk.extensions.sql.meta.provider.FullNameTableProvider

getSubProvider, registerKnownTableNames

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.extensions.sql.meta.provider.TableProvider

getSubProviders, supportsPartitioning

Method Details

create

public static DataCatalogTableProvider create(DataCatalogPipelineOptions options)

getTableType

public String getTableType()

Description copied from interface: TableProvider

Gets the table type this provider handles.

Specified by:

getTableType in interface TableProvider

createTable

public void createTable(Table table)

Description copied from interface: TableProvider

Creates a table.

Specified by:

createTable in interface TableProvider

dropTable

public void dropTable(String tableName)

Description copied from interface: TableProvider

Drops a table.

Specified by:

dropTable in interface TableProvider

Parameters:

tableName -

getTables

public Map<String,Table> getTables()

Description copied from interface: TableProvider

Get all tables from this provider.

Specified by:

getTables in interface TableProvider

getTable

public @Nullable Table getTable(String tableName)

Description copied from interface: TableProvider

Get a specific table from this provider it is present, or null if it is not present.

Specified by:

getTable in interface TableProvider

getTableByFullName

public Table getTableByFullName(TableName fullTableName)

Specified by:

getTableByFullName in class FullNameTableProvider

buildBeamSqlTable

public BeamSqlTable buildBeamSqlTable(Table table)

Description copied from interface: TableProvider

Build a BeamSqlTable using the given table meta info.

Specified by:

buildBeamSqlTable in interface TableProvider

createDataCatalogClient

@Internal
public static com.google.cloud.datacatalog.v1beta1.DataCatalogClient createDataCatalogClient(DataCatalogPipelineOptions options)

setSchemaIfNotPresent

@Internal
public boolean setSchemaIfNotPresent(String resource,
 Schema schema)

close

public void close()

Specified by:

close in interface AutoCloseable