Package org.apache.beam.sdk.io

Class ClassLoaderFileSystem

java.lang.Object

org.apache.beam.sdk.io.FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

org.apache.beam.sdk.io.ClassLoaderFileSystem

public class ClassLoaderFileSystem
extends FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

A read-only FileSystem implementation looking up resources using a ClassLoader.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ClassLoaderFileSystem.ClassLoaderFileSystemRegistrar	AutoService registrar for the ClassLoaderFileSystem.
static class	ClassLoaderFileSystem.ClassLoaderResourceId

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.FileSystem

FileSystem.LineageLevel

Field Summary

Fields

Modifier and Type	Field	Description
static final String	SCHEMA

Method Summary

Modifier and Type	Method	Description
protected void	copy(List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds, List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds)	Copies a List of file-like resources from one location to another.
protected WritableByteChannel	create(ClassLoaderFileSystem.ClassLoaderResourceId resourceId, CreateOptions createOptions)	Returns a write channel for the given FileSystem.
protected void	delete(Collection<ClassLoaderFileSystem.ClassLoaderResourceId> resourceIds)	Deletes a collection of resources.
protected String	getScheme()	Get the URI scheme which defines the namespace of the FileSystem.
protected List<MatchResult>	match(List<String> specs)	This is the entry point to convert user-provided specs to ResourceIds.
protected ClassLoaderFileSystem.ClassLoaderResourceId	matchNewResource(String path, boolean isDirectory)	Returns a new ResourceId for this filesystem that represents the named resource.
protected ReadableByteChannel	open(ClassLoaderFileSystem.ClassLoaderResourceId resourceId)	Returns a read channel for the given FileSystem.
protected void	rename(List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds, List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds, MoveOptions... moveOptions)	Renames a List of file-like resources from one location to another.

Methods inherited from class org.apache.beam.sdk.io.FileSystem

reportLineage, reportLineage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SCHEMA

public static final String SCHEMA

See Also:

• Constant Field Values

Method Details

match

protected List<MatchResult> match(List<String> specs)
                           throws IOException

Description copied from class: FileSystem

This is the entry point to convert user-provided specs to ResourceIds. Callers should use FileSystem.match(java.util.List<java.lang.String>) to resolve users specs ambiguities before calling other methods.

Implementation should handle the following ambiguities of a user-provided spec:

1. spec could be a glob or a uri. FileSystem.match(java.util.List<java.lang.String>) should be able to tell and choose efficient implementations.
2. The user-provided spec might refer to files or directories. It is common that users that wish to indicate a directory will omit the trailing /, such as in a spec of "/tmp/dir". The FileSystem should be able to recognize a directory with the trailing / omitted, but should always return a correct FileSystem (e.g., "/tmp/dir/" inside the returned MatchResult.

All FileSystem implementations should support glob in the final hierarchical path component of FileSystem. This allows SDK libraries to construct file system agnostic spec. FileSystems can support additional patterns for user-provided specs.

Specified by:

match in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Returns:

List<MatchResult> in the same order of the input specs.

Throws:

IOException - if all specs failed to match due to issues like: network connection, authorization. Exception for individual spec need to be deferred until callers retrieve metadata with MatchResult.metadata().

create

protected WritableByteChannel create(ClassLoaderFileSystem.ClassLoaderResourceId resourceId,
 CreateOptions createOptions)
                              throws IOException

Description copied from class: FileSystem

Returns a write channel for the given FileSystem.

The resource is not expanded; it is used verbatim.

Specified by:

create in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Parameters:

resourceId - the reference of the file-like resource to create

createOptions - the configuration of the create operation

Throws:

IOException

open

protected ReadableByteChannel open(ClassLoaderFileSystem.ClassLoaderResourceId resourceId)
                            throws IOException

Description copied from class: FileSystem

Returns a read channel for the given FileSystem.

The resource is not expanded; it is used verbatim.

If seeking is supported, then this returns a SeekableByteChannel.

Specified by:

open in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Parameters:

resourceId - the reference of the file-like resource to open

Throws:

IOException

copy

protected void copy(List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds,
 List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds)
             throws IOException

Description copied from class: FileSystem

Copies a List of file-like resources from one location to another.

The number of source resources must equal the number of destination resources. Destination resources will be created recursively.

Specified by:

copy in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Parameters:

srcResourceIds - the references of the source resources

destResourceIds - the references of the destination resources

Throws:

IOException

rename

protected void rename(List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds,
 List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds,
 MoveOptions... moveOptions)
               throws IOException

Description copied from class: FileSystem

Renames a List of file-like resources from one location to another.

The number of source resources must equal the number of destination resources. Destination resources will be created recursively.

Specified by:

rename in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Parameters:

srcResourceIds - the references of the source resources

destResourceIds - the references of the destination resources

moveOptions - move options specifying handling of error conditions

Throws:

IOException

delete

protected void delete(Collection<ClassLoaderFileSystem.ClassLoaderResourceId> resourceIds)
               throws IOException

Description copied from class: FileSystem

Deletes a collection of resources.

Specified by:

delete in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

Parameters:

resourceIds - the references of the resources to delete.

Throws:

IOException

matchNewResource

protected ClassLoaderFileSystem.ClassLoaderResourceId matchNewResource(String path,
 boolean isDirectory)

Description copied from class: FileSystem

Returns a new ResourceId for this filesystem that represents the named resource. The user supplies both the resource spec and whether it is a directory.

The supplied singleResourceSpec is expected to be in a proper format, including any necessary escaping, for this FileSystem.

This function may throw an IllegalArgumentException if given an invalid argument, such as when the specified singleResourceSpec is not a valid resource name.

Specified by:

matchNewResource in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

getScheme

protected String getScheme()

Description copied from class: FileSystem

Get the URI scheme which defines the namespace of the FileSystem.

Specified by:

getScheme in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>

See Also:

• RFC 2396