org.apache.beam.runners.dataflow.util

Interface Stager

All Known Implementing Classes:

GcsStager

public interface Stager

Interface for staging files needed for running a Dataflow pipeline.

Method Summary

Modifier and Type	Method and Description
java.util.List<DataflowPackage>	stageFiles(java.util.List<PackageUtil.StagedFile> filesToStage) Stage files and return a list of packages DataflowPackage objects describing th actual location at which each file was staged.
DataflowPackage	stageToFile(byte[] bytes, java.lang.String baseName) Stage bytes to a target file name wherever this stager stages things.

Method Detail

stageFiles

java.util.List<DataflowPackage> stageFiles(java.util.List<PackageUtil.StagedFile> filesToStage)

Stage files and return a list of packages DataflowPackage objects describing th actual location at which each file was staged.

The mechanism for staging is owned by the implementation. The only requirement is that the location specified in the returned DataflowPackage should, in fact, contain the contents of the staged file.

stageToFile

DataflowPackage stageToFile(byte[] bytes,
                            java.lang.String baseName)

Stage bytes to a target file name wherever this stager stages things.