org.apache.beam.sdk.testing

Class FileChecksumMatcher

java.lang.Object

org.hamcrest.BaseMatcher<T>

org.hamcrest.TypeSafeMatcher<PipelineResult>

org.apache.beam.sdk.testing.FileChecksumMatcher

All Implemented Interfaces:

java.io.Serializable, SerializableMatcher<PipelineResult>, Matcher<PipelineResult>, SelfDescribing

public class FileChecksumMatcher
extends TypeSafeMatcher<PipelineResult>
implements SerializableMatcher<PipelineResult>

Matcher to verify file checksum in E2E test.

For example:

 assertThat(job, new FileChecksumMatcher(checksumString, filePath));
 

or

 assertThat(job, new FileChecksumMatcher(checksumString, filePath, shardTemplate));
 

Checksum of outputs is generated based on SHA-1 algorithm. If output file is empty, SHA-1 hash of empty string (da39a3ee5e6b4b0d3255bfef95601890afd80709) is used as expected.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FileChecksumMatcher(java.lang.String expectedChecksum, org.apache.beam.sdk.util.ShardedFile shardedFile) Constructor using an entirely custom ShardedFile implementation.
FileChecksumMatcher(java.lang.String checksum, java.lang.String filePath) Constructor that uses default shard template.
FileChecksumMatcher(java.lang.String checksum, java.lang.String filePath, java.util.regex.Pattern shardTemplate) Constructor using a custom shard template.

Method Summary

Modifier and Type	Method and Description
void	describeMismatchSafely(PipelineResult pResult, Description description)
void	describeTo(Description description)
boolean	matchesSafely(PipelineResult pipelineResult)

Methods inherited from class org.hamcrest.TypeSafeMatcher

describeMismatch, matches

Methods inherited from class org.hamcrest.BaseMatcher

_dont_implement_Matcher___instead_extend_BaseMatcher_, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.hamcrest.Matcher

_dont_implement_Matcher___instead_extend_BaseMatcher_, describeMismatch, matches

Constructor Detail

FileChecksumMatcher

public FileChecksumMatcher(java.lang.String checksum,
                           java.lang.String filePath)

Constructor that uses default shard template.

Parameters:

checksum - expected checksum string used to verify file content.

filePath - path of files that's to be verified.

FileChecksumMatcher

public FileChecksumMatcher(java.lang.String checksum,
                           java.lang.String filePath,
                           java.util.regex.Pattern shardTemplate)

Constructor using a custom shard template.

Parameters:

checksum - expected checksum string used to verify file content.

filePath - path of files that's to be verified.

shardTemplate - template of shard name to parse out the total number of shards which is used in I/O retry to avoid inconsistency of filesystem. Customized template should assign name "numshards" to capturing group - total shard number.

FileChecksumMatcher

public FileChecksumMatcher(java.lang.String expectedChecksum,
                           org.apache.beam.sdk.util.ShardedFile shardedFile)

Constructor using an entirely custom ShardedFile implementation.

For internal use only.

Method Detail

matchesSafely

public boolean matchesSafely(PipelineResult pipelineResult)

Specified by:

matchesSafely in class TypeSafeMatcher<PipelineResult>

describeTo

public void describeTo(Description description)

Specified by:

describeTo in interface SelfDescribing

describeMismatchSafely

public void describeMismatchSafely(PipelineResult pResult,
                                   Description description)

Overrides:

describeMismatchSafely in class TypeSafeMatcher<PipelineResult>