public class RedisIO
extends java.lang.Object
read() provides a source which returns a bounded PCollection containing
key/value pairs as KV<String, String>.
To configure a Redis source, you have to provide Redis server hostname and port number. Optionally, you can provide a key pattern (to filter the keys). The following example illustrates how to configure a source:
pipeline.apply(RedisIO.read()
.withEndpoint("::1", 6379)
.withKeyPattern("foo*"))
It's also possible to specify Redis authentication and connection timeout with the corresponding methods:
pipeline.apply(RedisIO.read()
.withEndpoint("::1", 6379)
.withAuth("authPassword")
.withTimeout(60000)
.withKeyPattern("foo*"))
readKeyPatterns() can be used to request Redis server using input PCollection
elements as key pattern (as String).
pipeline.apply(...)
// here we have a PCollection<String> with the key patterns
.apply(RedisIO.readKeyPatterns().withEndpoint("::1", 6379))
// here we have a PCollection<KV<String,String>>
write() provides a sink to write key/value pairs represented as KV from an
incoming PCollection.
To configure the target Redis server, you have to provide Redis server hostname and port number. The following example illustrates how to configure a sink:
pipeline.apply(...)
// here we a have a PCollection<String, String> with key/value pairs
.apply(RedisIO.write().withEndpoint("::1", 6379))
writeStreams() appends the entries of a PCollection of key/value pairs
represented as KV to the Redis stream at the specified key using the XADD API.
To configure the target Redis server, you have to provide a Redis server hostname and port number. The following example illustrates how to configure a sink:
pipeline.apply(...)
// here we have a PCollection<KV<String, Map<String, String>>>
.apply(RedisIO.writeStreams().withEndpoint("::1", 6379))
Redis Streams optionally can be capped to a specific or exact length (see the documentation
for the XTRIM API); writeStreams() lets
you specify MAXLEN using the withMaxLen() option.
Trimming a stream to an exact length is noted in the Redis documentation as being inefficient;
the withApproximateTrim() boolean option will add the ~ prefix to MAXLEN,
which tells Redis to use "almost exact" trimming.
See the Redis Streams documentation for a deeper discussion of the issues involved.
The following example illustrates how to configure a sink with trimming:
pipeline.apply(...)
// here we have a PCollection<KV<String, Map<String, String>>>
.apply(RedisIO.writeStreams()
.withEndpoint("::1", 6379)
.withMaxLen(1024L)
.withApproximateTrim(true)
)
| Modifier and Type | Class and Description |
|---|---|
static class |
RedisIO.Read
Implementation of
read(). |
static class |
RedisIO.ReadKeyPatterns
Implementation of
readKeyPatterns(). |
static class |
RedisIO.Write
A
PTransform to write to a Redis server. |
static class |
RedisIO.WriteStreams
A
PTransform to write stream key pairs (https://redis.io/topics/streams-intro) to a
Redis server. |
| Modifier and Type | Method and Description |
|---|---|
static RedisIO.Read |
read()
Read data from a Redis server.
|
static RedisIO.ReadKeyPatterns |
readKeyPatterns()
Like
read() but executes multiple instances of the Redis query substituting each
element of a PCollection as key pattern. |
static RedisIO.Write |
write()
Write data to a Redis server.
|
static RedisIO.WriteStreams |
writeStreams()
Write stream data to a Redis server.
|
public static RedisIO.Read read()
public static RedisIO.ReadKeyPatterns readKeyPatterns()
read() but executes multiple instances of the Redis query substituting each
element of a PCollection as key pattern.public static RedisIO.Write write()
public static RedisIO.WriteStreams writeStreams()