org.apache.beam.sdk.io.splunk

Class SplunkIO

java.lang.Object

org.apache.beam.sdk.io.splunk.SplunkIO

public class SplunkIO
extends java.lang.Object

An unbounded sink for Splunk's Http Event Collector (HEC).

For more information, see the online documentation at Splunk HEC.

Writing to Splunk's HEC

The SplunkIO class provides a PTransform that allows writing SplunkEvent messages into a Splunk HEC end point.

It takes as an input a PCollection<SplunkEvent>, where each SplunkEvent represents an event to be published to HEC.

To configure a SplunkIO, you must provide at a minimum:

• url - HEC endpoint URL.
• token - HEC endpoint token.

The SplunkIO transform can be customized further by optionally specifying:

• parallelism - Number of parallel requests to the HEC.
• batchCount - Number of events in a single batch.
• disableCertificateValidation - Whether to disable ssl validation (useful for self-signed certificates)
• enableBatchLogs - Whether to enable batch logs.
• enableGzipHttpCompression - Whether HTTP requests sent to Splunk HEC should be GZIP encoded.

This transform will return any non-transient write failures via a PCollection<SplunkWriteError>, where each SplunkWriteError captures the error that occurred while attempting to write to HEC. These can be published to a dead-letter sink or reprocessed.

For example:

 PCollection<SplunkEvent> events = ...;

 PCollection<SplunkWriteError> errors =
         events.apply("WriteToSplunk",
              SplunkIO.write(url, token)
                  .withBatchCount(batchCount)
                  .withParallelism(parallelism)
                  .withDisableCertificateValidation(true));
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SplunkIO.Write Class SplunkIO.Write provides a PTransform that allows writing SplunkEvent records into a Splunk HTTP Event Collector end-point using HTTP POST requests.

Method Summary

Modifier and Type	Method and Description
static SplunkIO.Write	write(java.lang.String url, java.lang.String token) Write to Splunk's Http Event Collector (HEC).
static SplunkIO.Write	write(ValueProvider<java.lang.String> url, ValueProvider<java.lang.String> token) Same as write(String, String) but with ValueProvider.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

write

public static SplunkIO.Write write(java.lang.String url,
                                   java.lang.String token)

Write to Splunk's Http Event Collector (HEC).

Parameters:

url - splunk hec url

token - splunk hec authentication token

write

public static SplunkIO.Write write(ValueProvider<java.lang.String> url,
                                   ValueProvider<java.lang.String> token)

Same as write(String, String) but with ValueProvider.

Parameters:

url - splunk hec url

token - splunk hec authentication token