back to collapsed details

How do refinements relate?

Discarding
Accumulating
Google Cloud DataflowApache FlinkApache Spark (RDD/DStream based)Apache Spark Structured Streaming (Dataset based)IBM StreamsApache SamzaApache NemoHazelcast JetTwister2Python Direct FnRunnerGo Direct Runner

Yes : fully supported


Yes : fully supported


Yes : fully supported


Spark streaming natively discards elements after firing.

Partially : fully supported in batch mode


Yes : fully supported


Yes : fully supported


Yes : fully supported


Yes : fully supported


Yes : fully supported


Yes : fully supported


Requires that the accumulated pane fits in memory, after being passed through the combiner (if relevant)

Yes : fully supported


No


No


Yes : fully supported


Yes : fully supported


Yes : fully supported


Yes : fully supported


Yes : fully supported


Last updated on 2021/02/05

Have you found everything you were looking for?

Was it all useful and clear? Is there anything that you would like to change? Let us know!