Custom I/O patterns
Choosing between built-in and custom connectors
Built-in I/O connectors are tested and hardened, so use them whenever possible. Only use custom I/O connectors when:
- No built-in options exist
- Your pipeline pulls in a small subset of source data
For instance, use a custom I/O connector to enrich pipeline elements with a small subset of source data. If you’re processing a sales order and adding information to each purchase, you can use a custom I/O connector to pull the small subset of data into your pipeline (instead of processing the entire source).