Failure Traps

Failure Traps are the same as a data output sink (opposed to a source), except being bound to a particular tail element of the pipe assembly, traps can be bound to intermediate steps, like a Stream Assertion.

Whenever an operation fails and throws an exception, and there is an associated trap, the offending data will be saved to the file resource specified by the trap. This allows the job to continue processing without any data loss.

By design, clusters are hardware fault tolerant. Lose a node, the cluster continues working.

But software fault tolerance is a little harder. Failure Traps provide a means for the processing to continue without losing track of the data that caused the fault. For high fidelity applications, this may not be so attractive, but low fidelity applications (like web page indexing) this can dramatically improve processing reliability.