cascading.tap
Class MultiTap

java.lang.Object
  cascading.tap.Tap
      cascading.tap.SourceTap
          cascading.tap.MultiTap

All Implemented Interfaces:

FlowElement, Serializable

public class MultiTap
extends SourceTap

Class MultiTap is used to tie multiple Tap instances into a single resource. Effectively this will allow multiple files to be concatenated into the requesting pipe assembly, if they all share the same Scheme instance.

Note that order is not maintained by virtue of the underlying model. If order is necessary, use a unique sequence key to span the resources, like a line number.

Note that if multiple input files have the same Scheme (like TextLine), they may not contain the same semi-structure internally. For example, one file might be an Apache log file, and anoter might be a Log4J log file. If each one should be parsed differently, then they must be handled by different pipe assembly branches.

See Also:

Serialized Form

Field Summary

protected Tap[]

taps

Constructor Summary

protected	MultiTap(Scheme scheme)
	MultiTap(Tap... taps)

Method Summary

boolean	equals(Object object)
Path	getPath() Method getPath() always returns null.
long	getPathModified(JobConf conf) Returns the most current modified time.
Scheme	getScheme() Method getScheme returns the scheme of this Tap object.
Fields	getSourceFields() Method getSourceFields returns the sourceFields of this Tap object.
Tap[]	getTaps() Method getTaps returns the taps of this MultiTap object.
int	hashCode()
boolean	isReplace() Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.
boolean	pathExists(JobConf conf) Method pathExists return true if the path represented by this instance exists.
Tuple	source(Object key, Object value) Method source returns the source value as an instance of Tuple
void	sourceInit(JobConf conf) Method sourceInit initializes this instance as a source.
String	toString()

Methods inherited from class cascading.tap.SourceTap

deletePath, getSinkFields, isSink, makeDirs, openForRead, openForWrite, sink

Methods inherited from class cascading.tap.Tap

flowInit, getQualifiedPath, isAppend, isKeep, isSource, isUseTapCollector, outgoingScopeFor, resolveFields, resolveIncomingOperationFields, setScheme, setUseTapCollector, sinkInit, taps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

taps

protected Tap[] taps

Constructor Detail

MultiTap

protected MultiTap(Scheme scheme)

MultiTap

public MultiTap(Tap... taps)

Method Detail

getTaps

public Tap[] getTaps()

Method getTaps returns the taps of this MultiTap object.

Returns:

the taps (type Tap[]) of this MultiTap object.

getPath

public Path getPath()

Method getPath() always returns null. Since this class represents multiple resources, this is not one single path.

Specified by:

getPath in class Tap

Returns:

Path

getScheme

public Scheme getScheme()

Description copied from class: Tap

Method getScheme returns the scheme of this Tap object.

Overrides:

getScheme in class Tap

Returns:

the scheme (type Scheme) of this Tap object.

isReplace

public boolean isReplace()

Description copied from class: Tap

Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.

Overrides:

isReplace in class Tap

Returns:

boolean

getSourceFields

public Fields getSourceFields()

Description copied from class: Tap

Method getSourceFields returns the sourceFields of this Tap object.

Overrides:

getSourceFields in class Tap

Returns:

the sourceFields (type Fields) of this Tap object.

sourceInit

public void sourceInit(JobConf conf)
                throws IOException

Description copied from class: Tap

Method sourceInit initializes this instance as a source.

This method maybe called more than once if this Tap instance is used outside the scope of a Flow instance or if it participates in multiple times in a given Flow or across different Flows in a Cascade.

In the context of a Flow, it will be called after FlowListener.onStarting(cascading.flow.Flow)

Overrides:

sourceInit in class Tap

Parameters:

conf - of type JobConf

Throws:

IOException - on resource initialization failure.

source

public Tuple source(Object key,
                    Object value)

Description copied from class: Tap

Method source returns the source value as an instance of Tuple

Overrides:

source in class Tap

Parameters:

key - of type WritableComparable

value - of type Writable

Returns:

Tuple

pathExists

public boolean pathExists(JobConf conf)
                   throws IOException

Description copied from class: Tap

Method pathExists return true if the path represented by this instance exists.

Specified by:

pathExists in class Tap

Parameters:

conf - of type JobConf

Returns:

boolean

Throws:

IOException - when the status cannot be determined

getPathModified

public long getPathModified(JobConf conf)
                     throws IOException

Returns the most current modified time.

Specified by:

getPathModified in class Tap

Parameters:

conf - of type JobConf

Returns:

long

Throws:

IOException - when the modified date cannot be determined

equals

public boolean equals(Object object)

Overrides:

equals in class Tap

hashCode

public int hashCode()

Overrides:

hashCode in class Tap

toString

public String toString()

Overrides:

toString in class Object