cascading.scheme
Class TextLine

java.lang.Object
  cascading.scheme.Scheme
      cascading.scheme.TextLine

All Implemented Interfaces:

Serializable

public class TextLine
extends Scheme

A TextLine is a type of Scheme for plain text files. Files are broken into lines. Either line-feed or carriage-return are used to signal end of line.

By default, this scheme returns a Tuple with two fields, "offset" and "line". But if a Fields is passed on the constructor with one field, the return tuples will simply be the "line" value using the given field name.

If all the input files end with ".zip", the ZipInputFormat will be used. This is not bi-directional, so zip files cannot be written.

See Also:

Serialized Form

Field Summary

static Fields

DEFAULT_SOURCE_FIELDS Field DEFAULT_SOURCE_FIELDS

Constructor Summary

TextLine()
Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.

TextLine(Fields sourceFields)
Creates a new TextLine instance.

TextLine(Fields sourceFields, Fields sinkFields)
Creates a new TextLine instance.

TextLine(Fields sourceFields, Fields sinkFields, int numSinkParts)
Creates a new TextLine instance.

TextLine(Fields sourceFields, int numSinkParts)
Creates a new TextLine instance.

TextLine(int numSinkParts)
Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.

Method Summary

cascading.scheme.TextLine.Compress	getSinkCompression() Method getSinkCompression returns the sinkCompression of this TextLine object.
void	setSinkCompression(cascading.scheme.TextLine.Compress sinkCompression) Method setSinkCompression sets the sinkCompression of this TextLine object.
void	sink(Fields inFields, Tuple tuple, OutputCollector outputCollector) Method sink writes out the given Tuple instance to the outputCollector.
void	sinkInit(Tap tap, JobConf conf) Method sinkInit initializes this instance as a sink.
Tuple	source(Object key, Object value) Method source takes the given Hadoop key and value and returns a new Tuple instance.
void	sourceInit(Tap tap, JobConf conf) Method sourceInit initializes this instance as a source.

Methods inherited from class cascading.scheme.Scheme

equals, getNumSinkParts, getSinkFields, getSourceFields, hashCode, setNumSinkParts, setSinkFields, setSourceFields, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_SOURCE_FIELDS

public static final Fields DEFAULT_SOURCE_FIELDS

Field DEFAULT_SOURCE_FIELDS

Constructor Detail

TextLine

public TextLine()

Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file. By default, numSinkParts is set to 1.

TextLine

public TextLine(int numSinkParts)

Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.

Parameters:

numSinkParts - of type int

TextLine

public TextLine(Fields sourceFields,
                Fields sinkFields)

Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.

Parameters:

sourceFields - the source fields for this scheme

sinkFields - the sink fields for this scheme

TextLine

public TextLine(Fields sourceFields,
                Fields sinkFields,
                int numSinkParts)

Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.

Parameters:

sourceFields - the source fields for this scheme

sinkFields - the sink fields for this scheme

numSinkParts - of type int

TextLine

public TextLine(Fields sourceFields)

Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.

Parameters:

sourceFields - the source fields for this scheme

TextLine

public TextLine(Fields sourceFields,
                int numSinkParts)

Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples. The resulting data set will have numSinkParts.

Parameters:

sourceFields - the source fields for this scheme

numSinkParts - of type int

Method Detail

getSinkCompression

public cascading.scheme.TextLine.Compress getSinkCompression()

Method getSinkCompression returns the sinkCompression of this TextLine object.

Returns:

the sinkCompression (type Compress) of this TextLine object.

setSinkCompression

public void setSinkCompression(cascading.scheme.TextLine.Compress sinkCompression)

Method setSinkCompression sets the sinkCompression of this TextLine object.

Parameters:

sinkCompression - the sinkCompression of this TextLine object.

sourceInit

public void sourceInit(Tap tap,
                       JobConf conf)

Description copied from class: Scheme

Method sourceInit initializes this instance as a source.

Specified by:

sourceInit in class Scheme

Parameters:

tap - of type Tap

conf - of type JobConf

sinkInit

public void sinkInit(Tap tap,
                     JobConf conf)
              throws IOException

Description copied from class: Scheme

Method sinkInit initializes this instance as a sink.

Specified by:

sinkInit in class Scheme

Parameters:

tap - of type Tap

conf - of type JobConf

Throws:

IOException - on initialization failure

source

public Tuple source(Object key,
                    Object value)

Description copied from class: Scheme

Method source takes the given Hadoop key and value and returns a new Tuple instance.

Specified by:

source in class Scheme

Parameters:

key - of type WritableComparable

value - of type Writable

Returns:

Tuple

sink

public void sink(Fields inFields,
                 Tuple tuple,
                 OutputCollector outputCollector)
          throws IOException

Description copied from class: Scheme

Method sink writes out the given Tuple instance to the outputCollector.

Specified by:

sink in class Scheme

Parameters:

inFields - of type Fields

tuple - of type Tuple

outputCollector - of type OutputCollector

Throws:

IOException - when