Group (Cascading)

Overview

Package

Class

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

cascading.pipe
Class Group

java.lang.Object
  cascading.pipe.Pipe
      cascading.pipe.Group

All Implemented Interfaces:: FlowElement, Serializable

Direct Known Subclasses:: CoGroup, GroupBy

public class Group
extends Pipe
extends Pipe

The base class for GroupBy and CoGroup.

See Also:: Serialized Form

Field Summary
`protected Fields`	`declaredFields` Field declaredFields
`protected Map<String,Fields>`	`groupFieldsMap` Field groupFieldsMap
`protected Map<String,Fields>`	`sortFieldsMap` Field sortFieldsMap

Fields inherited from class cascading.pipe.Pipe
`previous`

Constructor Summary
`Group(Pipe... pipes)` Constructor Group creates a new Group instance.
`Group(Pipe pipe)` Constructor Group creates a new Group instance where grouping occurs on `Fields.ALL` fields.
`Group(Pipe[] pipes, Fields groupFields)` Constructor Group creates a new Group instance.
`Group(Pipe[] pipes, Fields[] groupFields)` Constructor Group creates a new Group instance.
`Group(Pipe[] pipes, Fields[] groupFields, Fields declaredFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(Pipe[] pipes, Fields groupFields, Fields sortFields)` Constructor Group creates a new Group instance.
`Group(Pipe[] pipes, Fields groupFields, Fields sortFields, boolean reverseOrder)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, Fields sortFields)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, Fields sortFields, boolean reverseOrder)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, int repeat)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, int repeat, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, int repeat, Fields declaredFields)` Constructor Group creates a new Group instance.
`Group(Pipe pipe, Fields groupFields, int repeat, Fields declaredFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields)` Constructor Group creates a new Group instance.
`Group(Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, Fields declaredFields)` Constructor Group creates a new Group instance.
`Group(Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, Fields declaredFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe... pipes)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe[] pipes, Fields groupFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe[] pipes, Fields[] groupFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe[] pipes, Fields[] groupFields, Fields declaredFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe[] pipe, Fields groupFields, Fields sortFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe[] pipes, Fields groupFields, Fields sortFields, boolean reverseOrder)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe pipe, Fields groupFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe pipe, Fields groupFields, Fields sortFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe pipe, Fields groupFields, Fields sortFields, boolean reverseOrder)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, Fields declaredFields)` Constructor Group creates a new Group instance.
`Group(String groupName, Pipe lhs, Fields lhsGroupFields, Pipe rhs, Fields rhsGroupFields, Fields declaredFields, CoGrouper coGrouper)` Constructor Group creates a new Group instance.

Method Summary
`void`	`collectReduceGrouping(Scope incomingScope, Scope outgoingScope, TupleEntry entry, OutputCollector output)` Method makeReduceGrouping makes a group Tuple[] of the form [ ['grpValue', ...] [ sourceName, [ 'value', ...] ] ] Since this is a join, we must track from which source a given tuple is sourced from so we can cogroup properly at the reduce stage.
`boolean`	`equals(Object object)`
`Fields`	`getDeclaredFields()` Method getDeclaredFields returns the declaredFields of this Group object.
`Map<String,Fields>`	`getGroupingSelectors()` Method getGroupingSelectors returns the groupingSelectors of this Group object.
`String`	`getName()` Get the name of this pipe.
`Pipe[]`	`getPrevious()` Get all the upstream pipes this pipe is connected to.
`Map<String,Fields>`	`getSortingSelectors()` Method getSortingSelectors returns the sortingSelectors of this Group object.
`int`	`hashCode()`
`boolean`	`isSorted()` Method isSorted returns true if this Group instance is sorting values other than the group fields.
`boolean`	`isSortReversed()` Method isSortReversed returns true if sorting is reversed.
`Iterator<Tuple>`	`iterateReduceValues(JobConf jobConf, Set<Scope> incomingScopes, Scope outgoingScope, WritableComparable key, Iterator values)` Method makeReduceValues wrapps the incoming Hadoop value stream as an iterator over `Tuple` instance.
`Scope`	`outgoingScopeFor(Set<Scope> incomingScopes)` Method outgoingScopeFor returns the Scope this FlowElement hands off to the next FlowElement.
`protected void`	`printInternal(StringBuffer buffer, Scope scope)`
`Fields`	`resolveFields(Scope scope)` Method resolveFields returns the actual field names represented by the given Scope.
`String`	`toString()`
`Tuple`	`unwrapGrouping(Tuple tuple)` Method unwrapGrouping tests if the given grouping key Tuple should be unwrapped if this Group instance is sorting.

Methods inherited from class cascading.pipe.Pipe
`getHeads, pipes, print, resolveIncomingOperationFields`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

Field Detail

groupFieldsMap

protected final Map<String,Fields> groupFieldsMap

Field groupFieldsMap

sortFieldsMap

protected Map<String,Fields> sortFieldsMap

Field sortFieldsMap

declaredFields

protected Fields declaredFields

Field declaredFields

Constructor Detail

Group

public Group(Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             Fields declaredFields)

Constructor Group creates a new Group instance.

Parameters:: lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; declaredFields - of type Fields

Group

public Group(Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             Fields declaredFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; declaredFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields)

Constructor Group creates a new Group instance.

Parameters:: lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields

Group

public Group(Pipe... pipes)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe...

Group

public Group(Pipe[] pipes,
             Fields[] groupFields)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe[]; groupFields - of type Fields[]

Group

public Group(String groupName,
             Pipe[] pipes,
             Fields[] groupFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipes - of type Pipe[]; groupFields - of type Fields[]

Group

public Group(Pipe[] pipes,
             Fields[] groupFields,
             Fields declaredFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe[]; groupFields - of type Fields[]; declaredFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(String groupName,
             Pipe[] pipes,
             Fields[] groupFields,
             Fields declaredFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipes - of type Pipe[]; groupFields - of type Fields[]; declaredFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(String groupName,
             Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             Fields declaredFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; declaredFields - of type Fields

Group

public Group(String groupName,
             Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             Fields declaredFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; declaredFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(String groupName,
             Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(String groupName,
             Pipe lhs,
             Fields lhsGroupFields,
             Pipe rhs,
             Fields rhsGroupFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; lhs - of type Pipe; lhsGroupFields - of type Fields; rhs - of type Pipe; rhsGroupFields - of type Fields

Group

public Group(String groupName,
             Pipe... pipes)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipes - of type Pipe...

Group

public Group(Pipe pipe,
             Fields groupFields,
             int repeat,
             Fields declaredFields)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; repeat - of type int; declaredFields - of type Fields

Group

public Group(Pipe pipe,
             Fields groupFields,
             int repeat,
             Fields declaredFields,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; repeat - of type int; declaredFields - of type Fields; coGrouper - of type CoGrouper

Group

public Group(Pipe pipe,
             Fields groupFields,
             int repeat,
             CoGrouper coGrouper)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; repeat - of type int; coGrouper - of type CoGrouper

Group

public Group(Pipe pipe,
             Fields groupFields,
             int repeat)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; repeat - of type int

Group

public Group(Pipe pipe)

Constructor Group creates a new Group instance where grouping occurs on Fields.ALL fields.

Parameters:: pipe - of type Pipe

Group

public Group(Pipe pipe,
             Fields groupFields)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields

Group

public Group(String groupName,
             Pipe pipe,
             Fields groupFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipe - of type Pipe; groupFields - of type Fields

Group

public Group(Pipe pipe,
             Fields groupFields,
             Fields sortFields)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; sortFields - of type Fields

Group

public Group(String groupName,
             Pipe pipe,
             Fields groupFields,
             Fields sortFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipe - of type Pipe; groupFields - of type Fields; sortFields - of type Fields

Group

public Group(Pipe pipe,
             Fields groupFields,
             Fields sortFields,
             boolean reverseOrder)

Constructor Group creates a new Group instance.

Parameters:: pipe - of type Pipe; groupFields - of type Fields; sortFields - of type Fields; reverseOrder - of type boolean

Group

public Group(String groupName,
             Pipe pipe,
             Fields groupFields,
             Fields sortFields,
             boolean reverseOrder)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipe - of type Pipe; groupFields - of type Fields; sortFields - of type Fields; reverseOrder - of type boolean

Group

public Group(Pipe[] pipes,
             Fields groupFields)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe; groupFields - of type Fields

Group

public Group(String groupName,
             Pipe[] pipes,
             Fields groupFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipes - of type Pipe; groupFields - of type Fields

Group

public Group(Pipe[] pipes,
             Fields groupFields,
             Fields sortFields)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe; groupFields - of type Fields; sortFields - of type Fields

Group

public Group(String groupName,
             Pipe[] pipe,
             Fields groupFields,
             Fields sortFields)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipe - of type Pipe; groupFields - of type Fields; sortFields - of type Fields

Group

public Group(Pipe[] pipes,
             Fields groupFields,
             Fields sortFields,
             boolean reverseOrder)

Constructor Group creates a new Group instance.

Parameters:: pipes - of type Pipe; groupFields - of type Fields; sortFields - of type Fields; reverseOrder - of type boolean

Group

public Group(String groupName,
             Pipe[] pipes,
             Fields groupFields,
             Fields sortFields,
             boolean reverseOrder)

Constructor Group creates a new Group instance.

Parameters:: groupName - of type String; pipes - of type Pipe[]; groupFields - of type Fields; sortFields - of type Fields; reverseOrder - of type boolean

Method Detail

getDeclaredFields

public Fields getDeclaredFields()

Method getDeclaredFields returns the declaredFields of this Group object.

Returns:: the declaredFields (type Fields) of this Group object.

getName

public String getName()

Description copied from class: Pipe

Get the name of this pipe. Guaranteed non-null.

Overrides:: getName in class Pipe

Returns:: String the name of this pipe

getPrevious

public Pipe[] getPrevious()

Description copied from class: Pipe

Get all the upstream pipes this pipe is connected to.

Overrides:: getPrevious in class Pipe

Returns:: all the upstream pipes this pipe is connected to.

getGroupingSelectors

public Map<String,Fields> getGroupingSelectors()

Method getGroupingSelectors returns the groupingSelectors of this Group object.

Returns:: the groupingSelectors (type Map) of this Group object.

getSortingSelectors

public Map<String,Fields> getSortingSelectors()

Method getSortingSelectors returns the sortingSelectors of this Group object.

Returns:: the sortingSelectors (type Map) of this Group object.

isSorted

public boolean isSorted()

Method isSorted returns true if this Group instance is sorting values other than the group fields.

Returns:: the sorted (type boolean) of this Group object.

isSortReversed

public boolean isSortReversed()

Method isSortReversed returns true if sorting is reversed.

Returns:: the sortReversed (type boolean) of this Group object.

collectReduceGrouping

public void collectReduceGrouping(Scope incomingScope,
                                  Scope outgoingScope,
                                  TupleEntry entry,
                                  OutputCollector output)
                           throws IOException

Method makeReduceGrouping makes a group Tuple[] of the form [ ['grpValue', ...] [ sourceName, [ 'value', ...] ] ]

Since this is a join, we must track from which source a given tuple is sourced from so we can cogroup properly at the reduce stage.

Parameters:: incomingScope - of type Scope; outgoingScope - of type Scope; entry - of type TupleEntry; output - of type OutputCollector
Throws:: IOException - thrown by OutputCollector on collect

unwrapGrouping

public Tuple unwrapGrouping(Tuple tuple)

Method unwrapGrouping tests if the given grouping key Tuple should be unwrapped if this Group instance is sorting.

Parameters:: tuple - of type Tuple
Returns:: Tuple

iterateReduceValues

public Iterator<Tuple> iterateReduceValues(JobConf jobConf,
                                           Set<Scope> incomingScopes,
                                           Scope outgoingScope,
                                           WritableComparable key,
                                           Iterator values)

Method makeReduceValues wrapps the incoming Hadoop value stream as an iterator over Tuple instance.

Parameters:: jobConf - of type JobConf; incomingScopes - of type Set; outgoingScope - of type Scope; key - of type WritableComparable; values - of type Iterator @return Iterator
Returns:: a Tuple Iterator

outgoingScopeFor

public Scope outgoingScopeFor(Set<Scope> incomingScopes)

Description copied from interface: FlowElement

Method outgoingScopeFor returns the Scope this FlowElement hands off to the next FlowElement.

Specified by:: outgoingScopeFor in interface FlowElement
Overrides:: outgoingScopeFor in class Pipe

Parameters:: incomingScopes - of type Set
Returns:: Scope
See Also:: FlowElement#outgoingScopeFor(Set)

resolveFields

public Fields resolveFields(Scope scope)

Description copied from interface: FlowElement

Method resolveFields returns the actual field names represented by the given Scope. The scope may be incoming or outgoing in relation to this FlowElement instance.

Specified by:: resolveFields in interface FlowElement
Overrides:: resolveFields in class Pipe

Parameters:: scope - of type Scope
Returns:: Fields
See Also:: FlowElement.resolveFields(Scope)

equals

public boolean equals(Object object)

Overrides:: equals in class Pipe

hashCode

public int hashCode()

Overrides:: hashCode in class Pipe

toString

public String toString()

Overrides:: toString in class Pipe

printInternal

protected void printInternal(StringBuffer buffer,
                             Scope scope)

Overrides:: printInternal in class Pipe

Overview

Package

Class

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

cascading.pipe Class Group

groupFieldsMap

sortFieldsMap

declaredFields

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

Group

getDeclaredFields

getName

getPrevious

getGroupingSelectors

getSortingSelectors

isSorted

isSortReversed

collectReduceGrouping

unwrapGrouping

iterateReduceValues

outgoingScopeFor

resolveFields

equals

hashCode

toString

printInternal

cascading.pipe
Class Group