o != arg0 is the same as !(o == (arg0)).
o != arg0 is the same as !(o == (arg0)).
the object to compare against this object for dis-equality.
false if the receiver object is equivalent to the argument; true otherwise.
o == arg0 is the same as if (o eq null) arg0 eq null else o.equals(arg0).
o == arg0 is the same as if (o eq null) arg0 eq null else o.equals(arg0).
the object to compare against this object for equality.
true if the receiver object is equivalent to the argument; false otherwise.
o == arg0 is the same as o.equals(arg0).
o == arg0 is the same as o.equals(arg0).
the object to compare against this object for equality.
true if the receiver object is equivalent to the argument; false otherwise.
This method is used to cast the receiver object to be of type T0.
This method is used to cast the receiver object to be of type T0.
Note that the success of a cast at runtime is modulo Scala's erasure semantics. Therefore the expression1.asInstanceOf[String] will throw a ClassCastException at runtime, while the expressionList(1).asInstanceOf[List[String]] will not. In the latter example, because the type argument is erased as
part of compilation it is not possible to check whether the contents of the list are of the requested typed.
the receiver object.
This method creates and returns a copy of the receiver object.
This method creates and returns a copy of the receiver object.
The default implementation of the clone method is platform dependent.
a copy of the receiver object.
By default we only set two keys: io.
By default we only set two keys: io.serializations cascading.tuple.element.comparator.default Override this class, call base and ++ your additional map to set more options
Rather than give the full power of cascading's selectors, we have a simpler set of rules encoded below: 1) if the input is non-definite (ALL, GROUP, ARGS, etc.
Rather than give the full power of cascading's selectors, we have a simpler set of rules encoded below: 1) if the input is non-definite (ALL, GROUP, ARGS, etc...) ALL is the output. Perhaps only fromFields=ALL will make sense 2) If one of from or to is a strict super set of the other, SWAP is used. 3) If they are equal, REPLACE is used. 4) Otherwise, ALL is used.
one iteration of pagerank inputPagerank: <'src_id_input, 'mass_input> return <'src_id, 'mass_n, 'mass_input>
one iteration of pagerank inputPagerank: <'src_id_input, 'mass_input> return <'src_id, 'mass_n, 'mass_input>
Here is a highlevel view of the unweighted algorithm: let N: number of nodes inputPagerank(N_i): prob of walking to node i, d(N_j): N_j's out degree then pagerankNext(N_i) = (\sum_{j points to i} inputPagerank(N_j) / d_j) deadPagerank = (1 - \sum_{i} pagerankNext(N_i)) / N randomPagerank(N_i) = userMass(N_i) * ALPHA + deadPagerank * (1-ALPHA) pagerankOutput(N_i) = randomPagerank(N_i) + pagerankNext(N_i) * (1-ALPHA)
For weighted algorithm: let w(N_j, N_i): weight from N_j to N_i tw(N_j): N_j's total out weights then pagerankNext(N_i) = (\sum_{j points to i} inputPagerank(N_j) * w(N_j, N_i) / tw(N_j))
This method is used to test whether the argument (arg0) is a reference to the
receiver object (this).
This method is used to test whether the argument (arg0) is a reference to the
receiver object (this).
The eq method implements an [http://en.wikipedia.org/wiki/Equivalence_relation equivalence relation] on
non-null instances of AnyRef:
* It is reflexive: for any non-null instance x of type AnyRef, x.eq(x) returns true.
* It is symmetric: for any non-null instances x and y of type AnyRef, x.eq(y) returns true if and
only if y.eq(x) returns true.
* It is transitive: for any non-null instances x, y, and z of type AnyRef if x.eq(y) returns true and y.eq(z) returns true, then x.eq(z) returns true.
Additionally, the eq method has three other properties.
* It is consistent: for any non-null instances x and y of type AnyRef, multiple invocations of
x.eq(y) consistently returns true or consistently returns false.
* For any non-null instance x of type AnyRef, x.eq(null) and null.eq(x) returns false.
* null.eq(null) returns true.
When overriding the equals or hashCode methods, it is important to ensure that their behavior is
consistent with reference equality. Therefore, if two objects are references to each other (o1 eq o2), they
should be equal to each other (o1 == o2) and they should hash to the same value (o1.hashCode == o2.hashCode).
the object to compare against this object for reference equality.
true if the argument is a reference to the receiver object; false otherwise.
This method is used to compare the receiver object (this) with the argument object (arg0) for equivalence.
This method is used to compare the receiver object (this) with the argument object (arg0) for equivalence.
The default implementations of this method is an [http://en.wikipedia.org/wiki/Equivalence_relation equivalence
relation]:
* It is reflexive: for any instance x of type Any, x.equals(x) should return true.
* It is symmetric: for any instances x and y of type Any, x.equals(y) should return true if and
only if y.equals(x) returns true.
* It is transitive: for any instances x, y, and z of type AnyRef if x.equals(y) returns true and
y.equals(z) returns true, then x.equals(z) should return true.
If you override this method, you should verify that your implementation remains an equivalence relation.
Additionally, when overriding this method it is often necessary to override hashCode to ensure that objects
that are "equal" (o1.equals(o2) returns true) hash to the same
scala.Int
(o1.hashCode.equals(o2.hashCode)).
the object to compare against this object for equality.
true if the receiver object is equivalent to the argument; false otherwise.
Multi-entry fields.
Multi-entry fields. This are higher priority than Product conversions so that List will not conflict with Product.
This method is called by the garbage collector on the receiver object when garbage collection determines that there are no more references to the object.
This method is called by the garbage collector on the receiver object when garbage collection determines that there are no more references to the object.
The details of when and if the finalize method are invoked, as well as the interaction between finalizeand non-local returns and exceptions, are all platform dependent.
Returns a representation that corresponds to the dynamic class of the receiver object.
Returns a representation that corresponds to the dynamic class of the receiver object.
The nature of the representation is platform dependent.
a representation that corresponds to the dynamic class of the receiver object.
read the pregenerated nodes file <'src_id, 'dst_ids, 'weights, 'mass_prior>
read the pregenerated nodes file <'src_id, 'dst_ids, 'weights, 'mass_prior>
the total number of nodes, single line file
the total number of nodes, single line file
Returns a hash code value for the object.
Returns a hash code value for the object.
The default hashing algorithm is platform dependent.
Note that it is allowed for two objects to have identical hash codes (o1.hashCode.equals(o2.hashCode)) yet
not be equal (o1.equals(o2) returns false). A degenerate implementation could always return 0.
However, it is required that if two objects are equal (o1.equals(o2) returns true) that they have
identical hash codes (o1.hashCode.equals(o2.hashCode)). Therefore, when overriding this method, be sure
to verify that the behavior is consistent with the equals method.
the hash code value for the object.
This method is used to test whether the dynamic type of the receiver object is T0.
This method is used to test whether the dynamic type of the receiver object is T0.
Note that the test result of the test is modulo Scala's erasure semantics. Therefore the expression1.isInstanceOf[String] will return false, while the expression List(1).isInstanceOf[List[String]] will
return true. In the latter example, because the type argument is erased as part of compilation it is not
possible to check whether the contents of the list are of the requested typed.
true if the receiver object is an instance of erasure of type T0; false otherwise.
o.ne(arg0) is the same as !(o.eq(arg0)).
o.ne(arg0) is the same as !(o.eq(arg0)).
the object to compare against this object for reference dis-equality.
false if the argument is not a reference to the receiver object; true otherwise.
test convergence, if not yet, kick off the next iteration
test convergence, if not yet, kick off the next iteration
Wakes up a single thread that is waiting on the receiver object's monitor.
Wakes up a single thread that is waiting on the receiver object's monitor.
Wakes up all threads that are waiting on the receiver object's monitor.
Wakes up all threads that are waiting on the receiver object's monitor.
you should never call these directly, there are here to make the DSL work.
you should never call these directly, there are here to make the DSL work. Just know, you can treat a Pipe as a RichPipe and vice-versa within a Job
Useful to convert f : Any* to Fields.
Useful to convert f : Any* to Fields. This handles mixed cases ("hey", 'you). Not sure we should be this flexible, but given that Cascading will throw an exception before scheduling the job, I guess this is okay.
Handles treating any TupleN as a Fields object.
Handles treating any TupleN as a Fields object. This is low priority because List is also a Product, but this method will not work for List (because List is Product2(head, tail) and so productIterator won't work as expected. Lists are handled by an implicit in FieldConversions, which have higher priority.
'* means Fields.
'* means Fields.ALL, otherwise we take the .name
Returns a string representation of the object.
Returns a string representation of the object.
The default representation is platform dependent.
a string representation of the object.
weighted page rank for the given graph, start from the given pagerank, perform one iteartion, test for convergence, if not yet, clone itself and start the next page rank job with updated pagerank as input.
This class is very similar to the PageRank class, main differences are: 1. supported weighted pagerank 2. the reset pagerank is pregenerated, possibly through a previous job 3. dead pagerank is evenly distributed
Options: --pwd: working directory, will read/generate the following files there numnodes: total number of nodes nodes: nodes file <'src_id, 'dst_ids, 'weights, 'mass_prior> pagerank: the page rank file eg pagerank_0, pagerank_1 etc totaldiff: the current max pagerank delta Optional arguments: --weighted: do weighted pagerank, default false --curiteration: what is the current iteration, default 0 --maxiterations: how many iterations to run. Default is 20 --jumpprob: probability of a random jump, default is 0.1 --threshold: total difference before finishing early, default 0.001