All Implemented Interfaces:

Serializable, JavaRDDLike<scala.Tuple2<K,V>,JavaPairRDD<K,V>>

public class JavaHadoopRDD<K,V> extends JavaPairRDD<K,V>

See Also:

• Serialized Form

• Constructor Summary

  Constructors

  JavaHadoopRDD(HadoopRDD<K,V> rdd, scala.reflect.ClassTag<K> kClassTag, scala.reflect.ClassTag<V> vClassTag)

• Method Summary

  scala.reflect.ClassTag<K>

  kClassTag()

  mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,Iterator<scala.Tuple2<K,V>>,Iterator<R>> f, boolean preservesPartitioning)

  Maps over a partition, providing the InputSplit that was used as the base of the partition.

  scala.reflect.ClassTag<V>

  vClassTag()

  Methods inherited from class org.apache.spark.api.java.JavaPairRDD

  aggregateByKey, aggregateByKey, aggregateByKey, cache, classTag, coalesce, coalesce, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, collectAsMap, combineByKey, combineByKey, combineByKey, combineByKey, countApproxDistinctByKey, countApproxDistinctByKey, countApproxDistinctByKey, countByKey, countByKeyApprox, countByKeyApprox, distinct, distinct, filter, filterByRange, filterByRange, first, flatMapValues, foldByKey, foldByKey, foldByKey, fromJavaRDD, fromRDD, fullOuterJoin, fullOuterJoin, fullOuterJoin, groupByKey, groupByKey, groupByKey, groupWith, groupWith, groupWith, intersection, join, join, join, keys, leftOuterJoin, leftOuterJoin, leftOuterJoin, lookup, mapValues, partitionBy, persist, rdd, reduceByKey, reduceByKey, reduceByKey, reduceByKeyLocally, repartition, repartitionAndSortWithinPartitions, repartitionAndSortWithinPartitions, rightOuterJoin, rightOuterJoin, rightOuterJoin, sample, sample, sampleByKey, sampleByKey, sampleByKeyExact, sampleByKeyExact, saveAsHadoopDataset, saveAsHadoopFile, saveAsHadoopFile, saveAsHadoopFile, saveAsNewAPIHadoopDataset, saveAsNewAPIHadoopFile, saveAsNewAPIHadoopFile, setName, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, subtract, subtract, subtract, subtractByKey, subtractByKey, subtractByKey, toRDD, union, unpersist, unpersist, values, wrapRDD

  Methods inherited from interface org.apache.spark.api.java.JavaRDDLike

  aggregate, cartesian, checkpoint, collect, collectAsync, collectPartitions, context, count, countApprox, countApprox, countApproxDistinct, countAsync, countByValue, countByValueApprox, countByValueApprox, flatMap, flatMapToDouble, flatMapToPair, fold, foreach, foreachAsync, foreachPartition, foreachPartitionAsync, getCheckpointFile, getNumPartitions, getStorageLevel, glom, groupBy, groupBy, id, isCheckpointed, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitions, mapPartitionsToDouble, mapPartitionsToDouble, mapPartitionsToPair, mapPartitionsToPair, mapPartitionsWithIndex, mapToDouble, mapToPair, max, min, name, partitioner, partitions, pipe, pipe, pipe, pipe, pipe, reduce, saveAsObjectFile, saveAsTextFile, saveAsTextFile, take, takeAsync, takeOrdered, takeOrdered, takeSample, takeSample, toDebugString, toLocalIterator, top, top, treeAggregate, treeAggregate, treeAggregate, treeReduce, treeReduce, zip, zipPartitions, zipWithIndex, zipWithUniqueId

• Constructor Details

  • JavaHadoopRDD

    public JavaHadoopRDD(HadoopRDD<K,V> rdd, scala.reflect.ClassTag<K> kClassTag, scala.reflect.ClassTag<V> vClassTag)

• Method Details

  • kClassTag

    public scala.reflect.ClassTag<K> kClassTag()

    Overrides:

    kClassTag in class JavaPairRDD<K,V>

  • mapPartitionsWithInputSplit

    public <R> JavaRDD<R> mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,Iterator<scala.Tuple2<K,V>>,Iterator<R>> f, boolean preservesPartitioning)

    Maps over a partition, providing the InputSplit that was used as the base of the partition.

  • vClassTag

    public scala.reflect.ClassTag<V> vClassTag()

    Overrides:

    vClassTag in class JavaPairRDD<K,V>