Object

geotrellis.spark.io.hadoop

HadoopGeoTiffRDD

Related Doc: package hadoop

Permalink

object HadoopGeoTiffRDD

Allows for reading of whole or windowed GeoTiff as RDD[(K, V)]s through Hadoop FileSystem API.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. HadoopGeoTiffRDD
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class Options(tiffExtensions: Seq[String] = ..., crs: Option[CRS] = None, timeTag: String = GEOTIFF_TIME_TAG_DEFAULT, timeFormat: String = GEOTIFF_TIME_FORMAT_DEFAULT, maxTileSize: Option[Int] = None, numPartitions: Option[Int] = None, chunkSize: Option[Int] = None) extends RasterReader.Options with Product with Serializable

    Permalink

    This case class contains the various parameters one can set when reading RDDs from Hadoop using Spark.

    This case class contains the various parameters one can set when reading RDDs from Hadoop using Spark.

    tiffExtensions

    Read all file with an extension contained in the given list.

    crs

    Override CRS of the input files. If None, the reader will use the file's original CRS.

    timeTag

    Name of tiff tag containing the timestamp for the tile.

    timeFormat

    Pattern for java.time.format.DateTimeFormatter to parse timeTag.

    maxTileSize

    Maximum allowed size of each tiles in output RDD. May result in a one input GeoTiff being split amongst multiple records if it exceeds this size. If no maximum tile size is specific, then each file file is read fully.

    numPartitions

    How many partitions Spark should create when it repartitions the data.

    chunkSize

    How many bytes should be read in at a time.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final val GEOTIFF_TIME_FORMAT_DEFAULT: String("yyyy:MM:dd HH:mm:ss")

    Permalink
  5. final val GEOTIFF_TIME_TAG_DEFAULT: String("TIFFTAG_DATETIME")

    Permalink
  6. object Options extends Serializable

    Permalink
  7. def apply[I, K, V](pathsToDimensions: RDD[(Path, (Int, Int))], pathToKey: (Path, I) ⇒ K, options: Options)(implicit rr: RasterReader[Options, (I, V)]): RDD[(K, V)]

    Permalink

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    pathsToDimensions

    RDD keyed by GeoTiff path with (cols, rows) tuple as value.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  8. def apply[K, V](path: Path, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (K, V)]): RDD[(K, V)]

    Permalink

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    path

    Hdfs GeoTiff path.

    options

    An instance of Options that contains any user defined or default settings.

  9. def apply[I, K, V](path: Path, pathToKey: (Path, I) ⇒ K, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (I, V)]): RDD[(K, V)]

    Permalink

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    Creates a RDD[(K, V)] whose K and V depends on the type of the GeoTiff that is going to be read in.

    path

    Hdfs GeoTiff path.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  10. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  11. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  14. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  16. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  17. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  18. def multiband[K](path: Path, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (K, MultibandTile)]): RDD[(K, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values where V is a MultibandTile.

    Creates RDDs with the [(K, V)] values where V is a MultibandTile. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  19. def multiband[I, K](path: Path, pathToKey: (Path, I) ⇒ K, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (I, MultibandTile)]): RDD[(K, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values where V is a MultibandTile.

    Creates RDDs with the [(K, V)] values where V is a MultibandTile. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  20. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  21. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  22. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  23. def singleband[K](path: Path, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (K, Tile)]): RDD[(K, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values where V is a Tile.

    Creates RDDs with the [(K, V)] values where V is a Tile. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  24. def singleband[I, K](path: Path, pathToKey: (Path, I) ⇒ K, options: Options)(implicit sc: SparkContext, rr: RasterReader[Options, (I, Tile)]): RDD[(K, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values where V is a Tile.

    Creates RDDs with the [(K, V)] values where V is a Tile. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  25. def spatial(path: Path, pathToKey: (Path, ProjectedExtent) ⇒ ProjectedExtent, options: Options)(implicit sc: SparkContext): RDD[(ProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  26. def spatial(path: Path, options: Options)(implicit sc: SparkContext): RDD[(ProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  27. def spatial(path: Path)(implicit sc: SparkContext): RDD[(ProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

  28. def spatialMultiband(path: Path, pathToKey: (Path, ProjectedExtent) ⇒ ProjectedExtent, options: Options)(implicit sc: SparkContext): RDD[(ProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  29. def spatialMultiband(path: Path, options: Options)(implicit sc: SparkContext): RDD[(ProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  30. def spatialMultiband(path: Path)(implicit sc: SparkContext): RDD[(ProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being ProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

  31. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  32. def temporal(path: Path, pathToKey: (Path, TemporalProjectedExtent) ⇒ TemporalProjectedExtent, options: Options)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  33. def temporal(path: Path, options: Options)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  34. def temporal(path: Path)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, Tile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and Tile, respectively. It assumes that the provided files are SinglebandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

  35. def temporalMultiband(path: Path, pathToKey: (Path, TemporalProjectedExtent) ⇒ TemporalProjectedExtent, options: Options)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    pathToKey

    function to transform input key basing on the Path information.

    options

    An instance of Options that contains any user defined or default settings.

  36. def temporalMultiband(path: Path, options: Options)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

    options

    An instance of Options that contains any user defined or default settings.

  37. def temporalMultiband(path: Path)(implicit sc: SparkContext): RDD[(TemporalProjectedExtent, MultibandTile)]

    Permalink

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively.

    Creates RDDs with the [(K, V)] values being TemporalProjectedExtent and MultibandTile, respectively. It assumes that the provided files are MultibandGeoTiffs.

    path

    Hadoop path to recursively search for GeoTiffs.

  38. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  39. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  40. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  41. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped