Options

This case class contains the various parameters one can set when reading RDDs from S3 using Spark.

TODO: Add persistLevel option

tiffExtensions: Read all file with an extension contained in the given list.
crs: Override CRS of the input files. If None, the reader will use the file's original CRS.
timeTag: Name of tiff tag containing the timestamp for the tile.
timeFormat: Pattern for java.time.format.DateTimeFormatter to parse timeTag.
maxTileSize: Maximum allowed size of each tiles in output RDD. May result in a one input GeoTiff being split amongst multiple records if it exceeds this size. If no maximum tile size is specific, then each file file is read fully. 1024 by defaut.
numPartitions: How many partitions Spark should create when it repartitions the data.
partitionBytes: Desired partition size in bytes, at least one item per partition will be assigned. This option is incompatible with the maxTileSize option. 128 Mb by default.
chunkSize: How many bytes should be read in at a time.
delimiter: Delimiter to use for S3 objet listings. See
getS3Client: A function to instantiate an S3Client. Must be serializable.

Linear Supertypes

Serializable, Serializable, Product, Equals, RasterReader.Options, AnyRef, Any

Instance Constructors

new Options(tiffExtensions: Seq[String] = ..., crs: Option[CRS] = None, timeTag: String = GEOTIFF_TIME_TAG_DEFAULT, timeFormat: String = GEOTIFF_TIME_FORMAT_DEFAULT, maxTileSize: Option[Int] = Some(DefaultMaxTileSize), numPartitions: Option[Int] = None, partitionBytes: Option[Long] = Some(DefaultPartitionBytes), chunkSize: Option[Int] = None, delimiter: Option[String] = None, getS3Client: () ⇒ S3Client = () => S3Client.DEFAULT)

tiffExtensions
Read all file with an extension contained in the given list.
crs
Override CRS of the input files. If None, the reader will use the file's original CRS.
timeTag
Name of tiff tag containing the timestamp for the tile.
timeFormat
Pattern for java.time.format.DateTimeFormatter to parse timeTag.
maxTileSize
Maximum allowed size of each tiles in output RDD. May result in a one input GeoTiff being split amongst multiple records if it exceeds this size. If no maximum tile size is specific, then each file file is read fully. 1024 by defaut.
numPartitions
How many partitions Spark should create when it repartitions the data.
partitionBytes
Desired partition size in bytes, at least one item per partition will be assigned. This option is incompatible with the maxTileSize option. 128 Mb by default.
chunkSize
How many bytes should be read in at a time.
delimiter
Delimiter to use for S3 objet listings. See
getS3Client
A function to instantiate an S3Client. Must be serializable.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
val chunkSize: Option[Int]

How many bytes should be read in at a time.
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val crs: Option[CRS]

Override CRS of the input files.
Override CRS of the input files. If None, the reader will use the file's original CRS.

Definition Classes
Options → Options
val delimiter: Option[String]

Delimiter to use for S3 objet listings.
Delimiter to use for S3 objet listings. See
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
val getS3Client: () ⇒ S3Client

A function to instantiate an S3Client.
A function to instantiate an S3Client. Must be serializable.
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val maxTileSize: Option[Int]

Maximum allowed size of each tiles in output RDD.
Maximum allowed size of each tiles in output RDD. May result in a one input GeoTiff being split amongst multiple records if it exceeds this size. If no maximum tile size is specific, then each file file is read fully. 1024 by defaut.
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val numPartitions: Option[Int]

How many partitions Spark should create when it repartitions the data.
def parseTime(tags: Tags): ZonedDateTime

Definition Classes
Options
val partitionBytes: Option[Long]

Desired partition size in bytes, at least one item per partition will be assigned.
Desired partition size in bytes, at least one item per partition will be assigned. This option is incompatible with the maxTileSize option. 128 Mb by default.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val tiffExtensions: Seq[String]

Read all file with an extension contained in the given list.
val timeFormat: String

Pattern for java.time.format.DateTimeFormatter to parse timeTag.
Pattern for java.time.format.DateTimeFormatter to parse timeTag.

Definition Classes
Options → Options
lazy val timeFormatter: DateTimeFormatter

Definition Classes
Options
val timeTag: String

Name of tiff tag containing the timestamp for the tile.
Name of tiff tag containing the timestamp for the tile.

Definition Classes
Options → Options
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object Options | package S3GeoTiffRDD

Instance Constructors

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val chunkSize: Option[Int]

def clone(): AnyRef

val crs: Option[CRS]

val delimiter: Option[String]

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

final def getClass(): Class[_]

val getS3Client: () ⇒ S3Client

final def isInstanceOf[T0]: Boolean

val maxTileSize: Option[Int]

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val numPartitions: Option[Int]

def parseTime(tags: Tags): ZonedDateTime

val partitionBytes: Option[Long]

final def synchronized[T0](arg0: ⇒ T0): T0

val tiffExtensions: Seq[String]

val timeFormat: String

lazy val timeFormatter: DateTimeFormatter

val timeTag: String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from RasterReader.Options

Inherited from AnyRef

Inherited from Any

Ungrouped