Package

geotrellis.spark.io

s3

Permalink

package s3

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. s3
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class AmazonS3Client extends S3Client

    Permalink
  2. abstract class BaseS3RecordReader[K, V] extends RecordReader[K, V] with LazyLogging

    Permalink

    This is the base class for readers that will create key value pairs for object requests.

    This is the base class for readers that will create key value pairs for object requests. Subclass must extend [readObjectRequest] method to map from S3 object requests to (K,V)

  3. class BytesS3InputFormat extends S3InputFormat[String, Array[Byte]]

    Permalink
  4. class S3AttributeStore extends BlobLayerAttributeStore

    Permalink

    Stores and retrieves layer attributes in an S3 bucket in JSON format

  5. trait S3Client extends LazyLogging

    Permalink
  6. class S3CollectionLayerReader extends CollectionLayerReader[LayerId]

    Permalink

    Handles reading raster RDDs and their metadata from S3.

  7. trait S3CollectionReader extends AnyRef

    Permalink
  8. case class S3GeoTiffInfoReader(bucket: String, prefix: String, getS3Client: () ⇒ S3Client = () => S3Client.DEFAULT, delimiter: Option[String] = None, decompress: Boolean = false, streaming: Boolean = true, tiffExtensions: Seq[String] = ...) extends GeoTiffInfoReader with Product with Serializable

    Permalink
  9. abstract class S3InputFormat[K, V] extends InputFormat[K, V] with LazyLogging

    Permalink

    Reads keys from s3n URL using AWS Java SDK.

    Reads keys from s3n URL using AWS Java SDK. The number of keys per InputSplits are controlled by S3 pagination. If AWS credentials are not part of the URL they will be discovered using [DefaultAWSCredentialsProviderChain]:

    • EnvironmentVariableCredentialsProvider
    • SystemPropertiesCredentialsProvider
    • ProfileCredentialsProvider
    • InstanceProfileCredentialsProvider
  10. class S3InputSplit extends InputSplit with Writable with LazyLogging

    Permalink

    Represents are batch of keys to be read from an S3 bucket.

    Represents are batch of keys to be read from an S3 bucket. AWS credentials have already been discovered and provided by the S3InputFormat.

  11. class S3LayerCopier extends LayerCopier[LayerId]

    Permalink
  12. class S3LayerDeleter extends LazyLogging with LayerDeleter[LayerId]

    Permalink
  13. case class S3LayerHeader(keyClass: String, valueClass: String, bucket: String, key: String) extends LayerHeader with Product with Serializable

    Permalink
  14. class S3LayerManager extends LayerManager[LayerId]

    Permalink
  15. class S3LayerProvider extends AttributeStoreProvider with LayerReaderProvider with LayerWriterProvider with ValueReaderProvider

    Permalink

    Provides S3LayerReader instance for URI with s3 scheme.

    Provides S3LayerReader instance for URI with s3 scheme. The uri represents S3 bucket an prefix of catalog root. ex: s3://<bucket>/<prefix-to-catalog>

  16. class S3LayerReader extends FilteringLayerReader[LayerId] with LazyLogging

    Permalink

    Handles reading raster RDDs and their metadata from S3.

  17. class S3LayerUpdater extends LayerUpdater[LayerId] with LazyLogging

    Permalink
  18. class S3LayerWriter extends LayerWriter[LayerId] with LazyLogging

    Permalink

    Handles writing Raster RDDs and their metadata to S3.

  19. trait S3RDDReader extends AnyRef

    Permalink
  20. trait S3RDDWriter extends AnyRef

    Permalink
  21. abstract class S3RecordReader[K, V] extends BaseS3RecordReader[K, V]

    Permalink

    This reader will fetch bytes of each key one at a time using [AmazonS3Client.getObject].

    This reader will fetch bytes of each key one at a time using [AmazonS3Client.getObject]. Subclass must extend [read] method to map from S3 object bytes to (K,V)

  22. class S3ValueReader extends OverzoomingValueReader

    Permalink
  23. class SaveToS3Methods[K] extends MethodExtensions[RDD[(K, Array[Byte])]]

    Permalink
  24. abstract class StreamingS3RecordReader[K, V] extends BaseS3RecordReader[K, V]

    Permalink

    This reader will stream bytes of each key one at a time using [AmazonS3Client.getObject].

    This reader will stream bytes of each key one at a time using [AmazonS3Client.getObject]. Subclass must extend [read] method to map from S3RangeReader to (K,V)

  25. class TiffTagsS3InputFormat extends S3InputFormat[GetObjectRequest, TiffTags]

    Permalink

    Reads the tiff tags of GeoTiffs on S3, avoiding full file read.

  26. implicit class withSaveToS3Methods[K] extends SaveToS3Methods[K]

    Permalink
  27. class GeoTiffS3InputFormat extends S3InputFormat[ProjectedExtent, Tile]

    Permalink

    Read single band GeoTiff from S3

    Read single band GeoTiff from S3

    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) GeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

  28. class GeoTiffS3RecordReader extends S3RecordReader[ProjectedExtent, Tile]

    Permalink
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) GeoTiffS3RecordReader is deprecated, use S3GeoTiffRDD instead

  29. class MultibandGeoTiffS3InputFormat extends S3InputFormat[ProjectedExtent, MultibandTile]

    Permalink

    Read multi band GeoTiff from S3

    Read multi band GeoTiff from S3

    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) MultibandGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

  30. class TemporalGeoTiffS3InputFormat extends S3InputFormat[TemporalProjectedExtent, Tile]

    Permalink

    Read single band GeoTiff from S3

    Read single band GeoTiff from S3

    This can be configured with the hadoop configuration by providing: TemporalGeoTiffS3InputFormat.GEOTIFF_TIME_TAG; default of "TIFFTAG_DATETIME" TemporalGeoTiffS3InputFormat.GEOTIFF_TIME_FORMAT; default is ""yyyy:MM:dd HH:mm:ss""

    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) TemporalGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

  31. class TemporalGeoTiffS3RecordReader extends S3RecordReader[TemporalProjectedExtent, Tile]

    Permalink
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) TemporalGeoTiffS3RecordReader is deprecated, use S3GeoTiffRDD instead

  32. class TemporalMultibandGeoTiffS3InputFormat extends S3InputFormat[TemporalProjectedExtent, MultibandTile]

    Permalink

    Read single band GeoTiff from S3

    Read single band GeoTiff from S3

    This can be configured with the hadoop configuration by providing: TemporalMultibandGeoTiffS3InputFormat.GEOTIFF_TIME_TAG; default of "TIFFTAG_DATETIME" TemporalMultibandGeoTiffS3InputFormat.GEOTIFF_TIME_FORMAT; default is ""yyyy:MM:DD HH:MM:SS""

    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) TemporalMultibandGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

Value Members

  1. object AmazonS3Client

    Permalink
  2. object S3AttributeStore

    Permalink
  3. object S3Client

    Permalink
  4. object S3CollectionLayerReader

    Permalink
  5. object S3CollectionReader extends S3CollectionReader

    Permalink
  6. object S3GeoTiffInfoReader extends Serializable

    Permalink
  7. object S3GeoTiffRDD extends LazyLogging

    Permalink

    The S3GeoTiffRDD object allows for the creation of whole or windowed RDD[(K, V)]s from files on S3.

  8. object S3InputFormat

    Permalink
  9. object S3LayerCopier

    Permalink
  10. object S3LayerDeleter

    Permalink
  11. object S3LayerHeader extends Serializable

    Permalink
  12. object S3LayerMover

    Permalink
  13. object S3LayerReader

    Permalink
  14. object S3LayerReindexer

    Permalink
  15. object S3LayerUpdater

    Permalink
  16. object S3LayerWriter

    Permalink
  17. object S3RDDReader extends S3RDDReader

    Permalink
  18. object S3RDDWriter extends S3RDDWriter

    Permalink
  19. object S3ValueReader

    Permalink
  20. object SaveToS3

    Permalink
  21. package testkit

    Permalink
  22. package util

    Permalink

Deprecated Value Members

  1. object GeoTiffS3InputFormat

    Permalink
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) GeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

  2. object TemporalGeoTiffS3InputFormat

    Permalink
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) TemporalGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead

Inherited from AnyRef

Inherited from Any

Ungrouped