package s3
- Alphabetic
- By Inheritance
- s3
- Implicits
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
abstract
class
BaseS3RecordReader[K, V] extends RecordReader[K, V]
This is the base class for readers that will create key value pairs for object requests.
This is the base class for readers that will create key value pairs for object requests. Subclass must extend [readObjectRequest] method to map from S3 object requests to (K,V)
- class BytesS3InputFormat extends S3InputFormat[String, Array[Byte]]
- trait Implicits extends Serializable
-
implicit
class
withSaveToS3Methods[K, V] extends SaveToS3Methods[K, V]
- Definition Classes
- Implicits
- class S3GeoTiffInfoReader extends GeoTiffInfoReader
-
abstract
class
S3InputFormat[K, V] extends InputFormat[K, V]
Reads keys from s3n URL using AWS Java SDK.
Reads keys from s3n URL using AWS Java SDK. The number of keys per InputSplits are controlled by S3 pagination. If AWS credentials are not part of the URL they will be discovered using [DefaultAWSCredentialsProviderChain]:
- EnvironmentVariableCredentialsProvider
- SystemPropertiesCredentialsProvider
- ProfileCredentialsProvider
- InstanceProfileCredentialsProvider
-
class
S3InputSplit extends InputSplit with Writable
Represents are batch of keys to be read from an S3 bucket.
Represents are batch of keys to be read from an S3 bucket. AWS credentials have already been discovered and provided by the S3InputFormat.
- class S3LayerManager extends LayerManager[LayerId]
-
class
S3LayerReader extends FilteringLayerReader[LayerId]
Handles reading raster RDDs and their metadata from S3.
-
class
S3LayerWriter extends LayerWriter[LayerId]
Handles writing Raster RDDs and their metadata to S3.
- class S3RDDReader extends Serializable
- class S3RDDWriter extends Serializable
-
abstract
class
S3RecordReader[K, V] extends BaseS3RecordReader[K, V]
This reader will fetch bytes of each key one at a time using [AmazonS3Client.getObject].
This reader will fetch bytes of each key one at a time using [AmazonS3Client.getObject]. Subclass must extend [read] method to map from S3 object bytes to (K,V)
-
class
S3SparkLayerProvider extends S3CollectionLayerProvider with LayerReaderProvider with LayerWriterProvider
Provides S3LayerReader instance for URI with
s3
scheme.Provides S3LayerReader instance for URI with
s3
scheme. The uri represents S3 bucket an prefix of catalog root. ex:s3://<bucket>/<prefix-to-catalog>
- class SaveToS3Methods[K, V] extends MethodExtensions[RDD[(K, V)]]
-
abstract
class
StreamingS3RecordReader[K, V] extends BaseS3RecordReader[K, V]
This reader will stream bytes of each key one at a time using [AmazonS3Client.getObject].
This reader will stream bytes of each key one at a time using [AmazonS3Client.getObject]. Subclass must extend [read] method to map from S3RangeReader to (K,V)
-
class
TiffTagsS3InputFormat extends S3InputFormat[GetObjectRequest, TiffTags]
Reads the tiff tags of GeoTiffs on S3, avoiding full file read.
-
class
GeoTiffS3InputFormat extends S3InputFormat[ProjectedExtent, Tile]
Read single band GeoTiff from S3
Read single band GeoTiff from S3
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) GeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead
-
class
GeoTiffS3RecordReader extends S3RecordReader[ProjectedExtent, Tile]
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) GeoTiffS3RecordReader is deprecated, use S3GeoTiffRDD instead
-
class
MultibandGeoTiffS3InputFormat extends S3InputFormat[ProjectedExtent, MultibandTile]
Read multi band GeoTiff from S3
Read multi band GeoTiff from S3
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) MultibandGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead
-
class
TemporalGeoTiffS3InputFormat extends S3InputFormat[TemporalProjectedExtent, Tile]
Read single band GeoTiff from S3
Read single band GeoTiff from S3
This can be configured with the hadoop configuration by providing: TemporalGeoTiffS3InputFormat.GEOTIFF_TIME_TAG; default of "TIFFTAG_DATETIME" TemporalGeoTiffS3InputFormat.GEOTIFF_TIME_FORMAT; default is ""yyyy:MM:dd HH:mm:ss""
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) TemporalGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead
-
class
TemporalGeoTiffS3RecordReader extends S3RecordReader[TemporalProjectedExtent, Tile]
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) TemporalGeoTiffS3RecordReader is deprecated, use S3GeoTiffRDD instead
-
class
TemporalMultibandGeoTiffS3InputFormat extends S3InputFormat[TemporalProjectedExtent, MultibandTile]
Read single band GeoTiff from S3
Read single band GeoTiff from S3
This can be configured with the hadoop configuration by providing: TemporalMultibandGeoTiffS3InputFormat.GEOTIFF_TIME_TAG; default of "TIFFTAG_DATETIME" TemporalMultibandGeoTiffS3InputFormat.GEOTIFF_TIME_FORMAT; default is ""yyyy:MM:DD HH:MM:SS""
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) TemporalMultibandGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead
Abstract Value Members
-
abstract
def
getClass(): Class[_]
- Definition Classes
- Any
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- Any
-
final
def
##(): Int
- Definition Classes
- Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
equals(arg0: Any): Boolean
- Definition Classes
- Any
-
def
hashCode(): Int
- Definition Classes
- Any
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
toString(): String
- Definition Classes
- Any
- object Implicits extends Implicits
- object S3GeoTiffInfoReader extends Serializable
-
object
S3GeoTiffRDD
The S3GeoTiffRDD object allows for the creation of whole or windowed RDD[(K, V)]s from files on S3.
- object S3InputFormat
- object S3LayerReader
- object S3LayerReindexer
- object S3LayerWriter
- object SaveToS3
Deprecated Value Members
-
object
GeoTiffS3InputFormat
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) GeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead
-
object
TemporalGeoTiffS3InputFormat
- Annotations
- @deprecated
- Deprecated
(Since version 1.0.0) TemporalGeoTiffS3InputFormat is deprecated, use S3GeoTiffRDD instead