cascading.tap.hadoop
Class S3HttpFileSystem

java.lang.Object
  FileSystem
      cascading.tap.hadoop.StreamedFileSystem
          cascading.tap.hadoop.S3HttpFileSystem

public class S3HttpFileSystem
extends StreamedFileSystem

Class S3HttpFileSystem provides a basic FileSystem for reading and writing remote S3 data.

To use this FileSystem, reference your S3 resources with the following URI pattern:
s3tp://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY@bucketname/key

Optionally these configuration/system properties can be set, instead of stuffing values into the URL authority: "fs.s3tp.awsAccessKeyId" and "fs.s3tp.awsSecretAccessKey".

Field Summary

static String

S3TP_SCHEME

Constructor Summary

S3HttpFileSystem()

Method Summary

FSDataOutputStream	create(Path path, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)
boolean	delete(Path path)
boolean	exists(Path path)
FileStatus	getFileStatus(Path path)
URI	getUri()
void	initialize(URI uri, Configuration conf)
FileStatus[]	listStatus(Path path)
boolean	mkdirs(Path path, FsPermission fsPermission)
FSDataInputStream	open(Path path, int i)

Methods inherited from class cascading.tap.hadoop.StreamedFileSystem

delete, getMD5SumFor, getWorkingDirectory, rename, setMD5SumFor, setWorkingDirectory

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

S3TP_SCHEME

public static final String S3TP_SCHEME

See Also:

Constant Field Values

Constructor Detail

S3HttpFileSystem

public S3HttpFileSystem()

Method Detail

initialize

public void initialize(URI uri,
                       Configuration conf)
                throws IOException

Throws:

IOException

getUri

public URI getUri()

create

public FSDataOutputStream create(Path path,
                                 FsPermission permission,
                                 boolean overwrite,
                                 int bufferSize,
                                 short replication,
                                 long blockSize,
                                 Progressable progress)
                          throws IOException

Overrides:

create in class StreamedFileSystem

Throws:

IOException

open

public FSDataInputStream open(Path path,
                              int i)
                       throws IOException

Throws:

IOException

mkdirs

public boolean mkdirs(Path path,
                      FsPermission fsPermission)
               throws IOException

Overrides:

mkdirs in class StreamedFileSystem

Throws:

IOException

delete

public boolean delete(Path path)
               throws IOException

Overrides:

delete in class StreamedFileSystem

Throws:

IOException

exists

public boolean exists(Path path)
               throws IOException

Throws:

IOException

listStatus

public FileStatus[] listStatus(Path path)
                        throws IOException

Overrides:

listStatus in class StreamedFileSystem

Throws:

IOException

getFileStatus

public FileStatus getFileStatus(Path path)
                         throws IOException

Throws:

IOException