metalus

This project aims to make writing Spark applications easier by abstracting the effort to assemble the driver into reusable steps and pipelines.

View project on GitHub
Documentation Home AWS Home

S3FileManager

Provides a FileManager implementation that works with the S3 file system. Full parameter descriptions are listed below:

  • accessKeyId - The API key to use when connecting.
  • secretAccessKey - The API secret to use when connecting.
  • region - The AWS region to connect through.
  • bucket - The S3 bucket being used.