metalus

This project aims to make writing Spark applications easier by abstracting the effort to assemble the driver into reusable steps and pipelines.

View project on GitHub
Documentation Home GCP Home

GCSFileManager

Provides a FileManager implementation that works with the GCS file system. Full parameter descriptions are listed below:

Storage Configuration

  • bucket - The GCS bucket being used.
  • storage - The GCS Storage object to use.

Parameterized Configuration

  • projectId - The Project Id that owns the bucket.
  • bucket - The GCS bucket being used.
  • jsonAuth - The JSON service account credential file as a string.

Credential Configuration

  • bucket - The GCS bucket being used.
  • credentials - The JSON service account credential file as a map.