metalus

This project aims to make writing Spark applications easier by abstracting the effort to assemble the driver into reusable steps and pipelines.

View project on GitHub

Documentation Home

Metalus GCP Step Library

The Metalus GCP is a step library with specific steps and utilities for working with GCP technologies.

Authorization

Support for GCS has been included in this library. By default, GCS and BigQuery will be authorized if credential information is provided. Additionally, for GCS the file system will be set unless the skipGCSFS parameter is set to true on the command line or as a global.

Step Classes

Extensions