metalus

This project aims to make writing Spark applications easier by abstracting the effort to assemble the driver into reusable steps and pipelines.

View project on GitHub

Documentation Home

Http Rest Client

This object provides a simple client for communicating with restful services. The client requires a url and optional Authorization class.

Authorization

The Authorization trait provides a single function named authorize that is provided a URL Connection.

Any parameters required to instantiate the Authorization instance need to be prefixed with authorization. so they are identified and passed in. To use basic authorization, consider this class signature:

class BasicAuthorization(username: String, password: String) extends Authorization

would need these command line parameters:

--authorization.class com.acxiom.pipeline.api.BasicAuthorization --authorization.username myuser --authorization.password mypasswd

Self-Signed Certificates

The HttpRestClient can accept self signed certificates. Either set the ALLOW_SELF_SIGNED_CERTS environment variable or use the allowSelfSignedCertificates parameter when creating the client.

Functions

Below is a list of functions that may called. Most functions take an optional map of header overrides.

Exists

This function will attempt to open a connection to the provided path to ensure it exists.

Get Last Modified Date

This function will open a connection to the provided path and return the last modified date.

Get Content Length

This function will open a connection to the provided path and return the content length.

Get Headers

This function will open a connection to the provided path and return the headers.

Get Input Stream

This function will open an input stream to the provided path. Calling functions are required to close the stream which will close the connection.

Get Output Stream

This function will open an output stream to the provided path. Calling functions are required to close the stream which will close the connection.

Get String Content

This function will perform a GET against the provided path and return the body as a string.

Post JSON Content

This function will take a given string and POST it to a path as JSON.

Put JSON Content

This function will take a given string and PUT it to a path as JSON.

Post String Content

This function will take a given string and POST it to a path as the provided content type. The default is JSON.

Put String Content

This function will take a given string and PUT it to a path as the provided content type. The default is JSON.

Delete

This function will attempt to run a DELETE operation against the provided path.