Skip to content

Latest commit

 

History

History
109 lines (76 loc) · 3.21 KB

README.md

File metadata and controls

109 lines (76 loc) · 3.21 KB
Raw

tap-neon

Singer tap for Neon Serverless Postgres.

Built with the Meltano Singer SDK.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening

Settings

Setting Required Default Description
api_key True None API Key for Neon Serverless Postgres
start_date False None Earliest datetime to get data from
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.

A full list of supported settings and capabilities is available by running: tap-neon --about

Source Authentication and Authorization

Read the docs to learn how to generate an API key.

Usage

You can easily run tap-neon by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-neon --version
tap-neon --help
tap-neon --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

pipx install hatch

Create and Run Tests

Run integration tests:

hatch run test:integration

You can also test the tap-neon CLI interface directly:

hatch run sync:console -- --about --format=json

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Go ahead and install Meltano if you haven't already.

  1. Install all plugins

    meltano install

  2. Check that the extractor is working properly

    meltano invoke tap-neon --version

  3. Execute an ELT pipeline

    meltano run tap-neon target-jsonl