Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Added more versions when stuff changed or was added #2658

Merged
merged 4 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/guides/sql-tap.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
# Building SQL taps

## Default type mapping
## Mapping SQL types to JSON Schema

Starting with version `0.41.0`, the Meltano Singer SDK provides a clean way to map SQL types to JSON Schema. This is useful when the SQL dialect you are using has custom types that need to be mapped accordingly to JSON Schema.

### Default type mapping

The Singer SDK automatically handles the most common SQLAlchemy column types, using [`functools.singledispatchmethod`](inv:python:py:class:#functools.singledispatchmethod) to process each type. See the [`SQLToJSONSchema`](connectors.sql.SQLToJSONSchema) reference documentation for details.

## Custom type mapping
### Custom type mapping

If the class above doesn't cover all the types supported by the SQLAlchemy dialect in your tap, you can subclass it and override or extend with a new method for the type you need to support:

Expand Down
20 changes: 16 additions & 4 deletions singer_sdk/connectors/sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,8 @@ class SQLToJSONSchema:
"""SQLAlchemy to JSON Schema type mapping helper.

This class provides a mapping from SQLAlchemy types to JSON Schema types.

.. versionadded:: 0.41.0
"""

@functools.singledispatchmethod
Expand Down Expand Up @@ -171,6 +173,10 @@ def string_to_jsonschema(self, column_type: sa.types.String) -> dict: # noqa: P

Args:
column_type (:column_type:`String`): The column type.

.. versionchanged:: 0.41.0
The :column_type:`length <String.params.length>` attribute is now used to
determine the maximum length of the string type.
"""
if column_type.length:
return th.StringType(max_length=column_type.length).type_dict # type: ignore[no-any-return]
Expand Down Expand Up @@ -241,12 +247,11 @@ def logger(self) -> logging.Logger:

@functools.cached_property
def sql_to_jsonschema(self) -> SQLToJSONSchema:
"""Return the type mapper object.
"""The SQL-to-JSON type mapper object for this SQL connector.

Override this method to provide a custom mapping for your SQL dialect.
Override this property to provide a custom mapping for your SQL dialect.

Returns:
The type mapper object.
.. versionadded:: 0.41.0
"""
return SQLToJSONSchema()

Expand Down Expand Up @@ -380,6 +385,10 @@ def to_jsonschema_type(

Returns:
The JSON Schema representation of the provided type.

.. versionchanged:: 0.40.0
Support for SQLAlchemy type classes and strings in the ``sql_type`` argument
was deprecated. Please pass a SQLAlchemy type object instead.
"""
if isinstance(sql_type, sa.types.TypeEngine):
return self.sql_to_jsonschema.to_jsonschema(sql_type)
Expand Down Expand Up @@ -445,6 +454,9 @@ def get_fully_qualified_name(

Returns:
The fully qualified name as a string.

.. versionchanged:: 0.40.0
A ``FullyQualifiedName`` object is now returned.
"""
return FullyQualifiedName(
table=table_name, # type: ignore[arg-type]
Expand Down
2 changes: 2 additions & 0 deletions singer_sdk/streams/rest.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,8 @@ def user_agent(self) -> str:

Returns:
The user agent string.

.. versionadded:: 0.40.0
"""
return self.config.get(
"user_agent",
Expand Down
Loading