Snowflake resource [Beta]
SnowflakeResource ¶
SnowflakeResource(
paths,
dataframes=None,
snowflake_client=None,
capture_schema_only=False,
force_dataframe=False,
columns_description=None,
)
Bases: Resource
Snowflake tables resource reference wrapper.
This feature is currently in Beta and is subject to change. To activate this feature please contact your sales representative.
This resource wraps Snowflake tables paths that you have stored in Snowflake with optional metadata and versioning. You pass it as an argument of your Vectice Dataset wrapper before logging it to an iteration.
from vectice import SnowflakeResource
connection_parameters = {
...
}
new_session = Session.builder.configs(connection_parameters).create()
sf_resource = SnowflakeResource(
snowflake_client=new_session,
paths="SNOWFLAKE_SAMPLE_DATA.TPCH_SF10.PART",
)
Vectice does not store your actual dataset. Instead, it stores your dataset's metadata, which is information about your dataset. These details are captured by resources tailored to the specific environment in use, such as: local (FileResource), Bigquery, S3, GCS...
Parameters:
Name | Type | Description | Default |
---|---|---|---|
paths |
str | list[str]
|
The pahts of the resources to get. |
required |
dataframes |
Optional
|
The dataframes allowing vectice to optionally compute more metadata about this resource such as columns stats. (Support Pandas, Spark) |
None
|
snowflake_client |
Optional
|
The Snowflake client to retrieve table metadatas. |
None
|
capture_schema_only |
Optional
|
A boolean parameter indicating whether to capture only the schema or both the schema and column statistics of the dataframes. |
False
|
force_dataframe |
Optional
|
A boolean parameter indicating whether the dataframe should be inferred from the table itself. |
False
|
columns_description |
Optional
|
A dictionary or path to a csv file to map the column's name to a specific description. Dictionary should follow the format { "column_name": "Description", ... } |
None
|