Regex patterns for schemas to filter in ingestion. Specify regex to only match the schema name. e.g. to match all tables in schema analytics, use the regex 'analytics'


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


key: domain

description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.

type: object


description: Whether views should be ingested.

type: boolean


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


key: username

description: Username credential. If not specified, detected with boto3 rules. See https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

type: string


key: password

description: Same detection scheme as username

format: password


key: database

description: The athena database to ingest from. If not set it will be autodetected


description: Aws region where your Athena database is located


description: AWS Role arn for Pyathena to assume in its connection


description: Duration to assume the AWS Role for. Maximum of 43200 (12 hours)

type: integer


description: [deprecated in favor of `query_result_location`] S3 query location


description: The name of your Amazon Athena Workgroups


description: Athena Catalog Name

default: awsdatacatalog


description: S3 path to the [query result bucket](https://docs.aws.amazon.com/athena/latest/ug/querying.html#query-results-specify-location) which should be used by AWS Athena to store results of thequeries executed by DataHub.


description: Base configuration class for stateful ingestion for source configs to inherit from.

default: PROD


key: env

description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


key: options

description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


Regex patterns for schemas to filter in ingestion. Specify regex to only match the schema name. e.g. to match all tables in schema analytics, use the regex 'analytics'


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: host URL


description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


description:
Any non-Dataset source that produces lineage to Datasets should inherit this class.
e.g. Orchestrators, Pipelines, BI Tools etc.

type: array


description: Tableau host URL.


description: Tableau username, must be set if authenticating using username/password.


description: Tableau password, must be set if authenticating using username/password.


description: Tableau token name, must be set if authenticating using a personal access token.


description: Tableau token value, must be set if authenticating using a personal access token.


key: site

description: Tableau Site. Always required for Tableau Online. Use emptystring to connect with Default site on Tableau Server.


description: Whether to verify SSL certificates. If using self-signed certificates, set to false or provide the path to the .pem certificate bundle.


description: When enabled, extracts column-level lineage from Tableau Datasources


description: Environment to use in namespace when constructing URNs.


description: A holder for platform -> platform_instance mappings to generate correct dataset urns


key: projects

description: [deprecated] Use project_pattern instead. List of tableau projects


Filter for specific Tableau projects. For example, use 'My Project' to ingest a root-level Project with name 'My Project', or 'My Project/Nested Project' to ingest a nested Project with name 'Nested Project'. By default, all Projects nested inside a matching Project will be included in ingestion. You can both allow and deny projects based on their name using their name, or a Regex pattern. Deny patterns always take precedence over allow patterns. By default, all projects will be ingested.


description: The separator used for the project_pattern field between project names. By default, we use a slash. You can change this if your Tableau projects contain slashes in their names, and you'd like to filter by project.


description: Default schema to use when schema is not found.


description: Ingest Tags from source. This will override Tags entered from UI


description: Ingest Owner from source. This will override Owner info entered from UI


description: Ingest details for tables external to (not embedded in) tableau as entities.


description: [advanced] Number of metadata objects (e.g. CustomSQLTable, PublishedDatasource, etc) to query at a time using the Tableau API.


description: [advanced] Number of workbooks to query at a time using the Tableau API.


Mappings to change generated dataset urns. Use only if you really know what you are doing.


description: [experimental] Extract usage statistics for dashboards and charts.


description: Ingest a URL to render an embedded Preview of assets within Tableau.


description: Ingest a URL to link out to from dashboards.


description: Ingest a URL to link out to from charts.


description: Whether to extract entire project hierarchy for nested projects.


description: [Experimental] Whether to extract lineage from unsupported custom sql queries using SQL parsing


description: Base configuration class for stateful ingestion for source configs to inherit from.


description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


Regex patterns for schemas to filter in ingestion. Specify regex to only match the schema name. e.g. to match all tables in schema analytics, use the regex 'analytics'


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: MSSQL host URL.


description: database (catalog). If set to Null, all databases will be considered for ingestion.


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


description: See https://docs.sqlalchemy.org/en/14/dialects/mssql.html#module-sqlalchemy.dialects.mssql.pyodbc.


description: Arguments to URL-encode when connecting. See https://docs.microsoft.com/en-us/sql/connect/odbc/dsn-connection-string-attribute?view=sql-server-ver15.


Regex patterns for databases to filter in ingestion.


description: Enable to convert the SQL Server assets urns to lowercase


description: Any source that produces dataset urns in a single environment should inherit this class

default: default


description: The environment that all assets produced by this connector belong to


description: URL for Nifi, ending with /nifi/. e.g. https://mynifi.domain/nifi/


key: auth

description: Nifi authentication. must be one of : NO_AUTH, SINGLE_USER, CLIENT_CERT, KERBEROS


description: time window to analyze provenance events for external datasets


regex patterns for filtering process groups


description: Site name to identify this site with, useful when using input and output ports receiving remote connections


description: Lookup to find site_name for site_url ending with /nifi/, required if using remote process groups in nifi flow


description: Nifi username, must be set for auth = "SINGLE_USER"


description: Nifi password, must be set for auth = "SINGLE_USER"


description: Path to PEM file containing the public certificates for the user/client identity, must be set for auth = "CLIENT_CERT"


description: Path to PEM file containing the client’s secret key


description: The password to decrypt the client_key_file


description: Path to PEM file containing certs for the root CA(s) for the NiFi


description: Config used by the LDAP Source.

default: (objectClass=*)


description: The instance of the platform that all assets produced by this recipe belong to


description: LDAP server URL.


description: LDAP user.


description: LDAP password.


description: LDAP DN.


key: filter

description: LDAP extractor filter.


description: Retrieved attributes list


description: A list of custom attributes to extract from the LDAP provider.


description: If set to true, any users without first and last names will be dropped.


description: Size of each page to fetch when extracting metadata.


description: Use LDAP extractor filter to search managers.


description: [deprecated] Use pagination_enabled


description: Use pagination while do search query (enabled by default).


description: Base configuration class for stateful ingestion for source configs to inherit from.

models: YES

sources: YES

seeds: YES

snapshots: YES

test_definitions: YES

test_results: YES

default: dbt:

default: PATCH


description: When enabled, emits lineage as incremental to existing lineage already in DataHub. When disabled, re-states lineage on each run.


description: When enabled, sql parser will run in isolated in a separate process. This can affect processing time but can protect from sql parser's mem leak.


description: Environment to use in namespace when constructing URNs.


DBT Stateful Ingestion Config.


description: The platform that dbt is loading onto. (e.g. bigquery / redshift / postgres etc.)


description: The platform instance for the platform that dbt is operating on. Use this if you have multiple instances of the same platform (e.g. redshift) and need to distinguish between them.


description: Use model identifier instead of model name if defined (if not, default to model name).


description: Controls for enabling / disabling metadata emission for different dbt entities (models, test definitions, test results, etc.)


description: Prefix added to tags during ingestion.


regex patterns for dbt model names to filter in ingestion.


description: mapping rules that will be executed against dbt meta properties. Refer to the section below on dbt meta automated mappings.


description: mapping rules that will be executed against dbt column meta properties. Refer to the section below on dbt meta automated mappings.


description: mapping rules that will be executed against dbt query_tag meta properties. Refer to the section below on dbt meta automated mappings.


description: Whether the new tags, terms and owners to be added will override the existing ones added only by this source or not. Value for this config can be "PATCH" or "OVERRIDE"


description: Whether or not to strip email id while adding owners using dbt meta actions.


description: When enabled, ownership info will be extracted from the dbt meta


description: Regex string to extract owner from the dbt node using the `(?P<name>...) syntax` of the [match object](https://docs.python.org/3/library/re.html#match-objects), where the group name must be `owner`. Examples: (1)`r"(?P<owner>(.*)): (\w+) (\w+)"` will extract `jdoe` as the owner from `"jdoe: John Doe"` (2) `r"@(?P<owner>(.*))"` will extract `alice` as the owner from `"@alice"`.


description: Prior to version 0.9.4.2, the assertion GUIDs did not include the environment. If you're using multiple dbt ingestion that are only distinguished by env, then you should set this flag to True.


description: When enabled, converts column URNs to lowercase to ensure cross-platform compatibility. If `target_platform` is Snowflake, the default is True.


description: When enabled, applies the mappings that are defined through the meta_mapping directives.


description: When enabled, applies the mappings that are defined through the `query_tag_mapping` directives.


description: The dbt Cloud metadata API endpoint.


key: token

description: The API token to use to authenticate with DBT Cloud.


description: The DBT Cloud account ID to use.


description: The dbt Cloud project ID to use.


description: The ID of the job to ingest metadata from.


description: The ID of the run to ingest metadata from. If not specified, we'll default to the latest run.


key: path

description: File path to folder or file to ingest, or URL to a remote file. If pointed to a folder, all files with extension {file_extension} (default json) within that folder will be processed.


description: When providing a folder to use to read files, set this field to control file extensions that you want the source to process. * is a special value that means process every file regardless of extension

default: .json


key: aspect

description: Set to an aspect to only read this aspect for ingestion.


description: When enabled, counts total number of records in the file before starting. Used for accurate estimation of completion time. Turn it off if startup time is too high.


description: Base configuration class for stateful ingestion for source configs to inherit from.


description: List of PathSpec. See [below](#path-spec) the details about PathSpec


description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


key: platform

description: The platform that this source connects to (either 's3' or 'file'). If not specified, the platform will be inferred from the path_specs.


AWS configuration


description: Whether or not to create tags in datahub from the s3 bucket


description: Whether or not to create tags in datahub from the s3 object


regex patterns for tables to profile


key: profiling

description: Data profiling configuration


description: Max amount of memory to grant Spark.


description: Spark configuration properties to set on the SparkSession. Put config property names into quotes. For example: '"spark.executor.memory": "2g"'


description: Maximum number of rows to use when inferring schemas for TSV and CSV files.


description: Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use.


description: Number of files to list to sample for schema inference. This will be ignored if sample_files is set to False in the pathspec.


description: Any source that is a primary producer of Dataset metadata should inherit this class

default: looker

default: 30 days


Pattern for providing dataset names to explores. Allowed variables are ['platform', 'env', 'project', 'model', 'name']


Pattern for providing browse paths to explores. Allowed variables are ['platform', 'env', 'project', 'model', 'name']


Pattern for providing dataset names to views. Allowed variables are ['platform', 'env', 'project', 'model', 'name']


Pattern for providing browse paths to views. Allowed variables are ['platform', 'env', 'project', 'model', 'name']


description: When enabled, attaches tags to measures, dimensions and dimension groups to make them more discoverable. When disabled, adds this information to the description of the column.


description: Default platform name. Don't change.


description: When enabled, extracts column-level lineage from Views and Explores


description: Looker API client id.


description: Looker API client secret.


description: Url to your Looker instance: `https://company.looker.com:19999` or `https://looker.company.com`, or similar. Used for making API calls to Looker and constructing clickable dashboard and chart urls.


Populates the [TransportOptions](https://github.com/looker-open-source/sdk-codegen/blob/94d6047a0d52912ac082eb91616c1e7c379ab262/python/looker_sdk/rtl/transport.py#L70) struct for looker client


Patterns for selecting dashboard ids that are to be included


Patterns for selecting chart ids that are to be included


description: Whether to include deleted dashboards and looks.


description: When enabled, extracts ownership from Looker directly. When disabled, ownership is left empty for dashboards and charts.


key: actor

description: This config is deprecated in favor of `extract_owners`. Previously, was the actor to use in ownership properties of ingested metadata.


description: When enabled, converts Looker user emails of the form name@domain.com to urn:li:corpuser:name when assigning ownership


description: Whether to skip ingestion of dashboards in personal folders. Setting this to True will only ingest dashboards in the Shared folder space.


description: Max parallelism for Looker API calls. Defaults to cpuCount or 40


description: Optional URL to use when constructing external URLs to Looker if the `base_url` is not the correct one to use. For example, `https://looker-public.company.com`. If not provided, the external base URL will default to `base_url`.


description: Whether to ingest usage statistics for dashboards. Setting this to True will query looker system activity explores to fetch historical dashboard usage.


description: Used only if extract_usage_history is set to True. Interval to extract looker dashboard usage history for. See https://docs.looker.com/reference/filter-expressions#date_and_time.


description: Produce URLs used to render Looker Explores as Previews inside of DataHub UI. Embeds must be enabled inside of Looker to use this feature.


description: Extract looks which are not part of any Dashboard. To enable this flag the stateful_ingestion should also be enabled.


description: Any source that is a primary producer of Dataset metadata should inherit this class

default: salesforce


description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


description: Salesforce username


description: Password for Salesforce user


description: Consumer key for Salesforce JSON web token access


description: Private key as a string for Salesforce JSON web token access


description: Security token for Salesforce username


description: Salesforce instance url. e.g. https://MyDomainName.my.salesforce.com


description: Connect to Sandbox instance of your Salesforce


description: Access token for instance url


description: Ingest Tags from source. This will override Tags entered from UI


Regex patterns for Salesforce objects to filter in ingestion.


description: Regex patterns for tables/schemas to describe domain_key domain key (domain_key can be any string like "sales".) There can be multiple domain keys specified.


Regex patterns for profiles to filter in ingestion, allowed by the `object_pattern`.


description: Base configuration class for stateful ingestion for source configs to inherit from.


description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


Deprecated in favour of database_pattern.


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: host URL


description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


Regex patterns for databases to filter in ingestion.


default: mysql+pymysql


description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


description: Whether views should be ingested.


description: MySQL host URL.


default: DAY

format: date-time


description: A holder for platform -> platform_instance mappings to generate correct dataset urns


description: Size of the time window to aggregate usage stats.


description: Latest date of lineage/usage to consider. Default: Current time in UTC


description: Earliest date of lineage/usage to consider. Default: Last full day in UTC (or hour, depending on `bucket_duration`). You can also specify relative time with respect to end_time such as '-7 days' Or '-7d'.


description: Whether tables should be ingested.


description: username


description: password


description: ClickHouse host URL.


description: database (catalog)


description: The part of the URI and it's used to provide additional configuration options or parameters for the database connection.


description: Whether table lineage should be ingested.


description: Any source that produces dataset urns in a single environment should inherit this class


description: MongoDB connection URI.


description: MongoDB username.


description: MongoDB password.


key: authMechanism

description: MongoDB authentication mechanism.


description: Additional options to pass to `pymongo.MongoClient()`.


key: enableSchemaInference

description: Whether to infer schemas.


key: schemaSamplingSize

description: Number of documents to use when inferring schema size. If set to `0`, all documents will be scanned.


key: useRandomSampling

description: If documents for schema inference should be randomly selected. If `False`, documents will be selected from start.


key: maxSchemaSize

description: Maximum number of fields to include in the schema.


regex patterns for databases to filter in ingestion.


regex patterns for collections to filter in ingestion.


description: Base configuration class for stateful ingestion for source configs to inherit from.

format: file-path

format: directory-path

format: uri

required: path platform


key: env


description: Set this to a single file-path or a directory-path (for recursive traversal) or a remote url. e.g. https://json.schemastore.org/petstore-v1.0.json


description: Set this to a platform that you want all schemas to live under. e.g. schemaregistry / schemarepo etc.


description: When enabled, uses the `$id` field in the json schema as the base uri for following references.


Use this if URI-s need to be modified during reference resolution. Simple string match - replace capabilities are supported.


key: file

description: File path or URL to business glossary file to ingest.

format: path


description: Generate guid urns instead of a plaintext path urn with the node/term's hierarchy.


description: The instance of the platform that all assets produced by this recipe belong to


key: options

description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


Regex patterns for schemas to filter in ingestion. Specify regex to only match the schema name. e.g. to match all tables in schema analytics, use the regex 'analytics'


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


key: domain

description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


key: username

description: username


key: password

description: password


description: host URL


key: database

description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


description: Base configuration class for stateful ingestion for source configs to inherit from.

default: public


Common config for S3 lineage generation


description: The environment that all assets produced by this connector belong to


description: A holder for platform -> platform_instance mappings to generate correct dataset urns


description: Size of the time window to aggregate usage stats.


description: Latest date of lineage/usage to consider. Default: Current time in UTC


description: Earliest date of lineage/usage to consider. Default: Last full day in UTC (or hour, depending on `bucket_duration`). You can also specify relative time with respect to end_time such as '-7 days' Or '-7d'.


description: The default schema to use if the sql parser fails to parse the schema with `sql_based` lineage collector


description: Whether table lineage should be ingested.


description: Whether lineage should be collected from copy commands


description: Whether lineage should be collected from unload commands


description: Whether to capture lineage query parser errors with dataset properties for debuggings


Which table lineage collector mode to use. Available modes are: [stl_scan_based, sql_based, mixed]


description: Any source that connects to a platform should inherit this class

default: connect-cluster


description: Platform instance mapping to use when constructing URNs. e.g.`platform_instance_map: { "hive": "warehouse" }`


description: The instance of the platform that all assets produced by this recipe belong to


description: URI to connect to.


description: Kafka Connect username.


description: Kafka Connect password.


description: Cluster to ingest from.


description: Whether to convert the urns of ingested lineage dataset to lowercase


regex patterns for connectors to filter for ingestion.


description: Provided Configurations


description: Platform instance mapping when multiple instances for a platform is available. Entry for a platform should be in either `platform_instance_map` or `connect_to_platform_map`. e.g.`connect_to_platform_map: { "postgres-connector-finance-db": "postgres": "core_finance_instance" }`


description: Provide lineage graph for sources connectors other than Confluent JDBC Source Connector, Debezium Source Connector, and Mongo Source Connector


description:
Common AWS credentials config.

Currently used by:
    - Glue source
    - SageMaker source


description: AWS access key ID. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS secret access key. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS session token. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS roles to assume. If using the string format, the role ARN can be specified directly. If using the object format, the role can be specified in the RoleArn field and additional available arguments are documented at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html?highlight=assume_role#STS.Client.assume_role


description: Named AWS profile to use. Only used if access key / secret are unset. If not set the default will be used


description: AWS region code.


description: The AWS service endpoint. This is normally [constructed automatically](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html), but can be overridden here.


description: A set of proxy configs to use with AWS. See the [botocore.config](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) docs for details.


description: The environment that all assets produced by this connector belong to


regex patterns for databases to filter in ingestion.


regex patterns for tables to filter in ingestion.


description: Whether to extract feature groups.


description: Whether to extract models.


description: Whether to extract AutoML jobs.


description: Base configuration class for stateful ingestion for source configs to inherit from.

default: druid


description: The instance of the platform that all assets produced by this recipe belong to


description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


regex patterns for schemas to filter in ingestion.


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: host URL


description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


default: glue

default: upstream


description: AWS access key ID. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS secret access key. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS session token. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS roles to assume. If using the string format, the role ARN can be specified directly. If using the object format, the role can be specified in the RoleArn field and additional available arguments are documented at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html?highlight=assume_role#STS.Client.assume_role


description: Named AWS profile to use. Only used if access key / secret are unset. If not set the default will be used


description: AWS region code.


description: The AWS service endpoint. This is normally [constructed automatically](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html), but can be overridden here.


description: A set of proxy configs to use with AWS. See the [botocore.config](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) docs for details.


description: The environment that all assets produced by this connector belong to


regex patterns for databases to filter in ingestion.


regex patterns for tables to filter in ingestion.


description: The instance of the platform that all assets produced by this recipe belong to


description: The platform to use for the dataset URNs. Must be one of ['glue', 'athena'].


description: When enabled, extracts ownership from Glue directly and overwrites existing owners. When disabled, ownership is left empty for datasets.


description: Whether to extract Glue transform jobs.


description: Whether to ignore unsupported connectors. If disabled, an error will be raised.


description: Whether to emit S3-to-Glue lineage.


description: If `upstream`, S3 is upstream to Glue. If `downstream` S3 is downstream to Glue.


description: regex patterns for tables to filter to assign domain_key.


description: The aws account id where the target glue catalog lives. If None, datahub will ingest glue in aws caller's account.


description: If set to True, ignore database resource links.


description: If an S3 Buckets Tags should be created for the Tables ingested by Glue. Please Note that this will not apply tags to any folders ingested, only the files.


description: If an S3 Objects Tags should be created for the Tables ingested by Glue.


description: Configs to ingest data profiles from glue table


description: Base configuration class for stateful ingestion for source configs to inherit from.


description: Size of the time window to aggregate usage stats.


description: Latest date of lineage/usage to consider. Default: Current time in UTC


description: Earliest date of lineage/usage to consider. Default: Last full day in UTC (or hour, depending on `bucket_duration`). You can also specify relative time with respect to end_time such as '-7 days' Or '-7d'.


description: Number of top queries to save to each table.


regex patterns for user emails to filter in usage.


description: Whether to display operational stats.


description: Whether to report read operational stats. Experimental.


description: Whether to format sql queries


description: Whether to ingest the top_n_queries.


description: A holder for platform -> platform_instance mappings to generate correct dataset urns


description: The instance of the platform that all assets produced by this recipe belong to


Deprecated in favour of database_pattern.


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: ClickHouse host URL.


description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


Regex patterns for databases to filter in ingestion.


description: The part of the URI and it's used to provide additional configuration options or parameters for the database connection.


description: Whether table lineage should be ingested.


description: Base configuration class for stateful ingestion for source configs to inherit from.


description: Size of the time window to aggregate usage stats.


description: Latest date of lineage/usage to consider. Default: Current time in UTC


description: Earliest date of lineage/usage to consider. Default: Last full day in UTC (or hour, depending on `bucket_duration`). You can also specify relative time with respect to end_time such as '-7 days' Or '-7d'.


description: Number of top queries to save to each table.


regex patterns for user emails to filter in usage.


description: Whether to display operational stats.


description: Whether to report read operational stats. Experimental.


description: Whether to format sql queries


description: Whether to ingest the top_n_queries.


description: The environment that all assets produced by this connector belong to


description: The instance of the platform that all assets produced by this recipe belong to


Regex patterns for schemas to filter in ingestion. Specify regex to only match the schema name. e.g. to match all tables in schema analytics, use the regex 'analytics'


Regex patterns for tables to filter in ingestion. Specify regex to match the entire table name in database.schema.table format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns for views to filter in ingestion. Note: Defaults to table_pattern if not specified. Specify regex to match the entire view name in database.schema.view format. e.g. to match all views starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter tables (or specific columns) for profiling during ingestion. Note that only tables allowed by the `table_pattern` will be considered.


description: Attach domains to databases, schemas or tables during ingestion using regex patterns. Domain key can be a guid like *urn:li:domain:ec428203-ce86-4db3-985d-5a8ee6df32ba* or a string like "Marketing".) If you provide strings, then datahub will attempt to resolve this name to a guid, and will error out if this fails. There can be multiple domain keys specified.


description: Whether views should be ingested.


description: Whether tables should be ingested.


description: If the source supports it, include table lineage to the underlying storage location.


description: username


description: password


description: host URL


description: The name of the catalog from getting the usage


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


description: The email domain which will be appended to the users


description: The catalog name where the audit table can be found


description: The schema name where the audit table can be found


key: type

description: The type of the state provider to use. For DataHub use `datahub`


key: config

description: The configuration required for initializing the state provider. Default: The datahub_api config if set at pipeline level. Otherwise, the default DatahubClientConfig. See the defaults (https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/graph/client.py#L19).


key: StatefulStaleMetadataRemovalConfig

description: Base specialized config for Stateful Ingestion with stale metadata removal capability.


key: enabled

description: The type of the ingestion state provider registered with datahub.


description: Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.


key: AllowDenyPattern

description: A class to store allow deny regexes


key: allow

description: List of regex patterns to include in ingestion


key: deny

description: List of regex patterns to exclude from ingestion.


key: ignoreCase

description: Whether to ignore case sensitivity during pattern matching.


description: Whether to do profiling at lower freq or not. This does not do any scheduling just adds additional checks to when not to run profiling.


description: Number between 0 to 6 for day of week (both inclusive). 0 is Monday and 6 is Sunday. If not specified, defaults to Nothing and this field does not take affect.


description: Number between 1 to 31 for date of month (both inclusive). If not specified, defaults to Nothing and this field does not take affect.


description: Whether profiling should be done.


Experimental feature. To specify operation configs.


key: limit

description: Max number of documents to profile. By default, profiles all documents.


key: offset

description: Offset in documents to profile. By default, uses no offset.


description: Whether to report datasets or dataset columns which were not profiled. Set to `True` for debugging purposes.


description: Whether to turn off expensive profiling or not. This turns off profiling for quantiles, distinct_value_frequencies, histogram & sample_values. This also limits maximum number of fields being profiled to 10.


description: Whether to perform profiling at table-level only, or include column-level profiling as well.


description: Whether to profile for the number of nulls for each column.


description: Whether to profile for the number of distinct values for each column.


description: Whether to profile for the min value of numeric columns.


description: Whether to profile for the max value of numeric columns.


description: Whether to profile for the mean value of numeric columns.


description: Whether to profile for the median value of numeric columns.


description: Whether to profile for the standard deviation of numeric columns.


description: Whether to profile for the quantiles of numeric columns.


description: Whether to profile for distinct value frequencies.


description: Whether to profile for the histogram for numeric fields.


description: Whether to profile for the sample values for all columns.


description: Upper limit for number of sample values to collect for all columns.


description: A positive integer that specifies the maximum number of columns to profile for any table. `None` implies all columns. The cost of profiling goes up significantly as the number of columns to profile goes up.


description: Profile table only if it has been updated since these many number of days. If set to `null`, no constraint of last modified time for tables to profile. Supported only in `snowflake` and `BigQuery`.

type: number


description: Profile tables only if their size is less then specified GBs. If set to `null`, no limit on the size of tables to profile. Supported only in `snowflake` and `BigQuery`


description: Profile tables only if their row count is less then specified count. If set to `null`, no limit on the row count of tables to profile. Supported only in `snowflake` and `BigQuery`


description: Use an approximate query for row count. This will be much faster but slightly less accurate. Only supported for Postgres and MySQL.


description: Number of worker threads to use for profiling. Set to 1 to disable.


description: *This feature is still experimental and can be disabled if it causes issues.* Reduces the total number of queries issued and speeds up profiling by dynamically combining SQL queries where possible.


description: For partitioned datasets profile only the partition which matches the datetime or profile the latest one if not set. Only Bigquery supports this.


key: StatefulIngestionConfig

description: Basic Stateful Ingestion Specific Configuration for any source.


description: The type of the ingestion state provider registered with datahub.


description: username


description: password


description: MySQL host URL.


description: database (catalog)


description: [Deprecated] Alias to apply to database when ingesting.


description: URI of database to connect to. See https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls. Takes precedence over other connection parameters.


description: Any options specified here will be passed to [SQLAlchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine) as kwargs.


key: KafkaConsumerConnectionConfig

description: Configuration class for holding connectivity information for Kafka consumers


description: Extra schema registry config serialized as JSON. These options will be passed into Kafka's SchemaRegistryClient. https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html?#schemaregistryclient


description: The request timeout used when interacting with the Kafka APIs.


description: Extra consumer config serialized as JSON. These options will be passed into Kafka's DeserializingConsumer. See https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#deserializingconsumer and https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md .


key: BucketDuration

description: An enumeration.

enum: DAY HOUR


description: Size of the time window to aggregate usage stats.


description: Latest date of lineage/usage to consider. Default: Current time in UTC


description: Earliest date of lineage/usage to consider. Default: Last full day in UTC (or hour, depending on `bucket_duration`). You can also specify relative time with respect to end_time such as '-7 days' Or '-7d'.


description: Number of top queries to save to each table.


regex patterns for user emails to filter in usage.


description: Whether to display operational stats.


description: Whether to report read operational stats. Experimental.


description: Whether to format sql queries


description: Whether to ingest the top_n_queries.


description: Correction to pad start_time and end_time with. For handling the case where the read happens within our time range but the query completion event is delayed and happens after the configured end time.

format: time-delta


description: Whether to apply view's usage to its base tables. If set to False, uses sql parser and applies usage to views / tables mentioned in the query. If set to True, usage is applied to base tables only.


description: Project id to set the credentials


description: Private key id


description: Private key in a form of '-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n'


description: Client email


description: Client Id


description: Authentication uri


description: Token uri


description: Auth provider x509 certificate url


description: Authentication type


description: If not set it will be default to https://www.googleapis.com/robot/v1/metadata/x509/client_email


key: GitInfo

description: A reference to a Git repository, including a deploy key that can be used to clone it.

default: main


key: repo

description: Name of your Git repo e.g. https://github.com/datahub-project/datahub or https://gitlab.com/gitlab-org/gitlab. If organization/repo is provided, we assume it is a GitHub repo.


key: branch

description: Branch on which your files live by default. Typically main or master. This can also be a commit hash.


description: Template for generating a URL to a file in the repo e.g. '{repo_url}/blob/{branch}/{file_path}'. We can infer this for GitHub and GitLab repos, and it is otherwise required.It supports the following variables: {repo_url}, {branch}, {file_path}


description: A private key file that contains an ssh key that has been configured as a deploy key for this repository. Use a file where possible, else see deploy_key for a config field that accepts a raw string.


description: A private key that contains an ssh key that has been configured as a deploy key for this repository. See deploy_key_file if you want to use a file that contains this key.


description: The url to call `git clone` on. We infer this for github and gitlab repos, but it is required for other hosts.


description: The environment that the platform is located in. Leaving this empty will inherit defaults from the top level Looker configuration


description: Looker API client id.


description: Looker API client secret.


description: Url to your Looker instance: `https://company.looker.com:19999` or `https://looker.company.com`, or similar. Used for making API calls to Looker and constructing clickable dashboard and chart urls.


Populates the [TransportOptions](https://github.com/looker-open-source/sdk-codegen/blob/94d6047a0d52912ac082eb91616c1e7c379ab262/python/looker_sdk/rtl/transport.py#L70) struct for looker client


key: include

description: Path to table. Name variable `{table}` is used to mark the folder with dataset. In absence of `{table}`, file level dataset will be created. Check below examples for more details.


key: exclude

description: list of paths in glob pattern which will be excluded while scanning for the datasets


description: Files with extenstions specified here (subset of default value) only will be scanned to create dataset. Other files will be omitted.

default: csv tsv json parquet avro


description: For files without extension it will assume the specified file type. If it is not set the files without extensions will be skipped.


description: Display name of the dataset.Combination of named variables from include path and strings


description: Enable or disable processing compressed files. Currently .gz and .bz files are supported.


description: Not listing all the files but only taking a handful amount of sample file to infer the schema. File count and file size calculation will be disabled. This can affect performance significantly if enabled


description: Any source that produces s3 lineage from/to Datasets should inherit this class.


description: List of PathSpec. See below the details about PathSpec


key: LineageMode

description: An enumeration.


key: EmitDirective

description: A holder for directives for emission for specific types of entities

enum: YES NO ONLY


key: DBTEntitiesEnabled

description: Controls which dbt entities are going to be emitted by this source

default: YES


key: models

description: Emit metadata for dbt models when set to Yes or Only


key: sources

description: Emit metadata for dbt sources when set to Yes or Only


key: seeds

description: Emit metadata for dbt seeds when set to Yes or Only


key: snapshots

description: Emit metadata for dbt snapshots when set to Yes or Only


description: Emit metadata for test definitions when enabled when set to Yes or Only


description: Emit metadata for test results when set to Yes or Only


key: RoleArn

description: ARN of the role to assume.


key: ExternalId

description: External ID to use when assuming the role.


key: AwsConnectionConfig

description:
Common AWS credentials config.

Currently used by:
    - Glue source
    - SageMaker source
    - dbt source


description: AWS access key ID. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS secret access key. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS session token. Can be auto-detected, see [the AWS boto3 docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) for details.


description: AWS roles to assume. If using the string format, the role ARN can be specified directly. If using the object format, the role can be specified in the RoleArn field and additional available arguments are documented at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html?highlight=assume_role#STS.Client.assume_role


description: Named AWS profile to use. Only used if access key / secret are unset. If not set the default will be used


description: AWS region code.


description: The AWS service endpoint. This is normally [constructed automatically](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html), but can be overridden here.


description: A set of proxy configs to use with AWS. See the [botocore.config](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) docs for details.


key: GitReference

description: Reference to a hosted Git repository. Used to generate "view source" links.


description: Name of your Git repo e.g. https://github.com/datahub-project/datahub or https://gitlab.com/gitlab-org/gitlab. If organization/repo is provided, we assume it is a GitHub repo.


description: Branch on which your files live by default. Typically main or master. This can also be a commit hash.


description: Template for generating a URL to a file in the repo e.g. '{repo_url}/blob/{branch}/{file_path}'. We can infer this for GitHub and GitLab repos, and it is otherwise required.It supports the following variables: {repo_url}, {branch}, {file_path}


key: PrestoOnHiveConfigMode

description: An enumeration.

enum: hive presto presto-on-hive trino


description: DataHub platform instance name. To generate correct urn for upstream dataset, this should match with platform instance name used in ingestion recipe of other datahub sources.


key: env

description: The environment that all assets produced by DataHub platform ingestion source belong to

default: PROD


description: Whether ingest PowerBI user as Datahub Corpuser


description: Use PowerBI User email to ingest as corpuser, default is powerbi user identifier


description: Remove PowerBI User email suffix for example, @acryl.io


description: Take PBI dataset configuredBy as dataset owner if exist


description: Need to have certain authority to qualify as owner for example ['ReadWriteReshareExplore','Owner','Admin']


key: enabled

description: Whether to enable profiling for the elastic search source.


Experimental feature. To specify operation configs.


description:
List of regex patterns to remove from the name of the URN. All of the indices before removal of URNs are considered as the same dataset. These are applied in order for each URN.
        The main case where you would want to have multiple of these if the name where you are trying to remove suffix from have different formats.
        e.g. ending with -YYYY-MM-DD as well as ending -epochtime would require you to have 2 regex patterns to remove the suffixes across all URNs.


key: type

description: The type of the classifier to use. For DataHub,  use `datahub`


key: config

description: The configuration required for initializing the classifier. If not specified, uses defaults for classifer type.


description: Whether classification should be used to auto-detect glossary terms


description: Number of sample values used for classification.


description: Number of worker threads to use for classification. Set to 1 to disable.


Regex patterns to filter tables for classification. This is used in combination with other patterns in parent config. Specify regex to match the entire table name in `database.schema.table` format. e.g. to match all tables starting with customer in Customer database and public schema, use the regex 'Customer.public.customer.*'


Regex patterns to filter columns for classification. This is used in combination with other patterns in parent config. Specify regex to match the column name in `database.schema.table.column` format.


description: Optional mapping to provide glossary term identifier for info type


key: classifiers

description: Classifiers to use to auto-detect glossary terms. If more than one classifier, infotype predictions from the classifier defined later in sequence take precedance.

type: datahub


key: OAuthIdentityProvider

description: An enumeration.

enum: microsoft okta


key: provider

description: Identity provider for oauth.Supported providers are microsoft and okta.


description: Authority url of your identity provider


description: client id of your registered application


key: scopes

description: scopes required to connect to snowflake


description: Do you want to use certificate and private key to authenticate using oauth


description: client secret of the application if use_certificate = false


description: base64 encoded certificate content if use_certificate = true


description: base64 encoded private key content if use_certificate = true


key: TagOption

description: An enumeration.


description: Whether profiling should be done.


Experimental feature. To specify operation configs.


description: SQL Warehouse id, for running profiling queries.


description: Whether to perform profiling at table-level only or include column-level profiling as well.


key: pattern

description: Regex patterns to filter tables for profiling during ingestion. Specify regex to match the `catalog.schema.table` format. Note that only tables allowed by the `table_pattern` will be considered.


description: Whether to call ANALYZE TABLE as part of profile ingestion.If false, will ingest the results of the most recent ANALYZE TABLE call, if any.


description: Maximum time to wait for an ANALYZE TABLE query to complete.


description: Number of worker threads to use for profiling. Set to 1 to disable.


description: Multiplier for exponential backoff when waiting to retry


description: Maximum interval to wait when retrying


description: Maximum number of attempts to retry before failing


key: AdlsSourceConfig

description:
Common Azure credentials config.

https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-python


description: Base folder in hierarchical namespaces to start from.


description: Azure storage account container name.


description: Name of the Azure storage account.  See [Microsoft official documentation on how to create a storage account.](https://docs.microsoft.com/en-us/azure/storage/blobs/create-data-lake-storage-account)


description: Azure storage account access key that can be used as a credential. **An account key, a SAS token or a client secret is required for authentication.**


description: Azure storage account Shared Access Signature (SAS) token that can be used as a credential. **An account key, a SAS token or a client secret is required for authentication.**


description: Azure client secret that can be used as a credential. **An account key, a SAS token or a client secret is required for authentication.**


description: Azure client (Application) ID required when a `client_secret` is used as a credential.


description: Azure tenant (Directory) ID required when a `client_secret` is used as a credential.


description: Whether profiling should be done.


description: Whether to profile for the number of nulls for each column.


description: Whether to profile for the min value of numeric columns.


description: Whether to profile for the max value of numeric columns.


Experimental feature. To specify operation configs.


AWS configuration


description: Whether or not to create tags in datahub from the s3 bucket


description: # Whether or not to create tags in datahub from the s3 object


description: Access ID


description: Secret


description: A holder for platform -> platform mappings to generate correct dataset urns


description: A holder for database -> database mappings to generate correct dataset urns


key: NifiAuthType

description: An enumeration.


key: FileReadMode

description: An enumeration.

enum: STREAM BATCH AUTO


description: Whether to perform profiling at table-level only or include column-level profiling as well.


description: A positive integer that specifies the maximum number of columns to profile for any table. `None` implies all columns. The cost of profiling goes up significantly as the number of columns to profile goes up.


description: Whether to profile for the mean value of numeric columns.


description: Whether to profile for the median value of numeric columns.


description: Whether to profile for the standard deviation of numeric columns.


description: Whether to profile for the quantiles of numeric columns.


description: Whether to profile for distinct value frequencies.


description: Whether to profile for the histogram for numeric fields.


description: Whether to profile for the sample values for all columns.


key: SalesforceAuthType

description: An enumeration.


description: Whether profiling should be done. Supports only table-level profiling at this stage


key: match

description: Pattern to match on uri-s as part of reference resolution. See replace field


key: replace

description: Pattern to replace with as part of reference resolution. See match field


description: The parameter name for row count in glue table.


description: The parameter name for column count in glue table.


description: The parameter name for the count of unique value in a column.


description: The parameter name for the proportion of unique values in a column.


description: The parameter name for the count of null values in a column.


description: The parameter name for the proportion of null values in a column.


key: min

description: The parameter name for the min value of a column.


key: max

description: The parameter name for the max value of a column.


key: mean

description: The parameter name for the mean value of a column.


key: median

description: The parameter name for the median value of a column.


key: stdev

description: The parameter name for the standard deviation of a column.


Regex patterns for filtering partitions for profile. The pattern should be a string like: "{'key':'value'}".


Experimental feature. To specify operation configs.


key: items

description: Transformer configs see at https://datahubproject.io/docs/metadata-ingestion/docs/transformer


key: type

description: Transformer type


key: config

description: Transformer config


key: sink

description: sink


Linguistic Schema Definition Language schema


description: .fossa.yaml specification for FOSSA CLI 2.x or greater

required: name release

const: full

const: off

const: bundler

const: cargo

const: carthage

const: cocoapods

const: composer

const: conda

const: glide

const: godep

const: gradle

const: leiningen

const: maven

const: mix

const: npm

const: nimble

const: pub

const: rpm

const: renv

const: scala

const: swift

const: yarn

const: repomanifest

const: cabal

const: stack

const: nuspec

const: packagereference

const: paket

const: projectassetjson

const: pipenv

const: poetry

const: setuptools

const: perl

const: pnpm

const: pdm

const: ArchiveUpload

const: CLILicenseScan


key: project

description: The project fields allow you to configure settings for the project you are interacting with through the FOSSA API.


key: id

description:
The project ID defines a unique ID that the FOSSA API will use to reference this project. The project ID can be found in the UI on the project settings page listed as the `Project Locator` underneath the `Project Title` setting.

By default, it will use git remote origin url as project id if it's git repository. If it does not recognize version control system (vcs), project directory's name will be used.


key: name

description: The name field sets the projects visible name in the FOSSA dashboard. By default, this will be set to the project's ID.


key: team

description: The name of the team in your FOSSA organization to associate this project with.


key: policy

description: The id of the policy in your FOSSA organization to associate this project with. Mutually excludes `project.policy`.


key: link

description: An external link that will appear in the FOSSA UI for this specific project.


key: url

description: The URL of your project that will appear in FOSSA. This URL is intended to be the URL to the repository of this project.


key: jiraProjectKey

description: The Jira Project Key to associate with your project for improved issue triage. Refer to https://docs.fossa.com/docs/atlassian-jira#linking-fossa-projects-to-jira-projects for more information.


key: releaseGroup

description: The `name:` and `release:` of the release group's release to add your project to in the FOSSA dashboard. If you choose to associate a project with a release group, you **must** supply both name and release.


description: The name of release group.


key: release

description: The release associated with the release group


key: labels

description: A list of labels that are assigned to the project


key: telemetry

description: The telemetry fields are used to configure telemetry data collection and it's destination.

const: full

const: off


key: scope

description: project branch is an optional setting used for organizing project revisions in the FOSSA UI. The branch field is intended to function similar to how Git defines a branch.


description: Collects and sends ends telemetry data to server.


description: Does not send telemetry data to server.


key: commit

description: The commit is used to identify a specific scan for a project (determined by project.id). This is intended to be used identically to how Git treats commit hashes. If not provided, cli will parse current HEAD state from .git directory. If project does not have version control system, unix timestamp will be used.


key: revision

description: The revision fields are used to help FOSSA differentiate between one upload for a project and another, just as GitHub uses commit hashes and branch names.


key: branch


key: type

description: Target (package manager)

const: bundler

const: cargo

const: carthage

const: cocoapods

const: composer

const: conda

const: glide

const: godep

const: gradle

const: leiningen

const: maven

const: mix

const: npm

const: nimble

const: pub

const: rpm

const: renv

const: scala

const: swift

const: yarn

const: repomanifest


description: For bundler targets (ruby)

const: bundler


description: For cargo targets (rust)

const: cargo


description: For carthage targets (ios, objective-c)

const: carthage


description: For cocoapod targets (ios, objective-c, swift)

const: cocoapods


description: For composer targets (php)

const: composer


description: For conda targets

const: conda


description: For glide targets (golang)

const: glide


description: For godep targets (golang)

const: godep


description: For gradle targets (kotlin and java)

const: gradle


description: For leiningen targets (clojure)

const: leiningen


description: For maven targets (kotlin and java)

const: maven


description: For mix targets (elixir)

const: mix


description: For npm targets (javascript)

const: npm


description: For nimble targets (nim)

const: nimble


description: For pub targets (dart, flutter)

const: pub


description: For rebar3 targets (erlang)


description: For rpm targets

const: rpm


description: For renv targets (r)

const: renv


description: For scala targets

const: scala


description: For swift targets

const: swift


description: For yarn targets (javascript)

const: yarn


description: For repomanifest

const: repomanifest


description: For cabal targets (haskell)

const: cabal


description: For stack targets (haskell)

const: stack


description: For nuspec targets (dotnet)

const: nuspec


description: For package reference targets (dotnet)

const: packagereference


description: For paket targets (dotnet)

const: paket


description: For project asset json targets (dotnet)

const: projectassetjson


description: For pipenv targets (python)

const: pipenv


description: For poetry targets (python)

const: poetry


description: For setuptools targets (python)

const: setuptools


description: For perl targets (using *META.{json,yml})

const: perl


description: For pnpm targets (javascript)

const: pnpm


description: For pdm targets (python)

const: pdm


key: path

description: Associated path with target type (if any)


key: experimental

description: Experimental preferences with fossa cli.


key: gradle

description: Gradle preferences for all targets


key: configurations-only

description: Configurations to only include in analysis (by default excludes any other configurations not listed)


key: vendoredDependencies

description: the vendoredDependencies fields allow you to configure vendored dependency scans. Vendored dependency scans are described in https://github.com/fossas/fossa-cli/blob/master/docs/features/vendored-dependencies.md

const: ArchiveUpload

const: CLILicenseScan


key: forceRescans

description: If true, forces a re-scan of all vendored dependencies on every run. If false or not present, then we do not re-scan vendored dependencies that have been previously scanned. A vendored dependency has been previously scanned if a dependency with the same name and version has already been scanned by your organization. If no version is provided, then any change in the files being scanned will result in a rescan.


key: scanMethod

description: Vendored dependencies are scanned by the "Archive Upload" method, as described in https://github.com/fossas/fossa-cli/blob/master/docs/features/vendored-dependencies.md


description: Vendored dependencies are scanned by the "CLI-side license scan" method, as described in https://github.com/fossas/fossa-cli/blob/master/docs/features/vendored-dependencies.md


key: licenseScanPathFilters

description: licenseScanPathFilters allows you to filter which files are scanned when doing a CLILicenseScan. This setting does not apply if you use the ArchiveUpload method of scanning vendoredDependencies.


key: only

description: A list of globs that will be used to filter paths. If there are any entries in the `only` list, then only paths that match one or more of the globs in the `only` list will be scanned for licenses.


key: exclude

description: A list of globs that will be used to filter paths. If there are any entries in the `exclude` list, then paths that match any of the `exclude` entries will not be scanned for licenses.


key: version

description:
Specifies the version of configuration file. Versions 1 and 2 were used by CLI versions up until CLI 2.0.0 and are no longer supported.

CLI 2.x and greater only supports version 3.


key: server

description:
Sets the endpoint that the CLI will send requests to. This field should only be modified if your FOSSA account lives on a different server than app.fossa.com.

This is most commonly needed with on-premise instances of FOSSA.


key: apiKey

description: Sets the https://docs.fossa.com/docs/api-reference#api-tokens that is required for accessing the FOSSA API and uploading data (e.g. `fossa analyze`) or retrieving information (e.g. `fossa test`) about a project.


key: targets

description: The targets filtering allows you to specify the exact targets which be should be scanned.


description: The list of `only` targets that should be scanned.


description:
The list of `exclude` targets which should be excluded from scanning. The targets listed in the exclude section will override the targets listed in the only sections.

This feature is used most effectively to remove specific targets from a directory.


key: paths

description: The paths filtering section allows you to specify which paths should be scanned and which should not. The paths should be listed as their location from the root of your project.


description: The list of paths to only allow scanning within.


description: The list of paths to exclude from scanning in your directory.


description: The Compose file is a YAML file defining a multi-containers based application.

type: integer string

enum: host private

type: number string

type: string number

format: expose

type: string number null

type: string integer

enum: always never if_not_present build missing


description: declared for backward compatibility, ignored.


key: name

description: define the Compose project name, until user defines one explicitly.


key: include

description: compose sub-projects to be included.


key: CodeGeneration

description: The 'CodeGeneration' object defines global properties that are used to drive the underlying entity-driven code generation.

enum: Common Business Autonomous

enum: Common Business All None

enum: etag eTag _etag _eTag ETag ETAG

enum: Business Common

enum: None DataSvc Data

enum: None Absolute Relative RelativeOrAbsolute

enum: NameOnly NameAndKey

enum: None Lower Upper


key: refDataNamespace

description: Defaults to 'Company' + '.' (literal) + AppName + '.' (literal) + 'EntityUsing' + '.Entities' (literal).


key: refDataCommonNamespace

description: Defaults to 'Company' + '.' (literal) + AppName + '.Common.Entities' (literal).


key: refDataText

description: This is used where serializing within the Web API 'Controller' and the 'ExecutionContext.IsRefDataTextSerializationEnabled' is set to 'true' (which is automatically set where the url contains '$text=true').


key: refDataCache

description: Defaults to 'ReferenceDataCache'. A value of 'ReferenceDataCache' specifies a single-tenant cache; otherwise, 'ReferenceDataMultiTenantCache' for a multi-tenant cache leverageing the 'ExecutionContext.TenantId'.


key: refDataAppendToNamespace

description: Defaults to 'null'; being nothing to append.


key: entityScope

description: Defaults to 'Common' for backwards compatibility; 'Autonomous' is **recommended**. Determines where the entity is scoped/defined, being 'Common' or 'Business' (i.e. not externally visible). Additionally, there is the special case of 'Autonomous' where both a 'Common' and 'Business' entity are generated (where only the latter inherits from 'EntityBase', etc).


key: entityUsing

description: Defaults to 'Common' (unless 'EntityScope' is 'Autonomous' and then it will default to 'Business') which will add '.Common.Entities'. Additionally, 'Business' to add '.Business.Entities', 'All' to add both, and 'None' to exclude any. This can be overridden for each 'Entity'.


key: jsonSerializer

description: Defaults to 'Newtonsoft'. This can be overridden within the 'Entity'(s).


key: etagJsonName

description: Defaults to 'etag'. Note that the 'JsonName' can be set individually per property where required.


description: Typically used where referening a 'Type' from a Namespace that is not generated by default.


key: webApiAuthorize

description: This can be overridden within the 'Entity'(s) and/or their corresponding 'Operation'(s).


key: webApiAutoLocation

description: This will automatically set the 'Operation.WebApiLocation' for an 'Operation' named 'Create' where there is a corresponding named 'Get'. This can be overridden within the 'Entity'(s).


key: webApiRoutePrefix

description: This is the base (prefix) 'URI' prepended to all entity and underlying 'Operation'(s).


key: validatorLayer

description: Defaults to 'Business'. A value of 'Business' indicates that the Validators will be defined within the 'Business' namespace/assembly; otherwise, defined within the 'Common' namespace/assembly.


key: databaseName

description: Defaults to 'IDatabase'. This can be overridden within the 'Entity'(s).


key: databaseSchema

description: Defaults to 'dbo'.


key: entityFrameworkName

description: Defaults to 'IEfDb'. This can be overridden within the 'Entity'(s).


key: cosmosName

description: Defaults to 'ICosmosDb'. This can be overridden within the 'Entity'(s).


key: odataName

description: Defaults to 'IOData'. This can be overridden within the 'Entity'(s).


key: httpAgentName

description: Defaults to 'IHttpAgent'. This can be overridden within the 'Entity'(s).


key: refDataDefaultMapperConverter

description: Defaults to 'ReferenceDataCodeConverter<T>'. Where this value is suffixed by '<T>' or '{T}' this will automatically set 'Property.DataConverterIsGeneric' to 'true'.


key: eventPublish

description: Defaults to 'DataSvc'; unless the 'EventOutbox' is not 'None' where it will default to 'Data'. Used to enable the sending of messages to the likes of EventHub, Service Broker, SignalR, etc. This can be overridden within the 'Entity'(s).


key: eventOutbox

description: Defaults to 'None'. A value of 'Database' will result in the 'DatabaseEventOutboxInvoker' being used to orchestrate.


key: eventSourceRoot

description: The event source is only updated where an 'EventSourceKind' is not 'None'. This can be extended within the 'Entity'(s).


key: eventSourceKind

description: Defaults to 'None' (being the event source is not updated).


key: eventSubjectRoot

description: Used to enable the sending of messages to the likes of EventHub, Service Broker, SignalR, etc. This can be overridden within the 'Entity'(s).


key: eventSubjectFormat

description: Defaults to 'NameAndKey' (being the event subject name appended with the corresponding unique key.)'.


key: eventSubjectSeparator

description: Defaults to '.'. Used only where the subject is automatically inferred.


key: eventActionFormat

description: Defaults to 'None' (no formatting required, i.e. as-is)'.


key: eventCasing

description: Defaults to 'None' (no casing required, i.e. as-is)'.


key: eventTransaction

description: Usage will force a rollback of any underlying data transaction (where the provider supports TransactionScope) on failure, such as an 'EventPublish' error. This is by no means implying a Distributed Transaction (DTC) should be invoked; this is only intended for a single data source that supports a TransactionScope to guarantee reliable event publishing. Defaults to 'false'. This essentially defaults the 'Entity.EventTransaction' where not otherwise specified. This should only be used where 'EventPublish' is 'DataSvc' and a transactionally-aware data source is being used.


key: grpc

description: gRPC support is an explicit opt-in model. Must be set to 'true' for any of the subordinate gRPC capabilities to be code-generated. Will require each 'Entity', and corresponding 'Property' and 'Operation' to be opted-in specifically.


key: pathBase

description: Defaults to 'Company' (runtime parameter) + '.' + 'AppName' (runtime parameter). For example 'Beef.Demo'.


key: pathCommon

description: Defaults to 'PathBase' + '.Common' (literal). For example 'Beef.Demo.Common'.


key: pathBusiness

description: Defaults to 'PathBase' + '.Business' (literal). For example 'Beef.Demo.Business'.


key: pathApi

description: Defaults to 'PathBase' + '.' + 'ApiName' (runtime parameter). For example 'Beef.Demo.Api'.


key: namespaceBase


key: namespaceCommon

description: Defaults to 'NamespaceBase' + '.Common' (literal). For example 'Beef.Demo.Common'.


key: namespaceBusiness

description: Defaults to 'NamespaceBase' + '.Business' (literal). For example 'Beef.Demo.Business'.


key: namespaceApi

description: Defaults to 'NamespaceBase' + '.' + 'ApiName' (runtime parameter). For example 'Beef.Demo.Api'.


key: Entity

description: The 'Entity' is used as the primary configuration for driving the entity-driven code generation.

enum: int long Guid string

enum: SortOrder Id Code Text

enum: Database EntityFramework Cosmos OData HttpAgent None

enum: Public Private Protected


key: text

description: Overrides the Name (as sentence text) for the summary comments. It will be formatted as: 'Represents the {Text} entity.'. To create a '<see cref="XXX"/>' within use moustache shorthand (e.g. {{Xxx}}).


key: fileName

description: Overrides the Name as the code-generated file name.


key: entityScope

description: Defaults to the 'CodeGeneration.EntityScope'. Determines where the entity is scoped/defined, being 'Common' or 'Business' (i.e. not externally visible). Additionally, there is a special case of 'Autonomous' where both a 'Common' and 'Business' entity are generated (where only the latter inherits from 'EntityBase', etc).


key: entityUsing

description: Defaults to 'EntityScope' ('Autonomous' will result in 'Business'). A value of 'Common' will add '.Common.Entities', 'Business' will add '.Business.Entities', 'All' to add both, and 'None' to exclude any.


key: privateName

description: Overrides the 'Name' to be used for private fields. By default reformatted from 'Name'; e.g. 'FirstName' as '_firstName'.


key: argumentName

description: Overrides the 'Name' to be used for argument parameters. By default reformatted from 'Name'; e.g. 'FirstName' as 'firstName'.


key: constType

description: The .NET Type to be used for the 'const' values. Defaults to 'string'.


key: isInitialOverride

description: Set to either 'true' or 'false' to override as specified; otherwise, 'null' to check each property. Defaults to 'null'.


key: refDataType

description: Required to identify an entity as being Reference Data. Specifies the underlying .NET Type used for the Reference Data identifier.


key: refDataText

description: This is used where serializing within the Web API'Controller' and the 'ExecutionContext.IsRefDataTextSerializationEnabled' is set to 'true' (which is automatically set where the url contains '$text=true').


key: refDataSortOrder

description: Specifies the default sort order for the underlying Reference Data collection. Defaults to 'SortOrder'.


key: refDataStringFormat

description: The string format supports the standard composite formatting; where the following indexes are used: '{0}' for 'Id', '{1}' for 'Code' and '{2}' for 'Text'. Defaults to '{2}'.


key: inherits

description: Defaults to 'EntityBase' for a standard entity. For Reference Data it will default to 'ReferenceDataBaseXxx' depending on the corresponding 'RefDataType' value. See 'OmitEntityBase' if the desired outcome is to not inherit from any of the aforementioned base classes.


key: implementsAutoInfer

description: Will attempt to infer the following: 'IGuidIdentifier', 'IInt32Identifier', 'IInt64Identifier', 'IStringIdentifier', 'IETag' and 'IChangeLog'. Defaults to 'true'.


key: namespace

description: Appended to the end of the standard structure as follows: '{Company}.{AppName}.Common.Entities.{Namespace}'.


key: omitEntityBase

description: As such any of the 'EntityBase' related capabilites are not supported (are omitted from generation). The intention for this is more for the generation of simple internal entities.


key: jsonSerializer

description: Defaults to the 'CodeGeneration.JsonSerializer' configuration property where specified; otherwise, 'Newtonsoft'.


key: collectionResult

description: Enables the likes of additional paging state to be stored with the underlying collection.


key: collectionInherits

description: Defaults to 'EntityBaseCollection' or 'EntityBaseKeyedCollection' depending on 'CollectionKeyed'. For Reference Data it will default to 'ReferenceDataCollectionBase'.


key: collectionResultInherits

description: Defaults to 'EntityCollectionResult'.


key: autoImplement

description: Defaults to 'None'. Indicates that the implementation for the underlying 'Operations' will be auto-implemented using the selected data source (unless explicity overridden). When selected some of the related attributes will also be required (as documented). Additionally, the 'AutoImplement' indicator must be selected for each underlying 'Operation' that is to be auto-implemented.


key: dataCtor

description: Defaults to 'Public'.


key: dataCtorParams

description: Each constructor parameter should be formatted as 'Type' + '^' + 'Name'; e.g. 'IConfiguration^Config'. Where the 'Name' portion is not specified it will be inferred. Where the 'Type' matches an already inferred value it will be ignored.


key: dataExtensions

description: This can be overridden using 'Operation.DataExtensions'.


key: databaseName

description: Defaults to the 'CodeGeneration.DatabaseName' configuration property (its default value is 'IDatabase').


key: databaseSchema

description: Defaults to 'dbo'.


key: databaseCustomerMapper

description: Otherwise, by default, a 'Mapper' will be generated.


key: entityFrameworkName

description: Defaults to the 'CodeGeneration.EntityFrameworkName' configuration property (its default value is 'IEfDb').


key: entityFrameworkMapperInheritsFrom

description: Defaults to 'Model.{Name}'; i.e. an entity with the same name in the 'Model' namespace.


key: entityFrameworkCustomMapper


key: cosmosName

description: Defaults to the 'CodeGeneration.CosmosName' configuration property (its default value is 'ICosmosDb').


key: cosmosPartitionKey

description: Defaults to 'PartitionKey.None'.


key: cosmosCustomMapper


key: odataName

description: Defaults to the 'CodeGeneration.ODataName' configuration property (its default value is 'IOData').


key: odataCollectionName

description: The underlying 'Simple.OData.Client' will attempt to infer.


key: odataCustomMapper


key: httpAgentName

description: Defaults to 'CodeGeneration.HttpAgentName' configuration property (its default value is 'IHttpAgent').


key: httpAgentRoutePrefix

description: This is the base (prefix) 'URI' for the HTTP Agent endpoint and can be further extended when defining the underlying 'Operation'(s).


key: httpAgentModel

description: This can be overridden within the 'Operation'(s).


key: httpAgentReturnModel


key: dataSvcCaching

description: Defaults to 'true'.


key: dataSvcCtor

description: Defaults to 'Public'.


key: dataSvcCtorParams


key: dataSvcExtensions

description: This can be overridden using 'Operation.DataSvcExtensions'.


key: eventPublish

description: Defaults to the 'CodeGeneration.EventPublish' configuration property (inherits) where not specified. Used to enable the sending of messages to the likes of EventGrid, Service Broker, SignalR, etc. This can be overridden within the 'Operation'(s).


key: eventOutbox

description: Defaults to 'CodeGeneration.EventOutbox' configuration property (inherits) where not specified. A value of 'Database' will result in the 'DatabaseEventOutboxInvoker' being used to orchestrate.


key: eventSource

description: Defaults to 'Name' (as lowercase) appended with the '/{$key}' placeholder. Note: when used in code-generation the 'CodeGeneration.EventSourceRoot' will be prepended where specified. To include the entity id/key include a '{$key}' placeholder ('Create', 'Update' or 'Delete' operation only); for example: 'person/{$key}'. This can be overridden for the 'Operation'.


key: eventSubjectFormat

description: Defaults to 'CodeGeneration.EventSubjectFormat'.


key: eventCasing

description: Defaults to 'CodeGeneration.EventCasing'.


key: eventTransaction

description: Usage will force a rollback of any underlying data transaction (where the provider supports TransactionScope) on failure, such as an 'EventPublish' error. This is by no means implying a Distributed Transaction (DTC) should be invoked; this is only intended for a single data source that supports a TransactionScope to guarantee reliable event publishing. Defaults to 'CodeGeneration.EventTransaction'. This essentially defaults the 'Operation.DataSvcTransaction' where not otherwise specified. This should only be used where 'EventPublish' is 'DataSvc' and a transactionally-aware data source is being used.


key: managerCtor

description: Defaults to 'Public'.


key: managerCtorParams

description: Each constructor parameter should be formatted as 'Type' + '^' + 'Name'; e.g. 'IConfiguration^Config'. Where the 'Name' portion is not specified it will be inferred. Where the 'Type' matches an already inferred value it will be ignored.


key: managerExtensions

description: This can be overridden using 'Operation.ManagerExtensions'.


key: validator

description: Only used for defaulting the 'Create' and 'Update' operation types ('Operation.Type') where not specified explicitly.


key: iValidator


key: webApiRoutePrefix

description: This is the base (prefix) 'URI' for the entity and can be further extended when defining the underlying 'Operation'(s). The 'CodeGeneration.WebApiRoutePrefix' will be prepended where specified.


key: webApiAuthorize

description: Defaults to the 'CodeGeneration.WebApiAuthorize' configuration property (inherits) where not specified; can be overridden at the 'Operation' level also.


key: webApiCtor

description: Defaults to 'Public'.


key: webApiCtorParams


key: webApiAutoLocation

description: This will automatically set the 'Operation.WebApiLocation' for an 'Operation' named 'Create' where there is a corresponding named 'Get'. This is defaulted from the 'CodeGen.WebApiAutoLocation'.


key: webApiConcurrency

description: This provides an alternative where the underlying data source does not natively support optimistic concurrency (native support should always be leveraged as a priority). Where the 'Operation.Type' is 'Update' or 'Patch', the request ETag will be matched against the response for a corresponding 'Get' operation to verify no changes have been made prior to updating. For this to function correctly the .NET response Type for the 'Get' must be the same as that returned from the corresponding 'Create', 'Update' and 'Patch' (where applicable) as the generated ETag is a SHA256 hash of the resulting JSON. This defaults the 'Operation.WebApiConcurrency'.


key: webApiGetOperation

description: Defaults to 'Get'. Specify either just the method name (e.g. 'OperationName') or, interface and method name (e.g. 'IXxxManager.OperationName') to be invoked where in a different 'YyyManager.OperationName'.


key: dataModel

description: The model will be generated with 'OmitEntityBase = true'. Any reference data properties will be defined using their 'RefDataType' intrinsic 'Type' versus their corresponding (actual) reference data 'Type'.


key: excludeAll

description: Is a shorthand means for setting all of the other 'Exclude*' properties (with the exception of 'ExcludeEntity') to exclude.


key: excludeData

description: Defaults to 'Include' indicating _not_ to exlude. A value of 'Exclude' indicates to exclude all output; alternatively, 'RequiresMapper' indicates to at least output the corresponding 'Mapper' class.

enum: Include Exclude RequiresMapper


key: authRole

description: Used where not overridden specifically for an 'Operation'; i.e. acts as the default.


key: grpc

description: gRPC support is an explicit opt-in model (see 'CodeGeneration.Grpc' configuration); therefore, each corresponding 'Property' and 'Operation' will also need to be opted-in specifically.


key: Property

description: The 'Property' object defines an 'Entity' property and its charateristics.

enum: UseDefault None DateOnly DateTimeLocal DateTimeUtc DateTimeUnspecified

enum: UseDefault None Start End Both

enum: UseDefault None NullToEmpty EmptyToNull

enum: string int Guid

enum: Any AnyExceptCreate AnyExceptUpdate AnyExceptGet Get Create Update Delete

enum: Map Ignore Skip


key: text

description: By default the 'Text' will be the 'Name' reformatted as sentence casing. Depending on whether the 'Type' is 'bool', will appear in one of the two generated sentences. Where not 'bool' it will be: Gets or sets a value indicating whether {text}.'. Otherwise, it will be: Gets or sets the {text}.'. To create a '<see cref="XXX"/>' within use moustache shorthand (e.g. {{Xxx}}).


key: modelText

description: By default the 'ModelText' will be the 'Name' reformatted as sentence casing. Depending on whether the 'Type' is 'bool', will appear in one of the two generated sentences. Where not 'bool' it will be: Gets or sets a value indicating whether {text}.'. Otherwise, it will be: Gets or sets the {text}.'. To create a '<see cref="XXX"/>' within use moustache shorthand (e.g. {{Xxx}}).


key: type

description: Defaults to 'string'. To reference a Reference Data 'Type' always prefix with 'RefDataNamespace' (e.g. 'RefDataNamespace.Gender') or shortcut '^' (e.g. '^Gender'). This will ensure that the appropriate Reference Data 'using' statement is used. _Shortcut:_ Where the 'Type' starts with (prefix) 'RefDataNamespace.' or '^', and the correspondong 'RefDataType' attribute is not specified it will automatically default the 'RefDataType' to 'string.'


key: privateName

description: Overrides the 'Name' to be used for private fields. By default reformatted from 'Name'; e.g. 'FirstName' as '_firstName'.


key: argumentName

description: Overrides the 'Name' to be used for argument parameters. By default reformatted from 'Name'; e.g. 'FirstName' as 'firstName'.


key: uniqueKey

description: This is also used to simplify the parameter specification for an Entity Operation by inferrence.


key: isEntity

description: Will be inferred (default to 'true') where the 'Type' is 'ChangeLog' or the 'Type' is found as another 'Entity' within the code-generation configuration file.


key: dateTimeTransform

description: Defaults to 'UseDefault'. This is only applied where the 'Type' is 'DateTime'.


key: stringTrim

description: Defaults to 'UseDefault'. This is only applied where the 'Type' is 'string'.


key: stringTransform


key: default

description: Where the 'Type' is 'string' then the specified default value will need to be delimited. Any valid value assignment C# code can be used.


key: partitionKey

description: This will implement 'IPartitionKey' for the generated entity.


key: bubblePropertyChanges

description: Note that the 'IsEntity' property is also required to enable.


key: internalOnly

description: This is only applicable where the 'Entity.EntityScope' is 'Autonomous'. In this instance the 'Property' will be excluded from the 'Common' entity declaration.


key: refDataType

description: Defaults to 'string' (being the 'ReferenceDataBase.Code') where not specified and the corresponding 'Type' starts with (prefix) 'RefDataNamespace.' or '^'. Note: an 'Id' of type 'string' is currently not supported; the use of the 'Code' is the recommended approach.