gcp_datastream_stream
ancestors
Type: UNORDERED_LIST_STRING
backfill_all
Type: STRUCT
Provider name: backfillAll
Description: Automatically backfill objects included in the stream source configuration. Specific objects can be excluded.
mongodb_excluded_objects
Type: STRUCT
Provider name: mongodbExcludedObjects
Description: MongoDB data source objects to avoid backfilling
databases
Type: UNORDERED_LIST_STRUCT
Provider name: databases
Description: MongoDB databases in the cluster.
collections
Type: UNORDERED_LIST_STRUCT
Provider name: collections
Description: Collections in the database.
collection
Type: STRING
Provider name: collection
Description: Collection name.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Fields in the collection.
field
Type: STRING
Provider name: field
Description: Field name.
database
Type: STRING
Provider name: database
Description: Database name.
mysql_excluded_objects
Type: STRUCT
Provider name: mysqlExcludedObjects
Description: MySQL data source objects to avoid backfilling.
mysql_databases
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlDatabases
Description: Mysql databases on the server
database
Type: STRING
Provider name: database
Description: Database name.
mysql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlTables
Description: Tables in the database.
mysql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlColumns
Description: MySQL columns in the database. When unspecified as part of include/exclude objects, includes/excludes everything.
collation
Type: STRING
Provider name: collation
Description: Column collation.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The MySQL data type. Full data types list can be found here: https://dev.mysql.com/doc/refman/8.0/en/data-types.html
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
oracle_excluded_objects
Type: STRUCT
Provider name: oracleExcludedObjects
Description: Oracle data source objects to avoid backfilling.
oracle_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: oracleSchemas
Description: Oracle schemas/databases in the database server.
oracle_tables
Type: UNORDERED_LIST_STRUCT
Provider name: oracleTables
Description: Tables in the schema.
oracle_columns
Type: UNORDERED_LIST_STRUCT
Provider name: oracleColumns
Description: Oracle columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The Oracle data type.
encoding
Type: STRING
Provider name: encoding
Description: Column encoding.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
postgresql_excluded_objects
Type: STRUCT
Provider name: postgresqlExcludedObjects
Description: PostgreSQL data source objects to avoid backfilling.
postgresql_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlSchemas
Description: PostgreSQL schemas in the database server.
postgresql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlTables
Description: Tables in the schema.
postgresql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlColumns
Description: PostgreSQL columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The PostgreSQL data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
salesforce_excluded_objects
Type: STRUCT
Provider name: salesforceExcludedObjects
Description: Salesforce data source objects to avoid backfilling
objects
Type: UNORDERED_LIST_STRUCT
Provider name: objects
Description: Salesforce objects in the database server.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Salesforce fields. When unspecified as part of include objects, includes everything, when unspecified as part of exclude objects, excludes nothing.
data_type
Type: STRING
Provider name: dataType
Description: The data type.
name
Type: STRING
Provider name: name
Description: Field name.
nillable
Type: BOOLEAN
Provider name: nillable
Description: Indicates whether the field can accept nil values.
object_name
Type: STRING
Provider name: objectName
Description: Object name.
sql_server_excluded_objects
Type: STRUCT
Provider name: sqlServerExcludedObjects
Description: SQLServer data source objects to avoid backfilling
schemas
Type: UNORDERED_LIST_STRUCT
Provider name: schemas
Description: SQLServer schemas in the database server.
schema
Type: STRING
Provider name: schema
Description: Schema name.
tables
Type: UNORDERED_LIST_STRUCT
Provider name: tables
Description: Tables in the schema.
columns
Type: UNORDERED_LIST_STRUCT
Provider name: columns
Description: SQLServer columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The SQLServer data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
backfill_none
Type: STRUCT
Provider name: backfillNone
Description: Do not automatically backfill any objects.
create_time
Type: TIMESTAMP
Provider name: createTime
Description: Output only. The creation time of the stream.
customer_managed_encryption_key
Type: STRING
Provider name: customerManagedEncryptionKey
Description: Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.
destination_config
Type: STRUCT
Provider name: destinationConfig
Description: Required. Destination connection profile configuration.
bigquery_destination_config
Type: STRUCT
Provider name: bigqueryDestinationConfig
Description: BigQuery destination configuration.
append_only
Type: STRUCT
Provider name: appendOnly
Description: Append only mode
blmt_config
Type: STRUCT
Provider name: blmtConfig
Description: Optional. Big Lake Managed Tables (BLMT) configuration.
bucket
Type: STRING
Provider name: bucket
Description: Required. The Cloud Storage bucket name.
connection_name
Type: STRING
Provider name: connectionName
Description: Required. The bigquery connection. Format: {project}.{location}.{name}
file_format
Type: STRING
Provider name: fileFormat
Description: Required. The file format.
Possible values:
FILE_FORMAT_UNSPECIFIED - Default value.
PARQUET - Parquet file format.
root_path
Type: STRING
Provider name: rootPath
Description: The root path inside the Cloud Storage bucket.
table_format
Type: STRING
Provider name: tableFormat
Description: Required. The table format.
Possible values:
TABLE_FORMAT_UNSPECIFIED - Default value.
ICEBERG - Iceberg table format.
data_freshness
Type: STRING
Provider name: dataFreshness
Description: The guaranteed data freshness (in seconds) when querying tables created by the stream. Editing this field will only affect new tables created in the future, but existing tables will not be impacted. Lower values mean that queries will return fresher data, but may result in higher cost.
merge
Type: STRUCT
Provider name: merge
Description: The standard mode
single_target_dataset
Type: STRUCT
Provider name: singleTargetDataset
Description: Single destination dataset.
source_hierarchy_datasets
Type: STRUCT
Provider name: sourceHierarchyDatasets
Description: Source hierarchy datasets.
dataset_template
Type: STRUCT
Provider name: datasetTemplate
Description: The dataset template to use for dynamic dataset creation.
dataset_id_prefix
Type: STRING
Provider name: datasetIdPrefix
Description: If supplied, every created dataset will have its name prefixed by the provided value. The prefix and name will be separated by an underscore. i.e. _.
kms_key_name
Type: STRING
Provider name: kmsKeyName
Description: Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key. i.e. projects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{cryptoKey}. See https://cloud.google.com/bigquery/docs/customer-managed-encryption for more information.
location
Type: STRING
Provider name: location
Description: Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
destination_connection_profile
Type: STRING
Provider name: destinationConnectionProfile
Description: Required. Destination connection profile resource. Format: projects/{project}/locations/{location}/connectionProfiles/{name}
gcs_destination_config
Type: STRUCT
Provider name: gcsDestinationConfig
Description: A configuration for how data should be loaded to Cloud Storage.
avro_file_format
Type: STRUCT
Provider name: avroFileFormat
Description: AVRO file format configuration.
file_rotation_interval
Type: STRING
Provider name: fileRotationInterval
Description: The maximum duration for which new events are added before a file is closed and a new file is created. Values within the range of 15-60 seconds are allowed.
file_rotation_mb
Type: INT32
Provider name: fileRotationMb
Description: The maximum file size to be saved in the bucket.
json_file_format
Type: STRUCT
Provider name: jsonFileFormat
Description: JSON file format configuration.
compression
Type: STRING
Provider name: compression
Description: Compression of the loaded JSON file.
Possible values:
JSON_COMPRESSION_UNSPECIFIED - Unspecified json file compression.
NO_COMPRESSION - Do not compress JSON file.
GZIP - Gzip compression.
schema_file_format
Type: STRING
Provider name: schemaFileFormat
Description: The schema file format along JSON data files.
Possible values:
SCHEMA_FILE_FORMAT_UNSPECIFIED - Unspecified schema file format.
NO_SCHEMA_FILE - Do not attach schema file.
AVRO_SCHEMA_FILE - Avro schema format.
path
Type: STRING
Provider name: path
Description: Path inside the Cloud Storage bucket to write data to.
errors
Type: UNORDERED_LIST_STRUCT
Provider name: errors
Description: Output only. Errors on the Stream.
error_time
Type: TIMESTAMP
Provider name: errorTime
Description: The time when the error occurred.
error_uuid
Type: STRING
Provider name: errorUuid
Description: A unique identifier for this specific error, allowing it to be traced throughout the system in logs and API responses.
message
Type: STRING
Provider name: message
Description: A message containing more information about the error that occurred.
reason
Type: STRING
Provider name: reason
Description: A title that explains the reason for the error.
gcp_display_name
Type: STRING
Provider name: displayName
Description: Required. Display name.
labels
Type: UNORDERED_LIST_STRING
last_recovery_time
Type: TIMESTAMP
Provider name: lastRecoveryTime
Description: Output only. If the stream was recovered, the time of the last recovery. Note: This field is currently experimental.
name
Type: STRING
Provider name: name
Description: Output only. Identifier. The stream’s name.
organization_id
Type: STRING
parent
Type: STRING
project_id
Type: STRING
project_number
Type: STRING
resource_name
Type: STRING
satisfies_pzi
Type: BOOLEAN
Provider name: satisfiesPzi
Description: Output only. Reserved for future use.
satisfies_pzs
Type: BOOLEAN
Provider name: satisfiesPzs
Description: Output only. Reserved for future use.
source_config
Type: STRUCT
Provider name: sourceConfig
Description: Required. Source connection profile configuration.
mongodb_source_config
Type: STRUCT
Provider name: mongodbSourceConfig
Description: MongoDB data source configuration.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: MongoDB collections to exclude from the stream.
databases
Type: UNORDERED_LIST_STRUCT
Provider name: databases
Description: MongoDB databases in the cluster.
collections
Type: UNORDERED_LIST_STRUCT
Provider name: collections
Description: Collections in the database.
collection
Type: STRING
Provider name: collection
Description: Collection name.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Fields in the collection.
field
Type: STRING
Provider name: field
Description: Field name.
database
Type: STRING
Provider name: database
Description: Database name.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: MongoDB collections to include in the stream.
databases
Type: UNORDERED_LIST_STRUCT
Provider name: databases
Description: MongoDB databases in the cluster.
collections
Type: UNORDERED_LIST_STRUCT
Provider name: collections
Description: Collections in the database.
collection
Type: STRING
Provider name: collection
Description: Collection name.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Fields in the collection.
field
Type: STRING
Provider name: field
Description: Field name.
database
Type: STRING
Provider name: database
Description: Database name.
mysql_source_config
Type: STRUCT
Provider name: mysqlSourceConfig
Description: MySQL data source configuration.
binary_log_position
Type: STRUCT
Provider name: binaryLogPosition
Description: Use Binary log position based replication.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: MySQL objects to exclude from the stream.
mysql_databases
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlDatabases
Description: Mysql databases on the server
database
Type: STRING
Provider name: database
Description: Database name.
mysql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlTables
Description: Tables in the database.
mysql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlColumns
Description: MySQL columns in the database. When unspecified as part of include/exclude objects, includes/excludes everything.
collation
Type: STRING
Provider name: collation
Description: Column collation.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The MySQL data type. Full data types list can be found here: https://dev.mysql.com/doc/refman/8.0/en/data-types.html
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
gtid
Type: STRUCT
Provider name: gtid
Description: Use GTID based replication.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: MySQL objects to retrieve from the source.
mysql_databases
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlDatabases
Description: Mysql databases on the server
database
Type: STRING
Provider name: database
Description: Database name.
mysql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlTables
Description: Tables in the database.
mysql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: mysqlColumns
Description: MySQL columns in the database. When unspecified as part of include/exclude objects, includes/excludes everything.
collation
Type: STRING
Provider name: collation
Description: Column collation.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The MySQL data type. Full data types list can be found here: https://dev.mysql.com/doc/refman/8.0/en/data-types.html
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
max_concurrent_backfill_tasks
Type: INT32
Provider name: maxConcurrentBackfillTasks
Description: Maximum number of concurrent backfill tasks. The number should be non negative. If not set (or set to 0), the system’s default value will be used.
max_concurrent_cdc_tasks
Type: INT32
Provider name: maxConcurrentCdcTasks
Description: Maximum number of concurrent CDC tasks. The number should be non negative. If not set (or set to 0), the system’s default value will be used.
oracle_source_config
Type: STRUCT
Provider name: oracleSourceConfig
Description: Oracle data source configuration.
binary_log_parser
Type: STRUCT
Provider name: binaryLogParser
Description: Use Binary Log Parser.
log_file_directories
Type: STRUCT
Provider name: logFileDirectories
Description: Use Oracle directories.
archived_log_directory
Type: STRING
Provider name: archivedLogDirectory
Description: Required. Oracle directory for archived logs.
online_log_directory
Type: STRING
Provider name: onlineLogDirectory
Description: Required. Oracle directory for online logs.
oracle_asm_log_file_access
Type: STRUCT
Provider name: oracleAsmLogFileAccess
Description: Use Oracle ASM.
drop_large_objects
Type: STRUCT
Provider name: dropLargeObjects
Description: Drop large object values.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: Oracle objects to exclude from the stream.
oracle_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: oracleSchemas
Description: Oracle schemas/databases in the database server.
oracle_tables
Type: UNORDERED_LIST_STRUCT
Provider name: oracleTables
Description: Tables in the schema.
oracle_columns
Type: UNORDERED_LIST_STRUCT
Provider name: oracleColumns
Description: Oracle columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The Oracle data type.
encoding
Type: STRING
Provider name: encoding
Description: Column encoding.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: Oracle objects to include in the stream.
oracle_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: oracleSchemas
Description: Oracle schemas/databases in the database server.
oracle_tables
Type: UNORDERED_LIST_STRUCT
Provider name: oracleTables
Description: Tables in the schema.
oracle_columns
Type: UNORDERED_LIST_STRUCT
Provider name: oracleColumns
Description: Oracle columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The Oracle data type.
encoding
Type: STRING
Provider name: encoding
Description: Column encoding.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
log_miner
Type: STRUCT
Provider name: logMiner
Description: Use LogMiner.
max_concurrent_backfill_tasks
Type: INT32
Provider name: maxConcurrentBackfillTasks
Description: Maximum number of concurrent backfill tasks. The number should be non-negative. If not set (or set to 0), the system’s default value is used.
max_concurrent_cdc_tasks
Type: INT32
Provider name: maxConcurrentCdcTasks
Description: Maximum number of concurrent CDC tasks. The number should be non-negative. If not set (or set to 0), the system’s default value is used.
stream_large_objects
Type: STRUCT
Provider name: streamLargeObjects
Description: Stream large object values.
postgresql_source_config
Type: STRUCT
Provider name: postgresqlSourceConfig
Description: PostgreSQL data source configuration.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: PostgreSQL objects to exclude from the stream.
postgresql_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlSchemas
Description: PostgreSQL schemas in the database server.
postgresql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlTables
Description: Tables in the schema.
postgresql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlColumns
Description: PostgreSQL columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The PostgreSQL data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: PostgreSQL objects to include in the stream.
postgresql_schemas
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlSchemas
Description: PostgreSQL schemas in the database server.
postgresql_tables
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlTables
Description: Tables in the schema.
postgresql_columns
Type: UNORDERED_LIST_STRUCT
Provider name: postgresqlColumns
Description: PostgreSQL columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The PostgreSQL data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
schema
Type: STRING
Provider name: schema
Description: Schema name.
max_concurrent_backfill_tasks
Type: INT32
Provider name: maxConcurrentBackfillTasks
Description: Maximum number of concurrent backfill tasks. The number should be non negative. If not set (or set to 0), the system’s default value will be used.
publication
Type: STRING
Provider name: publication
Description: Required. The name of the publication that includes the set of all tables that are defined in the stream’s include_objects.
replication_slot
Type: STRING
Provider name: replicationSlot
Description: Required. Immutable. The name of the logical replication slot that’s configured with the pgoutput plugin.
salesforce_source_config
Type: STRUCT
Provider name: salesforceSourceConfig
Description: Salesforce data source configuration.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: Salesforce objects to exclude from the stream.
objects
Type: UNORDERED_LIST_STRUCT
Provider name: objects
Description: Salesforce objects in the database server.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Salesforce fields. When unspecified as part of include objects, includes everything, when unspecified as part of exclude objects, excludes nothing.
data_type
Type: STRING
Provider name: dataType
Description: The data type.
name
Type: STRING
Provider name: name
Description: Field name.
nillable
Type: BOOLEAN
Provider name: nillable
Description: Indicates whether the field can accept nil values.
object_name
Type: STRING
Provider name: objectName
Description: Object name.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: Salesforce objects to retrieve from the source.
objects
Type: UNORDERED_LIST_STRUCT
Provider name: objects
Description: Salesforce objects in the database server.
fields
Type: UNORDERED_LIST_STRUCT
Provider name: fields
Description: Salesforce fields. When unspecified as part of include objects, includes everything, when unspecified as part of exclude objects, excludes nothing.
data_type
Type: STRING
Provider name: dataType
Description: The data type.
name
Type: STRING
Provider name: name
Description: Field name.
nillable
Type: BOOLEAN
Provider name: nillable
Description: Indicates whether the field can accept nil values.
object_name
Type: STRING
Provider name: objectName
Description: Object name.
polling_interval
Type: STRING
Provider name: pollingInterval
Description: Required. Salesforce objects polling interval. The interval at which new changes will be polled for each object. The duration must be between 5 minutes and 24 hours.
source_connection_profile
Type: STRING
Provider name: sourceConnectionProfile
Description: Required. Source connection profile resource. Format: projects/{project}/locations/{location}/connectionProfiles/{name}
sql_server_source_config
Type: STRUCT
Provider name: sqlServerSourceConfig
Description: SQLServer data source configuration.
change_tables
Type: STRUCT
Provider name: changeTables
Description: CDC reader reads from change tables.
exclude_objects
Type: STRUCT
Provider name: excludeObjects
Description: SQLServer objects to exclude from the stream.
schemas
Type: UNORDERED_LIST_STRUCT
Provider name: schemas
Description: SQLServer schemas in the database server.
schema
Type: STRING
Provider name: schema
Description: Schema name.
tables
Type: UNORDERED_LIST_STRUCT
Provider name: tables
Description: Tables in the schema.
columns
Type: UNORDERED_LIST_STRUCT
Provider name: columns
Description: SQLServer columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The SQLServer data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
include_objects
Type: STRUCT
Provider name: includeObjects
Description: SQLServer objects to include in the stream.
schemas
Type: UNORDERED_LIST_STRUCT
Provider name: schemas
Description: SQLServer schemas in the database server.
schema
Type: STRING
Provider name: schema
Description: Schema name.
tables
Type: UNORDERED_LIST_STRUCT
Provider name: tables
Description: Tables in the schema.
columns
Type: UNORDERED_LIST_STRUCT
Provider name: columns
Description: SQLServer columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.
column
Type: STRING
Provider name: column
Description: Column name.
data_type
Type: STRING
Provider name: dataType
Description: The SQLServer data type.
length
Type: INT32
Provider name: length
Description: Column length.
nullable
Type: BOOLEAN
Provider name: nullable
Description: Whether or not the column can accept a null value.
ordinal_position
Type: INT32
Provider name: ordinalPosition
Description: The ordinal position of the column in the table.
precision
Type: INT32
Provider name: precision
Description: Column precision.
primary_key
Type: BOOLEAN
Provider name: primaryKey
Description: Whether or not the column represents a primary key.
scale
Type: INT32
Provider name: scale
Description: Column scale.
table
Type: STRING
Provider name: table
Description: Table name.
max_concurrent_backfill_tasks
Type: INT32
Provider name: maxConcurrentBackfillTasks
Description: Max concurrent backfill tasks.
max_concurrent_cdc_tasks
Type: INT32
Provider name: maxConcurrentCdcTasks
Description: Max concurrent CDC tasks.
transaction_logs
Type: STRUCT
Provider name: transactionLogs
Description: CDC reader reads from transaction logs.
state
Type: STRING
Provider name: state
Description: The state of the stream.
Possible values:
STATE_UNSPECIFIED - Unspecified stream state.
NOT_STARTED - The stream has been created but has not yet started streaming data.
RUNNING - The stream is running.
PAUSED - The stream is paused.
MAINTENANCE - The stream is in maintenance mode. Updates are rejected on the resource in this state.
FAILED - The stream is experiencing an error that is preventing data from being streamed.
FAILED_PERMANENTLY - The stream has experienced a terminal failure.
STARTING - The stream is starting, but not yet running.
DRAINING - The Stream is no longer reading new events, but still writing events in the buffer.
Type: UNORDERED_LIST_STRING
update_time
Type: TIMESTAMP
Provider name: updateTime
Description: Output only. The last update time of the stream.
