Data Ingestion

After running the analytics create-template command, you can configure the analytics pipeline run by editing the /data/conf/analytics_init_config.yml file.

Autonomous Identity supports data ingestion for three types of Identity Governance systems: general csv files, iiq for Sailpoint IdentityIQ files, and oim for Oracle Identity Manager files. You can configure the /data/conf/analytics_init_config.yml file for iiq or oim files.

Data Ingestion

Field	Sample	Description
ingestion:
drop_if_create:	True/False	Not used by these connectors
catalog_step	True/False	Creates a csv file with possible attributes from the client system. If set to `True`, the data pull does not occur. This property only needs to be set true in the first ingestion run to identify possible attributes.
staging:	True/False	Determines whether to pull data to `autoid_staging` or `autoid_base` keyspaces. If `staging` is set to `True`, the system pulls to `autoid_staging` keyspace; while set to `False`, it pulls to `autoid_base` keyspace. The `autoid_staging` keyspace lets us pull data without on-the-fly transformations, while also supporting pulling from multiple sources.
connector:
type:	`csv`, `iiq`, `oim`	Connector for type of source data. Currently only supporting three options.
batchsize:	1000	Integer. Specifies the number of rows or pages pulled as part of each batch.
timeout:	10	Integer representing seconds. Specifies the amount of time to wait before timeout.
change_reconciliation:
enabled:	True/False	Determines whether to pull data that filters only results that been modified after a certain date, specified by the `time` property.
time:	'2013-05-17T00:00:00Z','2020-05-17 00:00:00'	Specifies the data-time after which the data should be pulled. Note IIQ requires a time zone (first value).
features_mappings:
chief_yes_no	'User:CHIEF_YES_NO'	Value take from the csv generated in the catalog step. Note that `chief_yes_no` is not required, but is included here as an example. For reference, see Appendix A: The analytics_init_config.yml File.