Data Ingestion

After running the analytics create-template command, you can configure the analytics pipeline run by editing the /data/conf/analytics_init_config.yml file.

Autonomous Identity supports data ingestion for three types of Identity Governance systems: general csv files, iiq for Sailpoint IdentityIQ files, and oim for Oracle Identity Manager files. You can configure the /data/conf/analytics_init_config.yml file for iiq or oim files.

Data Ingestion
FieldSampleDescription
ingestion:
drop_if_create:True/FalseNot used by these connectors
catalog_stepTrue/False

Creates a csv file with possible attributes from the client system. If set to True, the data pull does not occur. This property only needs to be set true in the first ingestion run to identify possible attributes.

staging:True/False

Determines whether to pull data to autoid_staging or autoid_base keyspaces. If staging is set to True, the system pulls to autoid_staging keyspace; while set to False, it pulls to autoid_base keyspace. The autoid_staging keyspace lets us pull data without on-the-fly transformations, while also supporting pulling from multiple sources.

connector:
type:csv, iiq, oim

Connector for type of source data. Currently only supporting three options.

batchsize:1000

Integer. Specifies the number of rows or pages pulled as part of each batch.

timeout:10

Integer representing seconds. Specifies the amount of time to wait before timeout.

change_reconciliation:
enabled:True/False

Determines whether to pull data that filters only results that been modified after a certain date, specified by the time property.

time:'2013-05-17T00:00:00Z','2020-05-17 00:00:00'

Specifies the data-time after which the data should be pulled. Note IIQ requires a time zone (first value).

features_mappings:
chief_yes_no'User:CHIEF_YES_NO'

Value take from the csv generated in the catalog step. Note that chief_yes_no is not required, but is included here as an example. For reference, see Appendix A: The analytics_init_config.yml File.


Read a different version of :