Autonomous Identity 2021.8.2

Set Data Sources

After defining any new attributes, you must set your data sources, so that Autonomous Identity can import and ingest your data. Autonomous Identity supports three types of data source files:

  • Comma-separated values (CSV). A comma-separated values (CSV) file is a text file that uses a comma delimiter to separate each field value. Each line of text represents a record, consisting of one or more fields of data.

  • Java Database Connectivity (JDBC). Java Database Connectivity (JDBC) is a Java API that connects to and executes queries on databases, like Oracle, MySQL, PostgreSQL, and others.

  • Generic. Generic data sources are those data types from vendors that have neither CSV nor JDBC-based formats, such as JSON, or others.

Data Source Sync Types

Autonomous Identity supports partial or incremental data ingestion for faster and efficient data uploads. The four types are full, incremental, enrichment, and delete, and are summarized below:

Table: Summary of Data Sync Types
Sync Type Data Source In AutoID Result

Full

The records from the entity represents the full set of all records that you intend to ingest. For example:

  • 0, "amy.user"

  • 1, "bob.user"

An existing table may have the following:

  • 2, "walt.user"

  • 3, "kelly.user"

After the ingest job runs, all existing records are fully replaced:

  • 0, "amy.user"

  • 1, "bob.user"

Incremental

The records from the entity represents the records that you want to add to AutoID. For example:

  • 2, "walt.user"

  • 3, "kelly.user"

An existing table may have the following:

  • 0, "amy.user"

  • 1, "bob.user"

After the ingest job runs, the records in the data source are added to the existing records:

  • 0, "amy.user"

  • 1, "bob.user"

  • 2, "walt.user"

  • 3, "kelly.user"

Enrichment

The records from the entity represents changes to existing data, such as adding a department attribute. No new objects are added, but here you want to edit or "patch" in new attributes to existing records:

  • 0, "finance"

  • 1, "finance"

  • 2, "finance"

An existing table may have the following:

  • 0, "amy.user"

  • 1, "bob.user"

  • 2, "walt.user"

  • 3, "kelly.user"

After the ingest job runs, the attributes in the data source is added to the existing records. If attributes exist, they get updated. If attributes do not exist, they do not get updated, but you can add also attributes using mappings:

  • 0, "amy.user", "finance"

  • 1, "bob.user", "finance"

  • 2, "walt.user", "finance"

  • 3, "kelly.user"

Delete

The records from the entity represent records to be deleted, identified by the primary key:

  • 3, "kelly.user"

An existing table may have the following:

  • 0, "amy.user", "finance"

  • 1, "bob.user", "finance"

  • 2, "walt.user", "finance"

  • 3, "kelly.user"

After the ingest job completes, the records with the primary key are deleted:

  • 0, "amy.user", "finance"

  • 1, "bob.user", "finance"

  • 2, "walt.user", "finance"

CSV Data Sources

  1. Log in to the Autonomous Identity UI as an administrator.

  2. On the Autonomous Identity UI, click the Administration icon.

  3. Click Data Sources.

  4. Click the Add data source button.

  5. In the Add Data Source dialog box, click CSV, and then click Next.

  6. In the CSV Details dialog box, enter a human-readable name for your CSV file.

  7. Select the Sync Type. The options are as follows:

    • Full. Runs a full replacement of data if any.

    • Incremental. Adds new records to existing data.

    • Enrichment. Adds new attributes to existing data records.

    • Delete. Delete any existing data objects.

  8. Click Add Object, and then select the data source file.

    1. Click Applications, enter the path to the application.csv file. For example, /data/input/applications.csv.

    2. Click Assignments, enter the path to the assignments.csv file. For example, /data/input/assignments.csv.

    3. Click Entitlements, enter the path to the entitlements.csv file. For example, /data/input/entitlements.csv.

    4. Click Identities, enter the path to the identities.csv file. For example, /data/input/identities.csv.

  9. Click Save.

    See it in action
    set datasources csv

  10. Repeat the previous steps to add more CSV data source files if needed.

  11. Next, you must set the attribute mappings. This is a critical step to ensure a successful analytics run. See Set Attribute Mappings.

JDBC Data Sources

The following procedure assumes that you have set up Autonomous Identity with connectivity to a database, such as Oracle, MySQL, or PostgreSQL.

  1. Log in to the Autonomous Identity UI as an administrator.

  2. On the Autonomous Identity UI, click the Administration icon.

  3. Click Data Sources.

  4. Click the Add data source button.

  5. In the Add Data Source dialog box, click JDBC, and then click Next.

  6. In the JDBC Details dialog box, enter a human-readable name for your JDBC files.

  7. Select the Sync Type. The options are as follows:

    • Full. Runs a full replacement of data if any.

    • Incremental. Adds new records to existing data.

    • Enrichment. Adds new attributes to existing data records.

    • Delete. Delete any existing data objects.

  8. For Connection Settings, enter the following:

    1. Database Username. Enter a user name for the database user that connects to the data source.

    2. Database Password. Enter a password for the database user.

    3. Database Driver. Select the database driver. Options are: Mysql, Postgresql, and Oracle.

    4. Datbase Connect String. Enter the database connect string to the data source.

  9. Click Add Object, and then select the data source file.

    1. Click Applications, enter the path to the APPLICATIONS table. For example, select * from APPLICATIONS.

    2. Click Assignments, enter the path to the ASSIGNMENTS table. For example, select * from ASSIGNMENTS.

    3. Click Entitlements, enter the path to the ENTITLEMENTS table. For example, select * from ENTITLEMENTS.

    4. Click Identities, enter the path to the IDENTITIES table. For example, select * from IDENTITIES.

  10. Click Save.

    See it in action
    data sources jdbc

  11. Repeat the previous steps to add more JDBC data source files if necessary.

  12. Next, you must set the attribute mappings. This is a critical step to ensure a successful analytics run. See Set Attribute Mappings.

Generic Data Sources

  1. Log in to the Autonomous Identity UI as an administrator.

  2. On the Autonomous Identity UI, click the Administration icon.

  3. Click Data Sources.

  4. Click the Add data source button.

  5. In the Add Data Source dialog box, click Generic, and then click Next.

  6. In the Generic Details dialog box, enter a human-readable name for your generic files.

  7. Select the Sync Type. The options are as follows:

    • Full. Runs a full replacement of data if any.

    • Incremental. Adds new records to existing data.

    • Enrichment. Adds new attributes to existing data records.

    • Delete. Delete any existing data objects.

  8. For Connection Settings, enter the settings to connect to your database server. For example:

    {
  "username": "admin",
  "password": "Password123",
  "connectURL": "http://identity.generic.com"
}

  9. Click Add Object, and then select the data source file:

    1. Click Applications, enter the metadata for applications file. For example:

      {
  "appMetaUrl": "http://identity.generic.com?q=applications&appName=Ac*",
  "prefix": "autoid"
}

    2. Click Assignments, enter the metadata for the assignments file. For example:

      {
  "appMetaUrl": "http://identity.generic.com?q=assignments&userId=*",
  "prefix": "autoid"
}

    3. Click Entitlements, enter the metadata for the entitlements file. For example:

      {
  "appMetaUrl": "http://identity.generic.com?q=entitlements&appId=*",
  "prefix": "autoid"
}

    4. Click Identities, enter the metadata for the identities file. For example:

      {
  "appMetaUrl": "http://identity.generic.com?q=identities&userId=*",
  "prefix": "autoid"
}

  10. Click Save.

    See it in action
    data sources generic

  11. Repeat the previous steps to add more JDBC data source files if necessary.

  12. Next, you must set the attribute mappings. This is a critical step to ensure a successful analytics run. See Set Attribute Mappings.

Copyright © 2010-2022 ForgeRock, all rights reserved.