Generic and Explicit Object Mappings

There are two ways to map IDM objects to the tables in a JDBC database or to organizational units in DS:

Generic mapping, which allows you to store arbitrary objects without special configuration or administration.
Explicit mapping, which maps specific objects and properties to tables and columns in the JDBC database or to organizational units in DS.

By default, IDM uses a generic mapping for user-definable objects, for both a JDBC and a DS repository. A generic mapping speeds up initial deployment, and can make system maintenance more flexible by providing a stable database structure. In a test environment, generic tables enable you to modify the user and object model easily, without database access, and without the need to constantly add and drop table columns. However, generic mapping does not take full advantage of the underlying database facilities, such as validation within the database and flexible indexing. Using an explicit mapping generally results in a substantial performance improvement. It is therefore strongly advised that you change to an explicit mapping before deploying in a production environment. If you are integrating IDM with AM, and using a shared DS repository, you must use an explicit schema mapping.

These two mapping strategies are discussed in the following sections, for JDBC repositories and for DS repositories:

Generic and Explicit Mappings With a JDBC Repository

Generic Mappings With a JDBC Repository

Generic mapping speeds up development, and can make system maintenance more flexible by providing a stable database structure. However, generic mapping can have a performance impact and does not take full advantage of the database facilities (such as validation within the database and flexible indexing). In addition, queries can be more difficult to set up.

In a generic table, the entire object content is stored in a single large-character field named fullobject in the mainTable for the object. To search on specific fields, you can read them by referring to them in the corresponding properties table for that object. The disadvantage of generic objects is that, because every property you might like to filter by is stored in a separate table, you must join to that table each time you need to filter by anything.

The following diagram shows a pared down database structure for the default generic table, when using a MySQL repository. The diagram indicates the relationship between the main table and the corresponding properties table for each object.

Generic Tables Entity Relationship Diagram

These separate tables can make the query syntax particularly complex. For example, a simple query to return user entries based on a user name would need to be implemented as follows:

SELECT obj.objectid, obj.rev, obj.fullobject FROM ${_dbSchema}.${_mainTable} obj
INNER JOIN ${_dbSchema}.${_propTable} prop ON obj.id = prop.${_mainTable}_id
INNER JOIN ${_dbSchema}.objecttypes objtype ON objtype.id = obj.objecttypes_id
WHERE prop.propkey='/userName' AND prop.propvalue = ${uid} AND objtype.objecttype = ${_resource}",

The query can be broken down as follows:

Select the full object, the object ID, and the object revision from the main table:
```
SELECT obj.objectid, obj.rev, obj.fullobject FROM ${_dbSchema}.${_mainTable} obj
```

Join to the properties table and locate the object with the corresponding ID:

INNER JOIN ${_dbSchema}.${_propTable} prop  ON obj.id = prop.${_mainTable}_id

Join to the object types table to restrict returned entries to objects of a specific type. For example, you might want to restrict returned entries to managed/user objects, or managed/role objects:
```
INNER JOIN ${_dbSchema}.objecttypes objtype ON objtype.id = obj.objecttypes_id
```
Filter records by the userName property, where the userName is equal to the specified uid and the object type is the specified type (in this case, managed/user objects):
```
WHERE prop.propkey='/userName'
 AND prop.propvalue = ${uid}
 AND objtype.objecttype = ${_resource}",
```
The value of the uid field is provided as part of the query call, for example:
```
openidm.query("managed/user", { "_queryId": "for-userName", "uid": "jdoe" });
```

Tables for user definable objects use a generic mapping by default.

The following sample generic mapping object illustrates how managed/ objects are stored in a generic table:

"genericMapping" : {
    "managed/*" : {
        "mainTable" : "managedobjects",
        "propertiesTable" : "managedobjectproperties",
        "searchableDefault" : true,
        "properties" : {
            "/picture" : {
                "searchable" : false
            }
        }
    }
}

mainTable (string, mandatory)

Indicates the main table in which data is stored for this resource.

The complete object is stored in the fullobject column of this table. The table includes an objecttypes foreign key that is used to distinguish the different objects stored within the table. In addition, the revision of each stored object is tracked, in the rev column of the table, enabling multiversion concurrency control (MVCC). For more information, see "Manipulating Managed Objects Programmatically".

propertiesTable (string, mandatory)

Indicates the properties table, used for searches.

Note

PostgreSQL repositories do not use these properties tables to access specific properties. Instead, the PostgreSQL json_extract_path_text() function achieves this functionality.

The contents of the properties table is a defined subset of the properties, copied from the character large object (CLOB) that is stored in the fullobject column of the main table. The properties are stored in a one-to-many style separate table. The set of properties stored here is determined by the properties that are defined as searchable.

The stored set of searchable properties makes these values available as discrete rows that can be accessed with SQL queries, specifically, with WHERE clauses. It is not otherwise possible to query specific properties of the full object.

The properties table includes the following columns:

${_mainTable}_id corresponds to the id of the full object in the main table, for example, manageobjects_id, or genericobjects_id.
propkey is the name of the searchable property, stored in JSON pointer format (for example /mail).
proptype is the data type of the property, for example java.lang.String. The property type is obtained from the Class associated with the value.
propvalue is the value of property, extracted from the full object that is stored in the main table.
Regardless of the property data type, this value is stored as a string, so queries against it should treat it as such.

searchableDefault (boolean, optional)

Specifies whether all properties of the resource should be searchable by default. Properties that are searchable are stored and indexed. You can override the default for individual properties in the properties element of the mapping. The preceding example indicates that all properties are searchable, with the exception of the picture property.

For large, complex objects, having all properties searchable implies a substantial performance impact. In such a case, a separate insert statement is made in the properties table for each element in the object, every time the object is updated. Also, because these are indexed fields, the recreation of these properties incurs a cost in the maintenance of the index. You should therefore enable searchable only for those properties that must be used as part of a WHERE clause in a query.

Note

PostgreSQL repositories do not use the searchableDefault property.

properties

Lists any individual properties for which the searchable default should be overridden.

Note that if an object was originally created with a subset of searchable properties, changing this subset (by adding a new searchable property in the configuration, for example) will not cause the existing values to be updated in the properties table for that object. To add the new property to the properties table for that object, you must update or recreate the object.

Improving Generic Mapping Search Performance (JDBC)

All properties in a generic mapping are searchable by default. In other words, the value of the searchableDefault property is true unless you explicitly set it to false. Although there are no individual indexes in a generic mapping, you can improve search performance by setting only those properties that you need to search as searchable. Properties that are searchable are created within the corresponding properties table. The properties table exists only for searches or look-ups, and has a composite index, based on the resource, then the property name.

The sample JDBC repository configuration files (db/database/conf/repo.jdbc.json) restrict searches to specific properties by setting the searchableDefault to false for managed/user mappings. You must explicitly set searchable to true for each property that should be searched. The following sample extract from repo.jdbc.json indicates searches restricted to the userName property:

"genericMapping" : {
    "managed/user" : {
        "mainTable" : "manageduserobjects",
        "propertiesTable" : "manageduserobjectproperties",
        "searchableDefault" : false,
        "properties" : {
            "/userName" : {
                "searchable" : true
            }
        }
    }
}

With this configuration, IDM creates entries in the properties table only for userName properties of managed user objects.

If the global searchableDefault is set to false, properties that do not have a searchable attribute explicitly set to true are not written in the properties table.

Explicit Mappings With a JDBC Repository

Explicit mapping is more difficult to set up and maintain, but can take complete advantage of the native database facilities.

An explicit table offers better performance and simpler queries. There is less work in the reading and writing of data, because the data is all in a single row of a single table. In addition, it is easier to create different types of indexes that apply to only specific fields in an explicit table. The disadvantage of explicit tables is the additional work required in creating the table in the schema. Also, because rows in a table are inherently more simple, it is more difficult to deal with complex objects. Any non-simple key:value pair in an object associated with an explicit table is converted to a JSON string and stored in the cell in that format. This makes the value difficult to use, from the perspective of a query attempting to search within it.

You can have a generic mapping configuration for most managed objects, and an explicit mapping that overrides the default generic mapping in certain cases.

IDM provides a sample configuration, for each JDBC repository, that sets up an explicit mapping for the managed user object and a generic mapping for all other managed objects. This configuration is defined in the files named /path/to/openidm/db/repository/conf/repo.jdbc-repository-explicit-managed-user.json. To use this configuration, copy the file that corresponds to your repository to your project's conf/ directory and rename it repo.jdbc.json. Run the sample-explicit-managed-user.sql data definition script (in the path/to/openidm/db/repository/scripts directory) to set up the corresponding tables when you configure your JDBC repository.

IDM uses explicit mapping for internal system tables, such as the tables used for auditing.

Depending on the types of usage your system is supporting, you might find that an explicit mapping performs better than a generic mapping. Operations such as sorting and searching (such as those performed in the default UI) tend to be faster with explicitly-mapped objects, for example.

The following sample explicit mapping object illustrates how internal/user objects are stored in an explicit table:

"explicitMapping" : {
    "internal/user" : {
        "table" : "internaluser",
        "objectToColumn" : {
            "_id" : "objectid",
            "_rev" : { "column" : "rev", "isNotNull" : true },
            "password" : "pwd"
        }
    },
    ...
}

<resource-uri> (string, mandatory)

Indicates the URI for the resources to which this mapping applies, for example, internal/user.

table (string, mandatory)

The name of the database table in which the object (in this case internal users) is stored.

objectToColumn (string, mandatory)

The way in which specific managed object properties are mapped to columns in the table.

The mapping can be a simple one to one mapping, for example "userName": "userName", or a more complex JSON map or list. When a column is mapped to a JSON map or list, the syntax is as shown in the following examples:

"messageDetail" : { "column" : "messagedetail", "type" : "JSON_MAP" }

or

"roles" : { "column" : "roles", "type" : "JSON_LIST" }

Available column data types you can specify are STRING (the default), NUMBER, JSON_MAP, JSON_LIST, and FULLOBJECT.

You can also prevent a column from accepting a NULL value, by setting the property isNotNull to true. This property is optional; if the property is omitted, it will default to false. Specifying which columns do not allow a null value can improve performance when sorting and paginating large queries. The syntax is similar to when specifying a column type:

"createDate" : { "column" : "createDate", "isNotNull" : true }

Caution

Pay particular attention to the following caveats when you map properties to explicit columns in your database:

Support for data types in columns is restricted to numeric values (NUMBER), strings (STRING), and boolean values (BOOLEAN). Although you can specify other data types, IDM handles all other data types as strings. Your database will need to convert these types from a string to the alternative data type. This conversion is not guaranteed to work.
If the conversion does work, the format might not be the same when the data is read from the database as it was when it was saved. For example, your database might parse a date in the format 12/12/2012 and return the date in the format 2012-12-12 when the property is read.
Passwords are encrypted before they are stored in the repository. The length of the password column must be long enough to store the encrypted password value, which can vary depending on how it is encrypted and whether it is also hashed.
The sample-explicit-managed-user.sql file referenced in this section sets the password column to a length of 511 characters (VARCHAR(511) to account for the additional space an encrypted password requires. For more information about IDM encryption and an example encrypted password value, see "encrypt" and "Encoding Attribute Values".
If your data objects include virtual properties, you must include columns in which to store these properties. If you don't explicitly map the virtual properties, you will see errors similar to the following when you attempt to create the corresponding object:
```
{
    "code":400,
    "reason":"Bad Request",
    "message":"Unmapped fields [/property-name/0] for type managed/user and table openidm.managed_user"
}
```
When virtual properties are returned in the result of a query, the query previously persisted values of the requested virtual properties. To recalculate virtual property values in a query, you must set executeOnRetrieve to true in the query request parameters. For more information, see "Property Storage Triggers".

Convert an Explicit Mapped Object to a Generic Mapped Object (JDBC)

This procedure demonstrates how to migrate data to a different storage configuration within the same system using the migration service to convert the object data. After you finish the conversion, the converted objects are technically hybrid objects—generically mapped objects that have certain fields that are mapped to explicit columns.

Note

Important considerations before you start:

After you complete the process, object resource paths must stay the same to maintain any possible relationship references.
You must migrate data to an empty table. You can not simply modify an existing explicit table, as the existing records cannot be read through a generic mapping.
During the migration, changes made to the source object might not be transferred to the new object. To ensure everything is migrated correctly, run the migration during idle time, or when the system is least busy.

This procedure assumes that the repository configuration includes explicitly mapped object types, and that such objects already exist in the corresponding tables. For example:

"explicitMapping" : {
...
"managed/objectToConvert" : {
  "table" : "objecttoconvert",
  "objectToColumn" : {
    "_id" : "objectid",
    "_rev" : "rev",
    "desc" : "descr"
  }
}

Create the new generic table and associated properties table:

CREATE TABLE `openidm`.`objecttoconvert_gen` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT ,
`objecttypes_id` BIGINT UNSIGNED NOT NULL ,
`objectid` VARCHAR(255) NOT NULL ,
`rev` VARCHAR(38) NOT NULL ,
`descr` VARCHAR(255) NOT NULL ,
`fullobject` MEDIUMTEXT NULL ,
PRIMARY KEY (`id`) ,
UNIQUE INDEX `idx-objecttoconvert_object` (`objecttypes_id` ASC, `objectid` ASC) ,
INDEX `fk_objecttoconvert_objectypes` (`objecttypes_id` ASC) ,
CONSTRAINT `fk_objecttoconvert_objectypes`
    FOREIGN KEY (`objecttypes_id` )
        REFERENCES `openidm`.`objecttypes` (`id` )
        ON DELETE CASCADE
        ON UPDATE NO ACTION)
ENGINE = InnoDB;

CREATE TABLE IF NOT EXISTS `openidm`.`objecttoconvert_genproperties` (
`objecttoconvert_gen_id` BIGINT UNSIGNED NOT NULL ,
`propkey` VARCHAR(255) NOT NULL ,
`proptype` VARCHAR(32) NULL ,
`propvalue` VARCHAR(2000) NULL ,
`propindex` BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (`objecttoconvert_gen_id`, `propkey`, `propindex`),
INDEX `fk_objecttoconvertproperties_managedobjects` (`objecttoconvert_gen_id` ASC) ,
INDEX `idx_objecttoconvertproperties_propkey` (`propkey` ASC) ,
INDEX `idx_objecttoconvertproperties_propvalue` (`propvalue`(255) ASC) ,
CONSTRAINT `fk_objecttoconvertproperties_objecttoconvert`
FOREIGN KEY (`objecttoconvert_gen_id` )
REFERENCES `openidm`.`objecttoconvert_gen` (`id` )
ON DELETE CASCADE
ON UPDATE NO ACTION)
ENGINE = InnoDB;

Modify conf/repo.jdbc.json to map the object path in the generic mapping section to the empty generic table. If the migrated data will have additional searchable columns, add them now.

Create a conf/migration.json file with the following details:

Update the authentication settings to match the system configuration.
Modify the instanceUrl to point to the same system.

For example:

{
  "enabled" : true,
  "connection" : {
    "instanceUrl" : "http://localhost:8080/openidm/",
    "authType" : "basic",
    "userName" : "openidm-admin",
    "password" : "openidm-admin"
  },
  "mappings" : [
    {
      "target" : "repo/managed/objectToConvert_gen",
      "source" : "repo/managed/objectToConvert"
    }
  ]
}

Call the mapping service to view the mapping name that was generated:

curl --location \
--header 'X-OpenIDM-NoSession: true' \
--header 'X-OpenIDM-Username: openidm-admin' \
--header 'X-OpenIDM-Password: openidm-admin' \
--header 'Accept-API-Version: resource=1.0' \
--request POST 'http://localhost:8080/openidm/migration?_action=mappingNames'
[
  [
    "repoManagedObjecttoconvert_repoManagedObjecttoconvertGen"
  ]
]

Start the migration:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/migration?_action=migrate&mapping=repoManagedObjecttoconvert_repoManagedObjecttoconvertGen"
{
  "migrationResults": {
    "recons": [
      {
        "name": "repoManagedObjecttoconvert_repoManagedObjecttoconvertGen",
        "status": "PENDING"
      }
    ]
  }
}

You must wait until the migration is completed. To check the status of the migration:

curl \
--header 'X-OpenIDM-NoSession: true' \
--header 'X-OpenIDM-Username: openidm-admin' \
--header 'X-OpenIDM-Password: openidm-admin' \
--header 'Accept-API-Version: resource=1.0' \
--request POST 'http://localhost:8080/openidm/migration?_action=status'
{
  "migrationResults": {
    "recons": [
      {
        "name": "repoManagedObjecttoconvert_repoManagedObjecttoconvertGen",
        "status": {
          "_id": "820a1c66-6f1a-41d8-82a4-fc5a2d246326-424",
          "mapping": "repoManagedObjecttoconvert_repoManagedObjecttoconvertGen",
          "state": "SUCCESS",
          "stage": "COMPLETED_SUCCESS",
          "stageDescription": "reconciliation completed.",
          "progress": {
            "source": {
              "existing": {
                "processed": 0,
                "total": "9"
              }
            },
            "target": {
              "existing": {
                "processed": 0,
                "total": "?"
              },
              "created": 0,
              "unchanged": 0,
              "updated": 0,
              "deleted": 0
            },
            "links": {
              "existing": {
                "processed": 0,
                "total": "0"
              },
              "created": 0
            }
          },
          "situationSummary": {
            "SOURCE_IGNORED": 0,
            "FOUND_ALREADY_LINKED": 0,
            "UNQUALIFIED": 0,
            "ABSENT": 0,
            "TARGET_IGNORED": 0,
            "MISSING": 0,
            "ALL_GONE": 0,
            "UNASSIGNED": 0,
            "AMBIGUOUS": 0,
            "CONFIRMED": 0,
            "LINK_ONLY": 0,
            "SOURCE_MISSING": 0,
            "FOUND": 0
          },
          "statusSummary": {
            "SUCCESS": 0,
            "FAILURE": 9
          },
          "durationSummary": {
            "sourceObjectQuery": {
              "min": 26,
              "max": 33,
              "mean": 30,
              "count": 9,
              "sum": 277,
              "stdDev": 2
            },
            "sourceQuery": {
              "min": 37,
              "max": 37,
              "mean": 37,
              "count": 1,
              "sum": 37,
              "stdDev": 0
            },
            "auditLog": {
              "min": 0,
              "max": 1,
              "mean": 0,
              "count": 11,
              "sum": 9,
              "stdDev": 0
            },
            "linkQuery": {
              "min": 4,
              "max": 4,
              "mean": 4,
              "count": 1,
              "sum": 4,
              "stdDev": 0
            },
            "correlationQuery": {
              "min": 8,
              "max": 18,
              "mean": 15,
              "count": 9,
              "sum": 139,
              "stdDev": 4
            },
            "sourcePhase": {
              "min": 113,
              "max": 113,
              "mean": 113,
              "count": 1,
              "sum": 113,
              "stdDev": 0
            }
          },
          "parameters": {
            "sourceQuery": {
              "resourceName": "external/migration/repo/managed/objectToConvert",
              "queryFilter": "true",
              "_fields": "_id"
            },
            "targetQuery": {
              "resourceName": "repo/managed/objectToConvert_gen",
              "queryFilter": "true",
              "_fields": "_id"
            }
          },
          "started": "2021-01-20T18:22:34.026Z",
          "ended": "2021-01-20T18:22:34.403Z",
          "duration": 377,
          "sourceProcessedByNode": {}
        }
      }
    ]
  }
}

Note

Optionally, you can run the migration again to account for changes that may have occurred during the original migration.

The data is now migrated to the new tables, but IDM is still referencing the previous mapping.

Edit the repo.jdbc.json file:

Remove the old mapping from explicitMapping:

"explicitMapping" : {
...
"managed/objectToConvert" : {
  "table" : "objecttoconvert",
  "objectToColumn" : {
    "_id" : "objectid",
    "_rev" : "rev",
    "desc" : "descr"
  }
}

Modify the newly added genericMapping to point to the old resource path:

"genericMapping" : {
  ...
  "managed/objectToConvert" : {
    "mainTable" : "objecttoconvert_gen",
    "propertiesTable" : "objecttoconvert_genproperties",
    "searchableDefault" : false,
    "objectToColumn" : {
      "_id" : "objectid",
      "_rev" : "rev",
      "desc" : "descr"
    },
    "properties": {
      "/stringArrayField" : {
        "searchable" : true
      }
    }
  },
}

Run a JDBC update statement to reference the old resource path that the explicitMapping objects were configured as previously. Adjust the following example to match your repository requirements, as needed:
```
update openidm.objecttypes set objecttype = 'managed/objectToConvert' where objecttype = 'managed/objectToConvert_gen';
```

Generic and Explicit Mappings With a DS Repository

For both generic and explicit mappings, IDM maps object types using a dnTemplate property. The dnTemplate is effectively a pointer to where the object is stored in DS. For example, the following excerpt of the default repo.ds.json file shows how configuration objects are stored under the DN ou=config,dc=openidm,dc=forgerock,dc=com:

"config": {
    "dnTemplate": "ou=config,dc=openidm,dc=forgerock,dc=com"
}

Generic Mappings With a DS Repository

By default, IDM uses a generic mapping for all objects except the following:

Internal users, roles, and privileges
Links
Clustered reconciliation target IDs
Note
Clustered reconciliation is not currently supported with a DS repository.
Locks
Objects related to queued synchronization

With a generic mapping, all the properties of an object are stored as a single JSON blob in the fr-idm-json attribute. To create a new generic mapping, you need only specify the dnTemplate, that is, where the object will be stored in the directory tree.

You can specify a wildcard mapping, that stores all nested URIs under a particular branch of the directory tree, for example:

"managed/*": {
    "dnTemplate": "ou=managed,dc=openidm,dc=forgerock,dc=com"
}

With this mapping, all objects under managed/, such as managed/user and managed/device, will be stored in the branch ou=managed,dc=openidm,dc=forgerock,dc=com. You do not have to specify separate mappings for each of these objects. The mapping creates a new ou for each object. So, for example, managed/user objects will be stored under the DN ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com and managed/device objects will be stored under the DN ou=device,ou=managed,dc=openidm,dc=forgerock,dc=com.

Improving Generic Mapping Search Performance (DS)

By default, all generic objects are instances of the fr-idm-generic-obj object class and their properties are stored as a single JSON blob in the fr-idm-json attribute. The fr-idm-json attribute is indexed by default, which results in all attributes of a generic object being indexed. JDBC repositories behave in a similar way, with all generic objects being searchable by default.

To optimize search performance on specific generic resources, you can set up your own schema providers and indices as described in this section. For a detailed explanation of how indexes improve LDAP search performance, see Indexes in the DS Configuration Guide.

For an embedded or external DS repository, the following managed user properties are indexed by default:

userName (cn)
givenName
sn
mail
accountStatus

You can configure managed user indexes in the repository configuration (repo.ds.json) by adding indices and schemaProviders objects, as follows:

"indices" : {
    ...
    "fr-idm-managed-user-json" : {
        "type" : [ "EQUALITY" ]
    },
    ...
},
"schemaProviders" : {
    "IDM managed/user Json Schema" : {
        "matchingRuleName" : "caseIgnoreJsonQueryMatchManagedUser",
        "matchingRuleOid" : "1.3.6.1.4.1.36733.2.....",
        "caseSensitiveStrings" : false,
        "fields" : [ "accountStatus", "givenName", "mail", "sn", "userName" ]
    },
    ...
}

The indexed properties are listed in the array of fields for that managed object. To index additional managed user properties, add the property names to this array of fields.

To set up indexes on generic objects other than the managed user object, you must do the following:

Add the object to the DS schema.
The schema for an embedded DS repository is stored in the /path/to/openidm/db/openidm/opendj/db/schema/60-repo-schema.ldif file.
You can use the managed user object as an example of the schema syntax:
```
###
# Managed User
###
attributeTypes: ( 1.3.6.1.4.1.36733.2.3.1.13
  NAME 'fr-idm-managed-user-json'
  SYNTAX 1.3.6.1.4.1.36733.2.1.3.1
  EQUALITY caseIgnoreJsonQueryMatchManagedUser
  ORDERING caseIgnoreOrderingMatch
  SINGLE-VALUE
  X-ORIGIN 'OpenIDM DSRepoService')
objectClasses: ( 1.3.6.1.4.1.36733.2.3.2.6
  NAME 'fr-idm-managed-user'
  SUP top
  STRUCTURAL
  MUST ( fr-idm-managed-user-json )
  X-ORIGIN 'OpenIDM DSRepoService' )
```
For information about adding JSON objects to the DS schema, see Schema and JSON in the DS Configuration Guide.
Warning
If you delete the db/openidm directory, any additions you have made to the schema will be lost. If you have customized the schema, be sure to back up the 60-repo-schema.ldif file.
Add the object to the indices property in the conf/repo.ds.json file.
The following example sets up an equality index for a managed devices object:
```
"indices" : {
    ...
    "fr-idm-managed-devices-json" : {
        "type" : [ "EQUALITY" ]
    },
    ...
}
```

Add the object to the schemaProviders property in the conf/repo.ds.json file and list the properties that should be indexed.

The following example sets up indexes for the deviceName, brand, and assetNumber properties of the managed device object:

"schemaProviders" : {
    "Managed Device Json" : {
        "matchingRuleName" : "caseIgnoreJsonQueryMatchManagedDevice",
        "matchingRuleOid" : "1.3.6.1.4.1.36733.2.....",
        "caseSensitiveStrings" : false,
        "fields" : [ "deviceName", "brand", "assetNumber" ]
    }
}

For more information about indexing JSON attributes, see JSON Query Matching Rule Index in the DS Configuration Guide.

Note

The OIDs shown in this section are reserved for ForgeRock internal use. If you set up additional objects and attributes, or if you change the default schema, you must specify your own OIDs here.

Explicit Mappings With a DS Repository

The default configuration uses a generic mapping for managed user objects. To use an explicit mapping for managed user objects, change the repository configuration before you start IDM for the first time.

To set up an explicit mapping:

Copy the repo.ds-explicit-managed-user.json file to your project's conf directory, and rename that file repo.ds.json:

cp /path/to/openidm/db/ds/conf/repo.ds-explicit-managed-user.json project-dir/conf/repo.ds.json

Important

This file is configured for an embedded DS repository by default. To set up an explicit mapping for an external DS repository, change the value of the embedded property to false and add the following properties:

"security": {
  "trustManager": "file",
  "fileBasedTrustManagerType": "JKS",
  "fileBasedTrustManagerFile": "&{idm.install.dir}/security/truststore",
  "fileBasedTrustManagerPasswordFile": "&{idm.install.dir}/security/storepass"
},
"ldapConnectionFactories": {
  "bind": {
    "connectionSecurity": "startTLS",
    "heartBeatIntervalSeconds": 60,
    "heartBeatTimeoutMilliSeconds": 10000,
    "primaryLdapServers": [
      {
        "hostname": "localhost",
        "port": 31389
      }
    ],
    "secondaryLdapServers": []
  },
  "root": {
    "inheritFrom": "bind",
    "authentication": {
      "simple": {
        "bindDn": "uid=admin",
        "bindPassword": "password"
      }
    }
  }
}

For more information on these properties, see "DS Repository Configuration".

Start IDM.

IDM uses the DS REST to LDAP gateway to map JSON objects to LDAP objects stored in the directory. To create additional explicit mappings, you must specify the LDAP objectClasses to which the object is mapped, and how each property maps to its corresponding LDAP attributes. Specify at least the property type and the corresponding ldapAttribute. For relationships between objects, you must explicitly define those objects in the repository configuration.

The following excerpt shows an example of an explicit managed user object mapping:

"managed/user" : {
    "dnTemplate": "ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com",
    "objectClasses": [
        "person",
        "organizationalPerson",
        "inetOrgPerson",
        "fr-idm-managed-user-explicit",
        "inetuser"
    ],
    "properties": {
        "_id": {
            "type": "simple", "ldapAttribute": "uid", "isRequired": true, "writability": "createOnly"
        },
        "userName": {
            "type": "simple", "ldapAttribute": "cn"
        },
        "password": {
            "type": "json", "ldapAttribute": "fr-idm-password"
        },
        "accountStatus": {
            "type": "simple", "ldapAttribute": "fr-idm-accountStatus"
        },
        "roles": {
            "type": "json", "ldapAttribute": "fr-idm-role", "isMultiValued": true
        },
        "effectiveRoles": {
            "type": "json", "ldapAttribute": "fr-idm-effectiveRole", "isMultiValued": true
        },
        "effectiveAssignments": {
            "type": "json", "ldapAttribute": "fr-idm-effectiveAssignment", "isMultiValued": true
        },
        ...
    }
}

You do not need to map the _rev (revision) property of an object as this property is implicit in all objects and maps to the DS etag operational attribute.

If your data objects include virtual properties, you must include property mappings for these properties. If you don't explicitly map the virtual properties, you will see errors similar to the following when you attempt to create the corresponding object:

{
    "code": 400,
    "reason": "Bad Request",
    "message": "Unmapped fields..."
}

For more information about the REST to LDAP property mappings, see Mapping Configuration in the DS REST API Guide.

For performance reasons, the DS repository does not apply unique constraints to links. This behavior is different to the JDBC repositories, where uniqueness on link objects is enforced.

Important

DS currently has a default index entry limit of 4000. Therefore, you cannot query more than 4000 records unless you create a Virtual List View (VLV) index. A VLV index is designed to help DS respond to client applications that need to browse through a long list of objects.

You cannot create a VLV index on a JSON attribute. For generic mappings, IDM avoids this restriction by using client-side sorting and searching. However, for explicit mappings you must create a VLV index for any filtered or sorted results, such as results displayed in a UI grid. To configure a VLV index, use the dsconfig command described in Virtual List View Index in the DS Configuration Guide.

Specifying How IDM IDs Map to LDAP Entry Names

The DS REST2LDAP configuration lets you set a namingStrategy that specifies how LDAP entry names are mapped to JSON resources. When IDM stores its objects in a DS repository, this namingStrategy determines how the IDM _id value maps to the Relative Distinguished Name (RDN) of the corresponding DS object.

The namingStrategy is specified as part of the explicitMapping of an object in the repo.ds.json file. The following example shows a naming strategy configuration for an explicit managed user mapping:

"resourceMapping": {
    "defaultMapping": {
        "dnTemplate": "ou=generic,dc=openidm,dc=forgerock,dc=com"
    },
    ...
    "explicitMapping": {
        "managed/user": {
            "dnTemplate": "ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com",
            "objectClasses": [
                "person",
                "organizationalPerson",
                "inetOrgPerson",
                "fr-idm-managed-user-explicit"
            ],
            "namingStrategy": {
                "type": "clientDnNaming",
                "dnAttribute": "uid"
            },
            ...
        }
    }
}

The namingStrategy can be one of the following:

clientDnNaming - IDM provides an _id to DS and that _id is used to generate the DS RDN. In the following example, the IDM _id maps to the LDAP uid attribute:
```
{
    "namingStrategy": {
        "type": "clientDnNaming",
        "dnAttribute": "uid"
    }
}
```
With this default configuration, entries are stored in DS with a DN similar to the following:
"uid=idm-uuid,ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com"
Note
If these default DNs are suitable in your deployment, you do not have to change anything with regard to the naming strategy.
clientNaming - IDM provides an _id to DS but the DS RDN is derived from a different user attribute in the LDAP entry. In the following example, the RDN is the cn attribute. The _id that IDM provides for the object maps to the LDAP uid attribute:
```
{
    "namingStrategy": {
        "type": "clientNaming",
        "dnAttribute": "cn",
        "idAttribute": "uid"
    }
}
```
With this configuration, entries are stored in DS with a DN similar to the following:
"cn=username,ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com"

Specifying a namingStrategy is optional. If you do not specify a strategy, the default is clientDnNaming with the following configuration:

{
    "namingStrategy" : {
        "type" : "clientDnNaming",
        "dnAttribute" : "uid"
    },
    "properties: : {
        "_id": {
            "type": "simple",
            "ldapAttribute": "uid",
            "isRequired": true,
            "writability": "createOnly"
        },
        ...
    }
}

Note

If you do not set a dnAttribute as part of the naming strategy, the value of the dnAttribute is taken from the value of the ldapAttribute on the _id property.

Relationship Properties in a DS Repository

The IDM object model lets you define relationships between objects. In a DS repository, relationships are implemented using the reference and reverseReference REST to LDAP property types. For more information about the reference and reverseReference property types, read the JSON property mapping section of the DS HTTP User Guide.

Relationship properties must be defined in the repository configuration (repo.ds.json), for both generic and explicit object mappings.

The following property definitions for a managed/user object show how the relationship between a manager and their reports is defined in the repository configuration:

"managed/user" : {
    "dnTemplate" : "ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com",
    ...
    "properties" : {
        ...
        "reports" : {
            "type" : "reverseReference",
            "resourcePath" : "managed/user",
            "propertyName" : "manager",
            "isMultiValued" : true
        },
        "manager" : {
            "type" : "reference",
            "ldapAttribute" : "fr-idm-managed-user-manager",
            "primaryKey" : "uid",
            "resourcePath" : "managed/user",
            "isMultiValued" : false
        },
        ...
    }
}

This configuration sets the reports property as a reverseReference, or reverse relationship of the manager property. This means that if you add a manager to a user, the user automatically becomes one of the reports of that manager.

Note the ldapAttribute defined in the relationship object (fr-idm-managed-user-manager in this case). Your DS schema must include this attribute, and an object class that contains this attribute. Relationship attributes in the DS schema must use the Name and Optional JSON syntax.

The following example shows the DS schema definition for the IDM manager property:

attributeTypes: ( 1.3.6.1.4.1.36733.2.3.1.69
  NAME 'fr-idm-managed-user-manager'
  DESC 'Reference to a users manager'
  SINGLE-VALUE
  SYNTAX 1.3.6.1.4.1.36733.2.1.3.12
  EQUALITY nameAndOptionalCaseIgnoreJsonIdEqualityMatch
  X-STABILITY 'Internal' )

Important

If you define a relationship in the managed object configuration and you do not define that relationship as a reference or reverse reference in the repository configuration (repo.ds.json), you will be able to query the relationships, but filtering and sorting on those queries will not work. This is the case when you define relationship objects in the Admin UI—the relationship is defined only in the managed object configuration and not in the repository configuration.

In this case, queries such as the following are not supported:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/managed/user/_id/managedOrgs?_pageSize=50&_sortKeys=_id&_totalPagedResultsPolicy=ESTIMATE&_queryFilter=true"

This restriction includes delegated admin privilege filters.

Generic and Explicit Object Mappings

Generic and Explicit Mappings With a JDBC Repository

Generic Mappings With a JDBC Repository

Note

Note

Improving Generic Mapping Search Performance (JDBC)

Explicit Mappings With a JDBC Repository

Caution

Convert an Explicit Mapped Object to a Generic Mapped Object (JDBC)

Note

Note

Generic and Explicit Mappings With a DS Repository

Generic Mappings With a DS Repository

Note

Improving Generic Mapping Search Performance (DS)

Warning

Note

Explicit Mappings With a DS Repository

Important

Important

Specifying How IDM IDs Map to LDAP Entry Names

Note

Note

Relationship Properties in a DS Repository

Important