Directory Services 7.4.3

Configure indexes

You modify index configurations by using the dsconfig command. Configuration changes take effect after you rebuild the index with the new configuration, using the rebuild-index command. The dsconfig --help-database command lists subcommands for creating, reading, updating, and deleting index configuration.

Indexes are per directory backend rather than per base DN. To maintain separate indexes for different base DNs on the same server, put the entries in different backends.

Standard indexes

New index

The following example creates a new equality index for the description attribute:

$ dsconfig \
 create-backend-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name description \
 --set index-type:equality \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Approximate index

The following example adds an approximate index for the sn (surname) attribute:

$ dsconfig \
 set-backend-index-prop \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name sn \
 --add index-type:approximate \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Approximate indexes depend on the Double Metaphone matching rule.

Extensible match index

DS servers support matching rules defined in LDAP RFCs. They also define DS-specific extensible matching rules.

The following are DS-specific extensible matching rules:

Name: ds-mr-double-metaphone-approx

Double Metaphone Approximate Match described at http://aspell.net/metaphone/. The DS implementation always produces a single value rather than one or possibly two values.

Configure approximate indexes as described in Approximate index.

For an example using this matching rule, refer to Approximate match.

Name: ds-mr-user-password-exact

User password exact matching rule used to compare encoded bytes of two hashed password values for exact equality.

Name: ds-mr-user-password-equality

User password matching rule implemented as the user password exact matching rule.

Name: partialDateAndTimeMatchingRule

Partial date and time matching rule for matching parts of dates in time-based searches.

For an example using this matching rule, refer to Active accounts.

Name: relativeTimeOrderingMatch.gt

Greater-than relative time matching rule for time-based searches.

For an example using this matching rule, refer to Active accounts.

Name: relativeTimeOrderingMatch.lt

Less-than relative time matching rule for time-based searches.

For an example using this matching rule, refer to Active accounts.

The following example configures an extensible matching rule index for "later than" and "earlier than" generalized time matching on the ds-last-login-time attribute:

$ dsconfig \
 create-backend-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --set index-type:extensible \
 --set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.5 \
 --set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.6 \
 --index-name ds-last-login-time \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Notice the index-extensible-matching-rule setting takes an OID, not the name of the matching rule.

Indexes for attributes with few unique values

Big indexes fit the case where many, many entries have the same attribute value.

The default DS evaluation profile generates 100,000 user entries with addresses in the United States, so some st (state) attributes are shared by 4000 or more users. With a regular equality index, searches for some states reach the index entry limit, causing unindexed searches. A big index avoids this problem.

The following commands configure an equality index for the state attribute, and then build the new index:

$ dsconfig \
 create-backend-index \
 --backend-name dsEvaluation \
 --index-name st \
 --set index-type:big-equality \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

$ rebuild-index \
 --baseDn dc=example,dc=com \
 --index st \
 --hostname localhost \
 --port 4444 \
 --bindDn uid=admin \
 --bindPassword password \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin

Once the index is ready, a client application can page through all the users in a state with an indexed search:

$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=bjensen,ou=People,dc=example,dc=com \
 --bindPassword hifalutin \
 --baseDn dc=example,dc=com \
 --simplePageSize 5 \
 "(st=CA)" \
 cn
dn: uid=user.9,ou=People,dc=example,dc=com
cn: Abbe Abbate

dn: uid=user.123,ou=People,dc=example,dc=com
cn: Aili Aksel

dn: uid=user.124,ou=People,dc=example,dc=com
cn: Ailina Akyurekli

dn: uid=user.132,ou=People,dc=example,dc=com
cn: Ainslee Alary

dn: uid=user.264,ou=People,dc=example,dc=com
cn: Alphen Angell

Press RETURN to continue

When creating a big index that uses an extensible matching rule, model your work on the example in Extensible match index. Use the following options to set the index type and the matching rule. You must set at least one big-index-extensible-matching-rule:

  • --set index-type:big-extensible

  • --set big-index-extensible-matching-rule:OID

    Notice the big-index-extensible-matching-rule setting takes an OID, not the name of the matching rule.

    The OID must specify an equality matching rule for big indexes. For example, to create a big index for the sn (surname) attribute with caseIgnoreMatch, use --set big-index-extensible-matching-rule:2.5.13.2. To create a big index for password storage schemes, use --set big-index-extensible-matching-rule:1.3.6.1.4.1.36733.2.1.4.14.

Custom indexes for JSON

DS servers support attribute values that have JSON syntax. The following schema excerpt defines a json attribute with case-insensitive matching:

attributeTypes: ( json-attribute-oid NAME 'json'
  SYNTAX 1.3.6.1.4.1.36733.2.1.3.1 EQUALITY caseIgnoreJsonQueryMatch
  X-ORIGIN 'DS Documentation Examples' )

When you index a JSON attribute defined in this way, the default directory server behavior is to maintain index keys for each JSON field. Large or numerous JSON objects can result in large indexes, which is wasteful. If you know which fields are used in search filters, you can choose to index only those fields.

As described in Schema and JSON, for some JSON objects only a certain field or fields matter when comparing for equality. In these special cases, the server can ignore other fields when checking equality during updates, and you would not maintain indexes for other fields.

How you index a JSON attribute depends on the matching rule in the attribute’s schema definition, and on the JSON fields you expect to be used as search keys for the attribute.

Index JSON attributes

The examples that follow demonstrate these steps:

  1. Using the schema definition and the information in the following table, configure a custom schema provider for the attribute’s matching rule, if necessary.

    Matching Rule in Schema Definition Fields in Search Filter Custom Schema Provider Required?

    caseExactJsonQueryMatch

    caseIgnoreJsonQueryMatch

    Any JSON field

    No

    Custom JSON query matching rule

    Specific JSON field or fields

    Yes, refer to JSON query matching rule index

    Custom JSON equality or ordering matching rule

    Specific field(s)

    Yes, refer to JSON equality matching rule index

    A custom schema provider applies to all attributes using this matching rule.

  2. Add the schema definition for the JSON attribute.

  3. Configure the index for the JSON attribute.

  4. Add the JSON attribute values in the directory data.

JSON query matching rule index

This example illustrates the steps in Index JSON attributes.

If you installed a directory server with the ds-evaluation profile, the custom index configuration is already present.

The following command configures a custom, case-insensitive JSON query matching rule. This only maintains keys for the access_token and refresh_token fields:

$ dsconfig \
 create-schema-provider \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --provider-name "Custom JSON Query Matching Rule" \
 --type json-query-equality-matching-rule \
 --set enabled:true \
 --set case-sensitive-strings:false \
 --set ignore-white-space:true \
 --set matching-rule-name:caseIgnoreOAuth2TokenQueryMatch \
 --set matching-rule-oid:1.3.6.1.4.1.36733.2.1.4.1.1 \
 --set indexed-field:access_token \
 --set indexed-field:refresh_token \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

The following commands add schemas for an oauth2Token attribute that uses the matching rule:

$ ldapmodify \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password << EOF
dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( oauth2token-attribute-oid NAME 'oauth2Token'
  SYNTAX 1.3.6.1.4.1.36733.2.1.3.1 EQUALITY caseIgnoreOAuth2TokenQueryMatch
  SINGLE-VALUE X-ORIGIN 'DS Documentation Examples' )
-
add: objectClasses
objectClasses: ( oauth2token-attribute-oid NAME 'oauth2TokenObject' SUP top
  AUXILIARY MAY ( oauth2Token ) X-ORIGIN 'DS Documentation Examples' )
EOF

The following command configures an index using the custom matching rule implementation:

$ dsconfig \
 create-backend-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name oauth2Token \
 --set index-type:equality \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

For an example of how a client application could use this index, refer to JSON query filters.

JSON equality matching rule index

This example illustrates the steps in Index JSON attributes.

If you installed a directory server with the ds-evaluation profile, the custom index configuration is already present.

The following command configures a custom, case-insensitive JSON equality matching rule, caseIgnoreJsonTokenIdMatch:

$ dsconfig \
 create-schema-provider \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --provider-name "Custom JSON Token ID Matching Rule" \
 --type json-equality-matching-rule \
 --set enabled:true \
 --set case-sensitive-strings:false \
 --set ignore-white-space:true \
 --set matching-rule-name:caseIgnoreJsonTokenIDMatch \
 --set matching-rule-oid:1.3.6.1.4.1.36733.2.1.4.4.1 \
 --set json-keys:id \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

Notice that this example defines a matching rule with OID 1.3.6.1.4.1.36733.2.1.4.4.1. In production deployments, use a numeric OID allocated for your own organization.

The following commands add schemas for a jsonToken attribute, where the unique identifier is in the "id" field of the JSON object:

$ ldapmodify \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password << EOF
dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( jsonToken-attribute-oid NAME 'jsonToken'
  SYNTAX 1.3.6.1.4.1.36733.2.1.3.1 EQUALITY caseIgnoreJsonTokenIDMatch
  SINGLE-VALUE X-ORIGIN 'DS Documentation Examples' )
-
add: objectClasses
objectClasses: ( json-token-object-class-oid NAME 'JsonTokenObject' SUP top
  AUXILIARY MAY ( jsonToken ) X-ORIGIN 'DS Documentation Examples' )
EOF

The following command configures an index using the custom matching rule implementation:

$ dsconfig \
 create-backend-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name jsonToken \
 --set index-type:equality \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

For an example of how a client application could use this index, refer to JSON assertions.

Virtual list view index

The following example shows how to create a VLV index. This example applies where GUI users browse user accounts, sorting on surname then given name:

$ dsconfig \
 create-backend-vlv-index \
 --hostname localhost \
 --port 4444 \
 --bindDn uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name people-by-last-name \
 --set base-dn:ou=People,dc=example,dc=com \
 --set filter:"(|(givenName=*)(sn=*))" \
 --set scope:single-level \
 --set sort-order:"+sn +givenName" \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

When referring to a VLV index after creation, you must add vlv. as a prefix. In other words, if you named the VLV index people-by-last-name, refer to it as vlv.people-by-last-name when rebuilding indexes, changing index properties such as the index entry limit, or verifying indexes.

VLV for paged server-side sort

A special VLV index lets the server return sorted results. For example, users page through an entire directory database in a GUI. The user does not filter the data before displaying what is available.

The VLV index must have the following characteristics:

  • Its filter must be "always true," (&).

  • Its scope must cover the search scope of the requests.

  • Its base DN must match or be a parent of the base DN of the search requests.

  • Its sort order must match the sort keys of the requests in the order they occur in the requests, starting with the first sort key used in the request.

    For example, if the sort order of the VLV index is +l +sn +cn, then it works with requests having the following sort orders:

    • +l +sn +cn

    • +l +sn

    • +l

    • Or none for single-level searches.

    The VLV index sort order can include additional keys not present in a request.

The following example commands demonstrate creating and using a VLV index to sort paged results by locality, surname, and then full name. The l attribute is not indexed by default. This example makes use of the rebuild-index command described below. The directory superuser is not subject to resource limits on the LDAP search operation:

$ dsconfig \
 create-backend-vlv-index \
 --hostname localhost \
 --port 4444 \
 --bindDn uid=admin \
 --bindPassword password \
 --backend-name dsEvaluation \
 --index-name by-name \
 --set base-dn:ou=People,dc=example,dc=com \
 --set filter:"(&)" \
 --set scope:subordinate-subtree \
 --set sort-order:"+l +sn +cn" \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

$ rebuild-index \
 --hostname localhost \
 --port 4444 \
 --bindDn uid=admin \
 --bindPassword password \
 --baseDn dc=example,dc=com \
 --index vlv.by-name \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin

$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDn uid=admin \
 --bindPassword password \
 --baseDn dc=example,dc=com \
 --sortOrder +l,+sn,+cn \
 --simplePageSize 5 \
 "(&)" \
 cn l sn

dn: uid=user.93953,ou=People,dc=example,dc=com
cn: Access Abedi
l: Abilene
sn: Abedi

dn: uid=user.40283,ou=People,dc=example,dc=com
cn: Achal Abernathy
l: Abilene
sn: Abernathy

dn: uid=user.67240,ou=People,dc=example,dc=com
cn: Alaine Alburger
l: Abilene
sn: Alburger

dn: uid=user.26994,ou=People,dc=example,dc=com
cn: Alastair Alexson
l: Abilene
sn: Alexson

dn: uid=user.53853,ou=People,dc=example,dc=com
cn: Alev Allen
l: Abilene
sn: Allen

Press RETURN to continue ^C

You can also list the results with HDAP:

$ curl \
 --get \
 --cacert ca-cert.pem \
 --user uid=admin:password \
 --data '_queryFilter=true' \
 --data '_fields=cn,l,sn' \
 --data '_sortKeys=l,sn,cn' \
 --data '_pageSize=5' \
 --data '_prettyPrint=true' \
 --data 'scope=sub' \
 'https://localhost:8443/hdap/dc=com/dc=example/ou=People'
Show output 
{
  "result" : [ {
    "_id" : "dc=com/dc=example/ou=People/uid=user.93953",
    "_rev" : "<revision>",
    "cn" : [ "Access Abedi" ],
    "l" : [ "Abilene" ],
    "sn" : [ "Abedi" ]
  }, {
    "_id" : "dc=com/dc=example/ou=People/uid=user.40283",
    "_rev" : "<revision>",
    "cn" : [ "Achal Abernathy" ],
    "l" : [ "Abilene" ],
    "sn" : [ "Abernathy" ]
  }, {
    "_id" : "dc=com/dc=example/ou=People/uid=user.67240",
    "_rev" : "<revision>",
    "cn" : [ "Alaine Alburger" ],
    "l" : [ "Abilene" ],
    "sn" : [ "Alburger" ]
  }, {
    "_id" : "dc=com/dc=example/ou=People/uid=user.26994",
    "_rev" : "<revision>",
    "cn" : [ "Alastair Alexson" ],
    "l" : [ "Abilene" ],
    "sn" : [ "Alexson" ]
  }, {
    "_id" : "dc=com/dc=example/ou=People/uid=user.53853",
    "_rev" : "<revision>",
    "cn" : [ "Alev Allen" ],
    "l" : [ "Abilene" ],
    "sn" : [ "Allen" ]
  } ],
  "resultCount" : 5,
  "pagedResultsCookie" : "<cookie>",
  "totalPagedResultsPolicy" : "NONE",
  "totalPagedResults" : -1,
  "remainingPagedResults" : -1
}

Rebuild indexes

When you first import directory data, the directory server builds the indexes as part of the import process. DS servers maintain indexes automatically, updating them as directory data changes.

Only rebuild an index manually when it is necessary to do so. Rebuilding valid indexes wastes server resources, and is disruptive for client applications.

When you rebuild an index while the server is online, the index appears as degraded and unavailable while the server rebuilds it.

A search request that relies on an index in this state may temporarily fail as an unindexed search.

However, you must manually intervene when you:

  • Create a new index for a new directory attribute.

  • Create a new index for an existing directory attribute.

  • Change the server configuration in a way that affects the index, for example, by changing Index entry limits.

  • Verify an existing index, and find that it has errors or is not in a valid state.

Automate index rebuilds

To automate the process of rebuilding indexes, use the --rebuildDegraded option. This rebuilds only degraded indexes, and does not affect valid indexes:

$ rebuild-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN dc=example,dc=com \
 --rebuildDegraded \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin

Clear a new index for a new attribute

When you add a new attribute, as described in Update LDAP schema, and create an index for the new attribute, the new index appears as degraded or invalid. The attribute has not yet been used, and so the index is sure to be empty, not degraded.

In this case, you can safely use the rebuild-index --clearDegradedState command. The server can complete this operation quickly, because there is no need to scan the entire directory backend to rebuild a new, unused index.

In this example, an index has just been created for newUnusedAttribute. If the newly indexed attribute has already been used, rebuild the index instead of clearing the degraded state.

Before using the rebuild-index command, test the index status to make sure is has not been used:

  1. DS servers must be stopped before you use the backendstat command:

    $ stop-ds

  2. Check the status of the index(es).

    The indexes are prefixed with ! before the rebuild:

    $ backendstat \
 show-index-status \
 --backendID dsEvaluation \
 --baseDN dc=example,dc=com \
 | grep newUnusedAttribute

!  newUnusedAttribute.caseIgnoreMatch             ...
!  newUnusedAttribute.caseIgnoreSubstringsMatch:6 ...
!  newUnusedAttribute.presence                    ...

  3. Update the index information to fix the value of the unused index:

    $ rebuild-index \
 --offline \
 --baseDN dc=example,dc=com \
 --clearDegradedState \
 --index newUnusedAttribute

    Alternatively, you can first start the server, and perform this operation while the server is online.

  4. (Optional) With the server offline, check that the ! prefix has disappeared:

    $ backendstat \
 show-index-status \
 --backendID dsEvaluation \
 --baseDN dc=example,dc=com \
 | grep newUnusedAttribute

newUnusedAttribute.caseIgnoreMatch             ...
newUnusedAttribute.caseIgnoreSubstringsMatch:6 ...
newUnusedAttribute.presence                    ...

  5. Start the server if you have not done so already:

    $ start-ds

Rebuild an index

When you make a change that affects an index configuration, manually rebuild the index.

Individual indexes appear as degraded and are unavailable while the server rebuilds them. A search request that relies on an index in this state may temporarily fail as an unindexed search.

The following example rebuilds a degraded cn index immediately with the server online.

While the server is rebuilding the cn index, search requests that would normally succeed may fail:

$ rebuild-index \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN dc=example,dc=com \
 --index cn \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin

Avoid rebuilding all indexes at once

Rebuilding multiple non-degraded indexes at once is disruptive, and not recommended unless some change has affected all indexes.

If you use the --rebuildAll option, first take the backend offline, stop the server, or, at minimum, make sure that no applications connect to the server while it is rebuilding indexes.

Index entry limits

An index is a tree of key-value pairs. The key is what the search is trying to match. The value is a list of entry IDs.

As the number of entries in the directory grows, the list of entry IDs for some keys can become very large. For example, every entry in the directory has objectClass: top. If the directory maintains a substring index for mail, the number of entries ending in .com could be huge.

A directory server therefore defines an index entry limit. When the number of entry IDs for a key exceeds the limit, the server stops maintaining a list of IDs for that key. The limit effectively means a search using only that key is unindexed. Searches using other keys in the same index are not affected.

The following figure shows a fragment from a substring index for the mail attribute. The number of email addresses ending in com has exceeded the index entry limit. For the other substring keys, the entry ID lists are still maintained. To save space, the entry IDs are not shown in the figure.

Illustration of the index entry limit reached for the substring .COM

Ideally, the limit is set at the point where it becomes more expensive to maintain the entry ID list for a key, and to perform an indexed search than to perform an unindexed search. In practice, the limit is a tradeoff, with a default index entry limit value of 4000. Keep the default setting unless you have good reason to change it.

Check index entry limits

The following steps show how to get information about indexes where the index entry limit is exceeded for some keys. In this case, the directory server holds 100,000 user entries.

Use the backendstat show-index-status command:

  1. Stop DS servers before you use the backendstat command:

    $ stop-ds

  2. Non-zero values in the Over column of the output table indicate the number of keys for which the index-entry-limit setting has been exceeded. The keys that are over the limit are then listed below the table:

    $ backendstat show-index-status --backendID dsEvaluation --baseDN dc=example,dc=com

  Index Name                                          ...Over  Entry Limit...
------------------------------------------------------...-----------------...
  ...                                                 ...
  cn.caseIgnoreSubstringsMatch:6                      ...  14         4000...
  ...                                                 ...
  givenName.caseIgnoreSubstringsMatch:6               ...   9         4000...
  ...                                                 ...
  mail.caseIgnoreIA5SubstringsMatch:6                 ...  31         4000...
  ...                                                 ...
  objectClass.objectIdentifierMatch                   ...   4         4000...
  ...                                                 ...
  sn.caseIgnoreSubstringsMatch:6                      ...  14         4000...
  ...                                                 ...
  telephoneNumber.telephoneNumberSubstringsMatch:6    ...  10         4000...
  ...

Index: mail.caseIgnoreIA5SubstringsMatch:6
Over index-entry-limit keys: [.com] [0@exam] ...

Index: cn.caseIgnoreSubstringsMatch:6
Over index-entry-limit keys: [a] [an] [e] [er] [i] [k] [l] [n] [o] [on] [r] [s] [t] [y]

Index: givenName.caseIgnoreSubstringsMatch:6
Over index-entry-limit keys: [a] [e] [i] [ie] [l] [n] [na] [ne] [y]

Index: telephoneNumber.telephoneNumberSubstringsMatch:6
Over index-entry-limit keys: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Index: sn.caseIgnoreSubstringsMatch:6
Over index-entry-limit keys: [a] [an] [e] [er] [i] [k] [l] [n] [o] [on] [r] [s] [t] [y]

Index: objectClass.objectIdentifierMatch
Over index-entry-limit keys: [inetorgperson] [organizationalperson] [person] [top]

    For example, every user entry has the object classes listed, and every user entry has an email address ending in .com, so those values are not specific enough to be used in search filters.

    A non-zero value in the Over column represents a tradeoff. As described above, this is usually a good tradeoff, not a problem to be solved.

    For a detailed explanation of each column of the output, refer to backendstat show-index-status.

  3. Start the server:

    $ start-ds

On over index-entry-limit keys

The settings for this directory server are a good tradeoff. Unless you are observing many unindexed searches targeting keys in the Over index-entry-limit keys lists, there’s no reason to change the index-entry-limit settings.

A search might be indexed even though some keys are over the limit.

For example, as shown above, the objectClass value inetorgperson is over the limit. Yet, a search with a filter like (&(cn=Babs Jensen)(objectclass=inetOrgPerson)) is indexed:

$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=user.1,ou=people,dc=example,dc=com \
 --bindPassword password \
 --baseDN dc=example,dc=com \
 "(&(cn=Babs Jensen)(objectclass=inetOrgPerson))" cn
dn: uid=bjensen,ou=People,dc=example,dc=com
cn: Barbara Jensen
cn: Babs Jensen

The search is indexed because the equality index for cn is not over the limit, so the search term (cn=Babs Jensen) is enough for DS to find a match using that index.

If you look at the debugsearchindex output, you observe how DS uses the cn index, and skips the objectclass index. The overall search is clearly indexed:

Show debugsearchindex output 
$ ldapsearch \
 --hostname localhost \
 --port 1636 \
 --useSsl \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --bindDN uid=admin \
 --bindPassword password \
 --baseDN dc=example,dc=com \
 "(&(cn=Babs Jensen)(objectclass=inetOrgPerson))" \
 debugsearchindex | sed -n -e "s/^debugsearchindex: //p"
{
  "baseDn": "dc=example,dc=com",
  "scope": "sub",
  "filter": "(&(cn=Babs Jensen)(objectclass=inetOrgPerson))",
  "maxCandidateSize": 100000,
  "strategies": [{
    "name": "BaseObjectSearchStrategy",
    "diagnostic": "not applicable"
  }, {
    "name": "VlvSearchStrategy",
    "diagnostic": "not applicable"
  }, {
    "name": "AttributeIndexSearchStrategy",
    "filter": {
      "query": "INTERSECTION",
      "rank": "EXACT_MATCH",
      "filter": "(&(cn=Babs Jensen)(objectclass=inetOrgPerson))",
      "subQueries": [{
        "query": "FIRST_OF",
        "rank": "EXACT_MATCH",
        "filter": "(cn=Babs Jensen)",
        "subQueries": [{
          "query": "EXACT_MATCH",
          "rank": "EXACT_MATCH",
          "filter": "(cn=Babs Jensen)",
          "index": "cn.caseIgnoreMatch",
          "key": "babs jensen",
          "diagnostic": "indexed",
          "candidates": 1
        }, {
          "query": "MATCH_ALL",
          "rank": "MATCH_ALL",
          "filter": "(cn=Babs Jensen)",
          "index": "cn.presence",
          "diagnostic": "skipped"
        }],
        "diagnostic": "indexed",
        "candidates": 1,
        "retained": 1
      }, {
        "query": "FIRST_OF",
        "rank": "OBJECT_CLASS_EQUALITY_MATCH",
        "filter": "(objectclass=inetOrgPerson)",
        "subQueries": [{
          "query": "OBJECT_CLASS_EQUALITY_MATCH",
          "rank": "OBJECT_CLASS_EQUALITY_MATCH",
          "filter": "(objectclass=inetOrgPerson)",
          "subQueries": [{
            "query": "EXACT_MATCH",
            "rank": "EXACT_MATCH",
            "index": "objectClass.objectIdentifierMatch",
            "key": "inetorgperson",
            "diagnostic": "skipped"
          }, {
            "query": "EXACT_MATCH",
            "rank": "EXACT_MATCH",
            "index": "objectClass.objectIdentifierMatch",
            "key": "2.16.840.1.113730.3.2.2",
            "diagnostic": "skipped"
          }],
          "diagnostic": "skipped"
        }, {
          "query": "MATCH_ALL",
          "rank": "MATCH_ALL",
          "filter": "(objectclass=inetOrgPerson)",
          "index": "objectClass.presence",
          "diagnostic": "skipped"
        }],
        "diagnostic": "skipped"
      }],
      "diagnostic": "indexed",
      "candidates": 1
    },
    "diagnostic": "indexed",
    "candidates": 1
  }],
  "final": 1
}

Index entry limit changes

In rare cases, the index entry limit might be too low for a certain key. This could manifest itself as a frequent, useful search becoming unindexed with no reasonable way to narrow the search.

You can change the index entry limit on a per-index basis. Do not do this in production unless you can explain and show why the benefits outweigh the costs.

Changing the index entry limit significantly can result in serious performance degradation. Be prepared to test performance thoroughly before you roll out an index entry limit change in production.

To configure the index-entry-limit for an index or a backend:

  • Use the dsconfig set-backend-index-prop command to change the setting for a specific backend index.

  • (Not recommended) Use the dsconfig set-backend-prop command to change the setting for all indexes in the backend.

Copyright © 2010-2024 ForgeRock, all rights reserved.