Issues with indexes can be caused by one of the following:
- Degraded indexes
- Missing indexes
- Incomplete indexing
- Unnecessary indexes
- High index entry limits
- Poorly defined search filters
General health check
You can use one of the following commands (depending on DS/OpenDJ version) to provide a general health check on all indexes to help you evaluate how well attributes are indexed:
- DS / OpenDJ 3.x, you can use the following backendstat command; you
stop the server prior to running this command:
$ ./backendstat show-index-status --backendID userRoot --baseDN dc=example,dc=com
- OpenDJ 2.6.x, you can use the following dbtest command:
$ ./dbtest list-index-status --backendID userRoot --baseDN dc=example,dc=com
These commands return a list of all indexes and includes information about the type of index, the associated database name, whether the index is valid and whether the index has exceeded its index entry limit.
These commands can be expensive and take a long time to complete, as they reads all indexes for all backends.
Degraded indexes occur when you have created or made changes to indexes and the index becomes corrupt due to missing data. You can check for degraded indexes using the verify-index command. This is described in How do I verify indexes in DS/OpenDJ (All versions) are correct?
Any degraded indexes must be rebuilt. You can rebuild specific indexes, all indexes or only degraded indexes as described in How do I rebuild indexes in DS/OpenDJ (All versions)?
Missing indexes (unindexed) result in long search times (high etime values in the access log). This can even affect logging into AM/OpenAM if your user store is on DS/OpenDJ and an attribute involved in the login process is not indexed, for example, the LDAP SAML attribute.
Missing indexes need to be created and built.
See Unindexed searches causing slow searches and poor performance on DS/OpenDJ (All versions) server for further information on the symptoms and solution.
Incomplete indexing can also cause problems with slow searches, but sometimes it can be difficult to pinpoint which attribute is causing the issue if there are multiple attributes in the search filter.
You can use the debugsearchindex attribute with the ldapsearch command to identify issues such as unindexed attributes and exceeded entry limits. The debugsearchindex attribute tells DS/OpenDJ to return an entry containing debug information that describes how the search operation would have been processed (rather than actually performing the search). See the debugsearchindex example below for further information.
Unused or rarely used indexes can also impact performance as DS/OpenDJ has to maintain entries for all indexes.
You can run a script (topfilters) to find out which indexes are used most often to help you identify underused indexes. You can then remove these indexes using the dsconfig delete-backend-index or delete-local-db-index command depending on which version of DS/OpenDJ you are using.
The source for the topfilters script can be found here: Githutb: OpenDJ Utilities - topfilters.
Index entry limits are set to 4000 by default, which means that once the number of entries matching a given index exceeds this number, DS/OpenDJ stops maintaining the entries for that index as it is an inefficient use of resources. This default value should be sufficient for most indexes other than objectClass indexes.
Do not set the index-entry-limit to the number of entries in your backend.
If you have set your index's entry limits much higher than this for any of your indexes, you may find your performance is impacted as DS/OpenDJ has to maintain entries for very large indexes. You should try reducing the index entry limit back to the default value to see if this improves performance.
Typically you should aim to have specific search filters in your searches to produce more accurate matches and also reduce the time taken to search.
For example, a search filter such as (objectClass=person) is very generalized and will produce a lot of matches. Instead, you should focus your search by adding other attributes to your search filter that would produce the matches you require. For example, you could use a filter similar to this to reduce the number of matches:
You can use the information contained in the access logs to determine what you should include in your debugsearchindex. For example, the following sample access log snippets show high etimes for two different searches:
[19/Oct/2016:17:09:27 -0400] SEARCH conn=107918 op=1 msgID=2 base="ou=People,dc=example,dc=org" scope=one filter="(&(modifytimestamp>=20161019161001Z)(objectclass=person))" attrs="uid email" result=118 message="Client Disconnect" nentries=5 unindexed etime=16761 [19/Oct/2016:17:16:42 -0400] SEARCH conn=108320 op=1 msgID=2 base="ou=People,dc=example,dc=org" scope=one filter="(&(iscurrent=1)(objectclass=person))" attrs="uid email" result=118 message="Client Disconnect" nentries=9 unindexed etime=46961
You can then construct searches using this information to show which indexes are being used to help you identify incomplete or missing indexes. For example:
$ ./ldapsearch --hostname ds1.example.com --port 1389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN ou=People,dc=example,dc=org --searchScope one "(&(modifytimestamp>=20161019161001Z)(objectclass=person))" debugsearchindex dn: cn=debugsearch debugsearchindex: filter=(&(objectClass=person)[INDEX:objectClass.equality][LIMIT-EXCEEDED](modifyTimestamp>=20161019161001Z)[NOT-INDEXED])[NOT-INDEXED] scope=one[LIMIT-EXCEEDED:24482] final=[NOT-INDEXED]
$ ./ldapsearch --hostname ds1.example.com --port 1389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN ou=People,dc=example,dc=org --searchScope one "(&(iscurrent=1)(objectclass=person))" debugsearchindex dn: cn=debugsearch debugsearchindex: filter=(&(objectClass=person)[INDEX:objectClass.equality][LIMIT-EXCEEDED](iscurrent=1)[NOT-INDEXED])[NOT-INDEXED] scope=one[LIMIT-EXCEEDED:26021] final=[NOT-INDEXED]
See Administration Guide › Indexing Attribute Values › About debugsearchindex Values for further information on the results obtained from the debugsearchindex option.
Related Issue Tracker IDs