List index out of range: -1 error in IDM (All versions) when deployed in a cluster
The purpose of this article is to provide assistance if you encounter a "List index out of range: -1" error when multiple IDM instances are deployed in a cluster using DS as the repository. This error only occurs if you are using DS 7.1 or earlier as your repository.
Symptoms
An error similar to one of the following is shown in the IDM logs:
- Error performing cluster manager thread logic:[81] Nov 01, 2022 4:29:32.172 PM org.forgerock.openidm.cluster.ClusterManager$ClusterManagerThread$1 run SEVERE: Error performing cluster manager thread logic org.forgerock.json.JsonValueException: /: List index out of range: -1 at org.forgerock.json.JsonValue.put(JsonValue.java:1074) at org.forgerock.json.JsonValue.put(JsonValue.java:1129) at org.forgerock.openidm.repo.opendj.impl.GenericDJTypeHandler.transformFullObjectOutput(GenericDJTypeHandler.java:160) at org.forgerock.openidm.repo.opendj.impl.GenericDJTypeHandler.lambda$new$0(GenericDJTypeHandler.java:93) at org.forgerock.util.promise.Promises$CompletedPromise.then(Promises.java:191) at org.forgerock.util.promise.Promises$CompletedPromise.then(Promises.java:156) ... Caused by: org.forgerock.json.JsonValueException: /: List index out of range: -1
- Failed to query cluster nodes. Lock balancing aborted:[140] Nov 01, 2022 4:29:32.172 PM org.forgerock.openidm.sync.impl.queue.QueueConsumerFactory lambda$new$1 SEVERE: Failed to query cluster nodes. Lock balancing aborted. org.forgerock.json.resource.InternalServerErrorException: /: List index out of range: -1 at org.forgerock.opendj.rest2ldap.Rest2Ldap.asResourceException(Rest2Ldap.java:505) at org.forgerock.opendj.ldap.Utils.lambda$toPromise$6(Utils.java:234) at io.reactivex.rxjava3.internal.observers.ConsumerSingleObserver.onError(ConsumerSingleObserver.java:46) at io.reactivex.rxjava3.internal.operators.flowable.FlowableLastSingle$LastSubscriber.onError(FlowableLastSingle.java:94) at io.reactivex.rxjava3.internal.subscribers.BasicFuseableSubscriber.onError(BasicFuseableSubscriber.java:101) at io.reactivex.rxjava3.internal.subscribers.BasicFuseableSubscriber.onError(BasicFuseableSubscriber.java:101) at io.reactivex.rxjava3.internal.operators.flowable.FlowableDoOnEach$DoOnEachConditionalSubscriber.onError(FlowableDoOnEach.java:268) at io.reactivex.rxjava3.internal.subscribers.BasicFuseableConditionalSubscriber.fail(BasicFuseableConditionalSubscriber.java:110) at io.reactivex.rxjava3.internal.operators.flowable.FlowableDoOnEach$DoOnEachConditionalSubscriber.tryOnNext(FlowableDoOnEach.java:245) at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnConditionalSubscriber.runAsync(FlowableObserveOn.java:639) at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176) ... Caused by: org.forgerock.json.JsonValueException: /: List index out of range: -1
- Resource exception: 500 Internal Server Error:[72] Nov 01, 2022 4:29:32.172 PM org.forgerock.openidm.servlet.internal.ResourceFilters$3 lambda$handleRequestWithLogging$8 WARNING: Resource exception: 500 Internal Server Error: "/: List index out of range: -1" org.forgerock.json.resource.InternalServerErrorException: /: List index out of range: -1 at org.forgerock.opendj.rest2ldap.Rest2Ldap.asResourceException(Rest2Ldap.java:505) ... Caused by: org.forgerock.json.JsonValueException: /: List index out of range: -1
The following response is received when querying the cluster endpoint using REST: { "code": 500, "reason": "Internal Server Error", "message": "/: List index out of range: -1" }
Recent Changes
N/A
Causes
There are multiple fr-idm-json
attributes on one or more entries in the DS repository. These duplicate attributes could be within the config tables (for example, a schedule trigger) or within the user data itself.
The fr-idm-json
attribute is meant to be a single-valued attribute, so it should only exist once on any entry. Improvements have been made in DS 7.2 to improve how DS handles updates to the fr-idm-json
attribute to prevent this situation from occurring.
Solution
This issue can be resolved by manually removing the duplicate fr-idm-json
attributes from entries in the DS repository. You should work with your DS administrator to remove the oldest attributes where there are duplicates and just leave the newest one.
Once you have removed the duplicate entries, you should consider upgrading your repository to DS 7.2 or later to take advantage of the improvements made to the way this attribute is handled; you can download this from Backstage.
Ensure your resulting IDM and DS versions remain compatible: What versions of DS are compatible with IDM?
Finding duplicate entries
You can check for duplicate fr-idm-json
attributes by performing a ldapsearch on each of the DS instances, for example:
- DS 7.1 and later: $ ./ldapsearch --hostname localhost --port 1636 --useSsl --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --bindDN uid=admin --bindPassword password --basedn dc=openidm,dc=forgerock,dc=com -s sub "(fr-idm-json=*)" dn fr-idm-json
- DS 7: $ ./ldapsearch --hostname localhost --port 1636 --useSsl --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --bindDN uid=admin --bindPassword password --basedn dc=openidm,dc=forgerock,dc=com -s sub "(fr-idm-json=*)" dn fr-idm-json
Example response showing duplicate fr-idm-json
attributes on the uid=node1,ou=states,ou=cluster,dc=openidm,dc=forgerock,dc=com entry:dn: uid=node1,ou=states,ou=cluster,dc=openidm,dc=forgerock,dc=com
fr-idm-json: {"recoveryAttempts":1,"detectedDown":"0000000000000000000","type":"state","recoveryFinished":"0000000000000000000","instanceId":"node1","startup":"0000001668594727000","recoveringInstanceId":"null","state":1,"recoveringTimestamp":"0000000000000000000","recoveryStarted":"0000000000000000000","shutdown":"0000000000000000000","timestamp":"0000001668595405000"}
fr-idm-json: {"recoveryAttempts":0,"detectedDown":"0000000000000000000","type":"state","recoveryFinished":"0000000000000000000","instanceId":"node1","startup":"0000001668088349000","recoveringInstanceId":null,"state":1,"recoveringTimestamp":"0000000000000000000","recoveryStarted":"0000000000000000000","shutdown":"0000000000000000000","timestamp":"0000001668089407000"}If required, you can use an epoch converter to see which fr-idm-json
timestamps are the oldest.
See Also
Best practice for clustering in IDM
Related Training
N/A