Duplicate key [cot_name] after importing SAML2 metadata in AM 6.5.x or upgrading

Last updated Feb 24, 2021

The purpose of this article is to provide assistance if you encounter a "java.lang.IllegalStateException: Duplicate key [cot_name]" error after importing SAML2 metadata or upgrading to a later version of AM.

Symptoms

You encounter the Duplicate key [cot_name] error in the AM console (when trying to access an entity provider or import one), after upgrading or during a federation flow. In all cases you will notice duplicate Circle of Trust (COT) attributes if you look at the extended metadata. For example, there are two entries for examplecot here:

<Attribute name="cotlist"> <Value>examplecot</Value> </Attribute> <Attribute name="cotlist"> <Value>adifferentcot</Value> </Attribute> <Attribute name="cotlist"> <Value>examplecot</Value> </Attribute>

Console issues

Cannot access the entity provider via the console or importing the entity provider via the console fails.

The following error is shown in the Configuration debug log when this happens:

amConsole:12/12/2020 12:17:28:174 PM GMT: Thread[http-nio-18080-exec-2,5,main]: TransactionId[2d7a6150-7788-4574-ae27-7f9c427e1303-8211] ERROR: ConsoleServletBase.onUncaughtException java.lang.IllegalStateException: Duplicate key [cot_name] at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) at java.util.HashMap.merge(HashMap.java:1254) at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at com.sun.identity.saml2.meta.SAML2MetaUtils.getAttributes(SAML2MetaUtils.java:222) ...

Upgrade issue

Upgrade cannot proceed. After deploying the new war file and restarting the web application container in which AM runs, the Upgrade Available page appears but the Upgrade to Access Management 6.5.x.x button is disabled.

You will see a similar error to the one above in the amUpgrade log when this happens:

amUpgrade:12/12/2020 10:21:42:223 AM GMT: Thread[default task-67,5,main]: TransactionId[4ee579d3-4256-a9d5-aef8-0011354ef37e-81] ERROR: An error occurred while finding SAMLv2 IdP and SP entries that require update java.lang.IllegalStateException: Duplicate key [cot_name] at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)

Federation failure

Federation fails with an Internal Server Error (500) and you will see the following error in the Federation debug log:

libSAML2:12/12/2020 11:16:06:041 AM GMT: Thread[http-nio-8080-exec-4,5,main]: TransactionId[44d5e173-ec2f-4ec7-8a60-d9be9e066068-5460] ERROR: Error processing request │ java.lang.IllegalStateException: Duplicate key [cot_name] at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) at java.util.HashMap.merge(HashMap.java:1254)

Recent Changes

Imported SAML2 metadata via the console.

Upgraded to AM 6.5.x

Causes

The extended metadata contains duplicate cotlist attributes.

In AM 6.5 and later, verification takes place to ensure duplicate circle of trust entries do not exist. If they do, a failure occurs and the federation related activity (such as editing an entity provider via the console or a federation flow) fails. This verification did not happen in previous versions, which is why these issues can be seen after upgrading even when the metadata has not changed.

Solution

This issue can be resolved as follows depending on where the failure is happening:

If you are trying to import the entity provider via the console and it fails, edit the extended metadata to remove the duplicate cotlist attribute and import it again.
If you are trying to upgrade but cannot proceed because of this exception, you need to correct the metadata as described below in the pre-upgrade version and then retry the upgrade. Alternatively, you can delete the entity provider in the pre-upgrade version, upgrade and then re-create the entity provider. You can do this via ssoadm as described below or via the console as detailed in SAML v2.0 Guide › Configuring Identity Providers, Service Providers, and Circles of Trust.
If you cannot access the entity provider via the console or federation is failing, you need to correct the metadata as described below.

Note

It is recommended you use ssoadm to modify the metadata files because there is a known issue with using REST: OPENAM-16403 (REST - updating federation entities could lead to empty 'cotlist' value).

Correcting the metadata

Back up the AM configuration as described in How do I make a backup of configuration data in AM 5.x, 6.x or OpenAM 13.x?
Export the entity provider's standard and extended metadata files using the following ssoadm command: $ ./ssoadm export-entity -u [adminID] -f [passwordfile] -e [realmname] -y [entityID] -c saml2 -m [metadataXMLfile] -x [extendedXMLfile]replacing [adminID], [passwordfile], [realmname], [entityID], [metadataXMLfile] and [extendedXMLfile] with appropriate values.
Remove all the duplicate cotlist attributes from the extended metadata file so that each cotlist attribute is unique, for example: <Attribute name="cotlist"> <Value>examplecot</Value> </Attribute> <Attribute name="cotlist"> <Value>adifferentcot</Value> </Attribute>
Remove the entity provider before importing the modified metadata files using the following ssoadm command: $ ./ssoadm delete-entity -u [adminID] -f [passwordfile] -e [realmname] -y [entityID] -c saml2replacing [adminID], [passwordfile], [realmname] and [entityID] with appropriate values.
Import the modified standard and extended metadata files into AM using the following ssoadm command to re-create the entity provider: $ ./ssoadm import-entity -u [adminID] -f [passwordfile] -e [realmname] -t [entityCOT] -c saml2 -m [metadataXMLfile] -x [extendedXMLfile]replacing [adminID], [passwordfile], [realmname], [entityCOT], [metadataXMLfile] and [extendedXMLfile] with appropriate values.

Related Training

N/A

Related Issue Tracker IDs

OPENAM-16712 (Importing SAML2 Metadata with both IDP and SP with cot ends up with duplicated extended metadata)

OPENAM-13942 (SAML2 Circle of Trust - REST Update doesn't update the metadata of the provider)