Autonomous Identity 2021.3.5

Upgrading Autonomous Identity

Autonomous Identity provides an upgrade command to update your core software to the latest version while migrating your data.

The upgrade assumes the following:

  • Database Systems are the Same. If your current database is Apache Cassandra, you cannot upgrade to a MongoDB-based system. You will need to run a clean installation with the new version.

  • Host IPs should be the Same. Host IP addresses must be the same for existing components. You must update the ~/autoid-config/hosts file by adding the IP addresses for the Elasticsearch entries. See the instructions below.

  • Registry Key Required. To download the deployment images for the upgrade, you still need a registry key to log into the ForgeRock Google Cloud Registry (gcr.io). The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

Upgrade from Autonomous Identity 2021.3.x to 2021.3.5

The following instruction is for upgrade from Autonomous Identity 2021.3.x (2021.3.0, 2021.3.1, 2021.3.2, 2021.3.3, or 2021.3.4) to version 2021.3.5.

Upgrade to version 2021.3.5:

  1. Start on the target server, and back up your /data/conf configuration file. The upgrade overwrites this file when updating, so you must restore this file after running the upgrade.

    $ sudo mv /data/conf ~/backup-data-conf-2021.3.x

  2. Next, if you changed any analytic settings on your deployment, make note of your configuration, so that you can replicate those settings on the upgraded server. Log in to Autonomous Identity, navigate to Administration > Analytic Settings, and record your settings.

  3. On the deployer machine, back up the 2021.3.x ~/autoid-config directory or move it to another location.

    $ mv ~/autoid-config ~/backup-2021.3.x

  4. Create a new ~/autoid-config directory.

    $ mkdir ~/autoid-config

  5. Copy your autoid_registry_key.json, ansible.cfg, and vault.yml files from your backup directory to ~/autoid-config. If your vault.yml file is encrypted, copy the .autoid_vault_password file to ~/autoid-config.

  6. Copy your original SSH key into the new directory.

    $ cp ~/.ssh/id_rsa ~/autoid-config

  7. Change the permission on the SSH key.

    $ chmod 400 ~/autoid-config/id_rsa

  8. Check if you can successfully SSH to the target server.

    $ ssh -i ~/autoid-config/id_rsa autoid@<Target-IP-Address>

Last login: Wed Dec 15 18:19:14 2021

  9. Stop the stack.

    If you are upgrading a multi-node deployment, run this command on the Docker Manager node. 
    $ docker stack rm configuration-service consul-server consul-client nginx jas openldap selfservice swagger-ui ui api

    You should see:

    Removing service configuration-service_configuration-service
Removing service consul-server_consul-server
Removing service consul-client_consul-client
Removing service nginx_nginx
Removing service jas_jasnode
Removing service openldap_openldap
Removing service openldap_phpldapadmin
Removing service selfservice_selfservice
Removing service swagger-ui_swagger-ui
Removing service ui_zoran-ui
Removing service api_zoran-api

  10. Back up the /data/conf directory. This directory holds the configuration files used in 2021.3.x.

    $ cp -r /data/conf <backup-directory>

  11. Remove the analytics container of the analytics node:

    $ docker rm -f analytics

  12. Enter exit to end your SSH session.

  13. Repeat the restart Docker command:

    $ sudo systemctl restart docker

  14. On the deployer node, change to the ~/autoid-config directory.

    $ cd ~/autoid-config

  15. Log in to the ForgeRock Google Cloud Registry (gcr.io) using the registry key. The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

    $ docker login -u _json_key -p "$(cat autoid_registry_key.json)" https://gcr.io/forgerock-autoid

    You should see:

    Login Succeeded

  16. Run the create-template command to generate the deployer.sh script wrapper and configuration files. Note that the command sets the configuration directory on the target node to /config . The --user parameter eliminates the need to use sudo while editing the hosts file and other configuration files.

    $ docker run --user=$(id -u) -v ~/autoid-config:/config \
-it gcr.io/forgerock-autoid/deployer:2021.3.5 create-template

…​
d6c7c6f3303e: Pull complete
Digest: sha256:15225be65417f8bfb111adea37c83eb5e0d87140ed498bfb624a358f43fbbf
Status: Downloaded newer image for gcr.io/forgerock-autoid/autoid/dev-compact/deployer@sha256:15225be65417f8bfb111a
dea37c83eb5e0d87140ed498bfb624a358f43fbbf
Config template is copied to host machine directory mapped to /config

  17. Configure your upgraded system by editing the ~/autoid-config/vars.yml , ~/autoid-config/hosts , and ~/autoid-config/vault.yml files on the deployer machine.

    You must keep your configuration settings consistent from one system to another.

  18. Download the images. This step downloads software dependencies needed for the deployment and places them in the autoid-packages directory. Make sure you are in the ~/autoid-config directory.

    $ ./deployer.sh download-images

  19. SSH to the target node.

  20. Stop the Spark master and workers.

    $ /opt/autoid/spark/spark-2.4.4-bin-hadoop2.7/sbin/stop-all.sh

    You should see:

    localhost: stopping org.apache.spark.deploy.worker.Worker
stopping org.apache.spark.deploy.master.Master

  21. Exit your SSH session.

  22. Run the upgrade.

    1. If you are upgrading fron version 2021.3.0, 2021.3.1, or 2021.3.2, run the following:

      $ ./deployer.sh upgrade

$ docker stack rm configuration-service consul-server consul-client nginx jas openldap selfservice swagger-ui ui api

$ ./deployer.sh debug patch_log4j

    2. If you are upgrading from version 2021.3.3 or 2021.3.4, run the following:

      $ ./deployer.sh debug patch_log4j

  23. SSH to the target server.

  24. On the target server, restore your /data/conf configuration file from your previous installation.

    $ sudo mv ~/backup-data-conf-2021.3.x /data/conf

  25. On the target server, edit the /opt/autoid/res/jas/docker-compose.yml and set the JAS_AUTH_ENABLED to true.

    If you freshly installed 2021.3.5 or run an upgrade where the log4j patch was not applied previously, you can skip steps 24–27. 
    JAS_AUTH_ENABLED=true

  26. Restart the JAS container.

    $ docker stack rm jas

$ docker stack deploy -c /opt/autoid/res/jas/docker-compose.yml jas

  27. Remove the nginx container.

    $ docker stack rm nginx

  28. Redeploy the stack.

    $ docker stack deploy -c /opt/autoid/res/nginx/docker-compose.yml nginx

  29. Add a reference to Autonomous Identity JAS to your /etc/hosts or DNS server.

    <Public IP Address> autoid-ui.forgerock.com autoid-selfservice.forgerock.com autoid-jas.forgerock.com

  30. Re-apply your analytics settings to your upgraded server if you made changes on your previous Autonomous Identity machine. Log in to Autonomous Identity, navigate to Administration > Analytics Settings, and edit your changes.

  31. Log out and then log back in to Autonomous Identity.

You have successfully upgraded your Autonomous Identity server to 2021.3.5.

Upgrade from Autonomous Identity 2021.3.x to 2021.3.5 Air-Gapped

The following instructions are for upgrading from Autonomous Identity version 2021.3.x to 2021.3.5 on an air-gapped deployment.

Upgrade to version 2021.3.5 Air-Gapped:

  1. Start on the target server, and back up your /data/conf configuration file. The upgrade overwrites this file when updating, so you must restore this file after running the upgrade.

    $ sudo mv /data/conf ~/backup-data-conf-2021.3.x

  2. Next, if you changed any analytic settings on your deployment, make note of your configuration, so that you can replicate those settings on the upgraded server. Log in to Autonomous Identity, navigate to Administration > Analytic Settings, and record your settings.

  3. On the deployer machine, back up the 2021.3.x ~/autoid-config directory or move it to another location.

    $ mv ~/autoid-config ~/backup-2021.3.x

  4. Create a new ~/autoid-config directory.

    $ mkdir ~/autoid-config

  5. Copy your autoid_registry_key.json, ansible.cfg, and vault.yml files from your backup directory to ~/autoid-config. If your vault.yml file is encrypted, copy the .autoid_vault_password file to ~/autoid-config.

  6. Copy your original SSH key into the new directory.

    $ cp ~/.ssh/id_rsa ~/autoid-config

  7. Change the permission on the SSH key.

    $ chmod 400 ~/autoid-config/id_rsa

  8. Stop the stack.

    If you are upgrading a multi-node deployment, run this command on the Docker Manager node. 
    $ docker stack rm configuration-service consul-server consul-client nginx jas openldap selfservice swagger-ui ui api

    You should see:

    Removing service configuration-service_configuration-service
Removing service consul-server_consul-server
Removing service consul-client_consul-client
Removing service nginx_nginx
Removing service jas_jasnode
Removing service openldap_openldap
Removing service openldap_phpldapadmin
Removing service selfservice_selfservice
Removing service swagger-ui_swagger-ui
Removing service ui_zoran-ui
Removing service api_zoran-api

  9. For multinode deployments, run the following on the Docker Worker node:

    $ docker swarm leave

  10. From the deployer, restart Docker:

    $ sudo systemctl restart docker

  11. On the deployer node, change to the ~/autoid-config directory.

    $ cd ~/autoid-config

  12. Log in to the ForgeRock Google Cloud Registry (gcr.io) using the registry key. The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

    $ docker login -u _json_key -p "$(cat autoid_registry_key.json)" https://gcr.io/forgerock-autoid

    You should see:

    Login Succeeded

  13. Run the create-template command to generate the deployer.sh script wrapper and configuration files. Note that the command sets the configuration directory on the target node to /config. The --user parameter eliminates the need to use sudo while editing the hosts file and other configuration files.

    $ docker run --user=$(id -u) -v ~/autoid-config:/config \
-it gcr.io/forgerock-autoid/deployer:2021.3.5 create-template

  14. Configure your upgraded system by editing the ~/autoid-config/vars.yml , ~/autoid-config/hosts , and ~/autoid-config/vault.yml files on the deployer machine.

    You must keep your configuration settings consistent from one system to another.

  15. Download the images. This step downloads software dependencies needed for the deployment and places them in the autoid-packages directory. Make sure you are in the ~/autoid-config directory.

    $ ./deployer.sh download-images

  16. Create a tar file containing all of the Autonomous Identity binaries.

    $ tar czf autoid-packages.tgz deployer.sh autoid-packages/*

  17. Copy the autoid-packages.tgz , deployer.sh , and SSH key (id_rsa ) to a portable hard drive.

  18. On the air-gapped target machine, backup your previous ~/autoid-config directory, and then create a new ~/autoid-config directory.

    $ mkdir ~/autoid-config

  19. Copy the autoid-package.tgz tar file from the portable storage device.

  20. Unpack the tar file.

    $ tar xf autoid-packages.tgz -C ~/autoid-config

  21. Copy the SSH key to the ~/autoid-config directory.

  22. Change the privileges to the file.

    $ chmod 400 ~/autoid-config/id_rsa

  23. Change to the configuration directory.

    $ cd ~/autoid-config

  24. Import the deployer image.

    $ ./deployer.sh import-deployer

    You should see:

    …​
db631c8b06ee: Loading layer [=============================================⇒]   2.56kB/2.56kB
2d62082e3327: Loading layer [=============================================⇒]  753.2kB/753.2kB
Loaded image: gcr.io/forgerock-autoid/deployer:2021.3.5

  25. Create the configuration template using the create-template command. This command creates the configuration files: ansible.cfg , vars.yml , vault.yml and hosts.

    $ ./deployer.sh create-template

    You should see:

    Config template is copied to host machine directory mapped to /config

  26. Configure your upgraded system by editing the ~/autoid-config/vars.yml , ~/autoid-config/hosts , and ~/autoid-config/vault.yml files on the deployer machine.

    You must keep your configuration settings consistent from one system to another.

  27. Run the upgrade.

    1. If you are upgrading fron version 2021.3.0, 2021.3.1, or 2021.3.2, run the following:

      $ ./deployer.sh upgrade

$ docker stack rm configuration-service consul-server consul-client nginx jas openldap selfservice swagger-ui ui api

$ ./deployer.sh debug patch_log4j

    2. If you are upgrading from version 2021.3.3 or 2021.3.4, run the following:

      $ ./deployer.sh debug patch_log4j

  28. SSH to the target server.

  29. On the target server, restore your /data/conf configuration file from your previous installation.

    $ sudo mv ~/backup-data-conf-2021.3.x /data/conf

  30. On the target server, edit the /opt/autoid/res/jas/docker-compose.yml and set the JAS_AUTH_ENABLED to true.

    If you freshly installed 2021.3.5 or run an upgrade where the log4j patch was not applied previously, you can skip steps 29–32. 
    JAS_AUTH_ENABLED=true

  31. Restart the JAS container.

    $ docker stack rm jas

$ docker stack deploy -c /opt/autoid/res/jas/docker-compose.yml jas

  32. Remove the nginx container.

    $ docker stack rm nginx

  33. Redeploy the stack.

    $ docker stack deploy -c /opt/autoid/res/nginx/docker-compose.yml nginx

  34. Add a reference to Autonomous Identity JAS to your /etc/hosts or DNS server.

    <Public IP Address> autoid-ui.forgerock.com autoid-selfservice.forgerock.com autoid-jas.forgerock.com

  35. Re-apply your analytics settings to your upgraded server if you made changes on your previous Autonomous Identity machine. Log in to Autonomous Identity, navigate to Administration > Analytics Settings, and edit your changes.

  36. Log out and then log back in to Autonomous Identity.

You have successfully upgraded your Autonomous Identity server to 2021.3.5.

Upgrade Autonomous Identity 2020.10.2 to 2021.3.5

The following instruction is for upgrade from Autonomous Identity 2021.10.2 to version 2021.3.5.

Upgrade from 2020.10.2 to 2021.3.5:

  1. On the deployer machine, back up the 2020.10.2 ~/autoid-config directory or move it to another location.

    $ mv ~/autoid-config ~/backup-2020.10

  2. Create a new ~/autoid-config directory.

    $ mkdir ~/autoid-config

  3. Remove your known_files .

    $ rm ~/.ssh/known_hosts

  4. Copy your original SSH key into the new directory.

    $ cp ~/.ssh/id_rsa ~/autoid-config

  5. Change the permission on the SSH key.

    $ chmod 400 ~/autoid-config/id_rsa

  6. Check if you can successfully SSH to the target server.

    $ ssh -i ~/autoid-config/id_rsa autoid@<Target-IP-Address>

Last login: Tue Feb 08 18:19:14 2020

  7. Stop the stack.

    $ docker stack rm configuration-service consul-server consul-client nginx openldap selfservice swagger-ui ui api

  8. Remove the contents of the consul data:

    $ sudo rm -r /opt/autoid/mounts/consul-data/*

  9. Enter exit to end your SSH session.

  10. From the deployer, restart Docker:

    $ sudo systemctl restart docker

  11. On the deployer node, change to the ~/autoid-config directory.

    $ cd ~/autoid-config

  12. Log in to the ForgeRock Google Cloud Registry (gcr.io) using the registry key. The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

    $ docker login -u _json_key -p "$(cat autoid_registry_key.json)" https://gcr.io/forgerock-autoid

    You should see:

    Login Succeeded

  13. Run the create-template command to generate the deployer.sh script wrapper and configuration files. Note that the command sets the configuration directory on the target node to /config . The --user parameter eliminates the need to use sudo while editing the hosts file and other configuration files.

    $ docker run --user=$(id -u) -v ~/autoid-config:/config -it gcr.io/forgerock-autoid/deployer:2021.3.5 create-template

  14. Configure your upgraded system by editing the ~/autoid-config/vars.yml , ~/autoid-config/hosts , and ~/autoid-config/vault.yml files on the deployer machine.

    The key here is to keep your configuration settings consistent from one system to another.

  15. Download the images. This step downloads software dependencies needed for the deployment and places them in the autoid-packages directory. Make sure you are in the ~/autoid-config directory.

    $ ./deployer.sh download-images

  16. SSH to the target node.

  17. Stop Apache Spark so that the deployer can upgrade the version to 3.0.1.

    • Stop the Spark master and workers.

      $ /opt/autoid/spark/spark-2.4.4-bin-hadoop2.7/sbin/stop-all.sh

  18. Exit your SSH session.

  19. On the deployer node, run the upgrade.

    $ ./deployer.sh upgrade

  20. On the target node, take a backup of the /data/conf directory. This directory holds the configuration files used in 2020.10.x.

    $ cp -r /data/conf <backup_directory>

  21. Change to the upgrade directory:

    $ cd /opt/autoid/apache-livy/analytics-artifacts/upgrade

  22. Edit the upgrade.yml file. Assuming a Cassandra database (MongoDB will have analogous properties), add the following values in the cassandra section:

    • Add the IP addresses of the Cassandra machines where the 2020.10.x data is stored next to the hosts propert.

    • Add the password for the Cassandra account.

    • Add the keystore password for spark.cassandra.connection.ssl.keyStore.password.

    • And the truststore passwords for spark.cassandra.connection.ssl.trustStore.password.

      upgrade:
  old_base:         'autoid_base'
  old_analytics:    'autoid_analytics'
  old_ui:           'autoid'
  steps:            ['base', 'analytics', 'ui']
  ui_steps:         ['ui_tables', 'ui_tables_with_batch_history', 'ui_tables_with_date_history', 'ui_tables_with_history_only']
  tracker_file:     /tmp/upgrade_tracker_file.yaml
  batch_restart:    False
spark:
  logging_level: FATAL
  config:
    spark.scheduler.mode: FAIR
    spark.executor.memory: 10G
    spark.driver.memory: 20G
    spark.driver.maxResultSize: 5G
cassandra:
  hosts: []
  port: 9042
  username: zoran_dba
  password:
  ssl:
    enabled: true
    python:
      keyfile: /opt/autoid/certs/zoran-cassandra-client-key.pem
      certfile: /opt/autoid/certs/zoran-cassandra-client-cer.pem
    spark:
      spark.cassandra.connection.ssl.enabled: true
      spark.cassandra.connection.ssl.clientAuth.enabled: true
      spark.cassandra.connection.ssl.keyStore.password:
      spark.cassandra.connection.ssl.trustStore.password:
      spark.cassandra.connection.ssl.keyStore.path: /opt/autoid/certs/zoran-cassandra-client-keystore.jks
      spark.cassandra.connection.ssl.trustStore.path: /opt/autoid/certs/zoran-cassandra-server-truststore.jks
eps:
  tenant:      autonomous-iam
  batch_size:   10000

  23. Change to the analytics directory.

    $ cd /opt/autoid/apache-livy/analytics

  24. Upgrade the analytics:

    $ analytics upgrade

  25. Open the 2020.10.2 /data/conf/analytics_init_config.yml file. You will need to get some properties from that file later.

  26. Log in to the 2021.3.5 Autonomous Identity as an Admin account. Navigate to Administration > Entity Definitions.

  27. Open the Identities definition, this page contains the definition for the User attribute entity used by Autonomous Identity.

  28. Open the 2020.10.2 /data/conf/analytics_init_config.yml file and take the list of properties stored as ui_config > user_column_descriptions.

  29. On the 2021.3.5 UI, click the Add attribute button, and the add each attribute to the list. Note: You will see that usr_id (replaces usr_key), usr_name, and usr_manager (replaces usr_manager_key) are already present.

  30. The Add attribute requires five pieces of information that you must add. You will add attributes listed under ui_config > user_column_descriptions:

    1. Attribute Name. This field corresponds to the value before the colon in your user_column_descriptions list. For example, if you have 'JOBCODE_NAME':'Job Code Name', use 'JOBCODE_NAME'.

    2. Display Name. This field corresponds to the value after the colon in your user_column_descriptions. For example, if you have 'JOBCODE_NAME':'Job Code Name', use 'JOBCODE_NAME'.

    3. Dropdown. The dropdown shows the data type. Note that only attributes marked as 'Text' can be used in machine learning.

    4. Use in machine learning. This indicates whether this attribute should be used in training. If the attribute you are adding appears in assoc_rules > features_filter from the /data/conf/analytics_init_config.yml file, then click this box.

    5. Searchable. This indicates whether this attribute can be used as a filter in the UI. if the attribute appears in ui_config > usr_filtering_columns, then click this box.

    6. Once you have added all the attributes, click Save at the top of the Identities table.

  31. Repeat steps 27-28 for the Applications entities. In this case, you will add the attributes listed under ui_config >app_filter_columns:

    1. Attribute Name. This field corresponds to the value before the colon in your app_filter_columns list. For example, if you have 'APP_CRITICALITY':'Application Criticality', use 'APP_CRITICALITY'.

    2. Display Name. This field corresponds to the value after the colon in your app_filter_columns. For example, if you have 'APP_CRITICALITY':'Application Criticality', use 'APP_CRITICALITY'.

    3. Dropdown. The dropdown shows the data type.

    4. Searchable. This indicates whether you want the attribute to be filterable in the UI.

  32. Repeat steps 27-28 for the Entitlements entities. In this case, you will add the attributes listed under ui_config >ent_filter_columns:

    1. Attribute Name. This field corresponds to the value before the colon in your ent_filter_columns list. For example, if you have 'ENT_RISK_LEVEL':'Entitlement Risk', use 'ENT_RISK_LEVEL'.

    2. Display Name. This field corresponds to the value after the colon in your ent_filter_columns. For example, if you have 'ENT_RISK_LEVEL':'Entitlement Risk', use 'ENT_RISK_LEVEL'.

    3. Dropdown. The dropdown shows the data type.

    4. Searchable. This indicates whether you want the attribute to be filterable in the UI.

  33. Repeat steps 27-28 for the Assignments entities. You can add attributes, such as ent_id, usr_id, high_risk, is_assigned, and last_usage.

  34. Navigate to Administration > Analytics Settings. Compare the values in the 2020.10.2 analytics_config.yml and modify if required. Make sure to save your settings. The mappings are shown below:

Autonomous Identity Threshold Mappings

2020.10.2 analytics_config.yml

2021.3 UI Administration > Analytics Settings

etl > med_conf

Confidence Score Thresholds > Medium

etl > high_conf

Confidence Score Thresholds > High

prediction > recommend

Recommendation Threshold > Threshold

  1. Run create-assignment-index to generate a new index using the migrated data:

    $ analytics create-assignment-index

    You have successfully upgraded your Autonomous Identity server to 2021.3.5.

Copyright © 2010-2022 ForgeRock, all rights reserved.