Upgrading Autonomous Identity

Autonomous Identity 2020.10.0 provides upgrade commands to update your core software to the latest version while migrating your data.

The upgrade assumes the following:

  • Database Systems are the Same. If your current database is Apache Cassandra, you cannot upgrade to a MongoDB-based system. You will need to run a clean installation with the new version.

  • Host IPs should be the Same. Host IP addresses must be the same for existing components. You must update the ~/autoid-config/hosts file by adding the IP addresses for the Elasticsearch entries. See the instructions below.

  • Registry Key Required. To download the deployment images for the upgrade, you still need a registry key to log into the ForgeRock Google Cloud Registry (gcr.io). The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

  • Additional CSV Files Requires. The upgraded system requires the app_attributes.csv and ent_attributes.csv for attribute filtering on the Applications page. Copy these files to the /data/input directory. For more information on these files, see Data Preparation.

Upgrade to version 2020.10.0:

  1. On the deployer machine, back up the 2020.6.x ~/autoid-config directory or move it to another location.

    $ mv ~/autoid-config ~/backup-2020.6
  2. Create a new ~/autoid-config directory.

    $ mkdir ~/autoid-config
  3. Remove your known_files.

    $ rm ~/.ssh/known_hosts
  4. Copy your original SSH key into the new directory.

    $ cp ~/.ssh/id_rsa ~/autoid-config
  5. Change the permission on the SSH key.

    $ chmod 400 ~/autoid-config/id_rsa
  6. Check if you can successfully SSH to the target server.

    $ ssh autoid@<Target-IP-Address>
    Last login: Sun Sep 27 18:19:14 2020
  7. Stop the stack:

    $ docker stack rm configuration-service consul-server nginx openldap selfservice swagger-ui ui api consul-client

    You should see:

    Removing service configuration-service_configuration-service
    Removing service consul-server_consul-server
    Removing service nginx_nginx
    Removing service openldap_openldap
    Removing service openldap_phpldapadmin
    Removing service selfservice_selfservice
    Removing service swagger-ui_swagger-ui
    Removing service ui_zoran-ui
    Removing service api_zoran-api
    Removing service consul-client_consul-client
  8. Remove the contents of the consul data:

    $ sudo rm -r /opt/autoid/mounts/consul-data/*
  9. Enter exit to end your SSH session.

  10. On the deployer node, change to the ~/autoid-config directory.

    $ cd ~/autoid-config
  11. Log in to the ForgeRock Google Cloud Registry (gcr.io) using the registry key. The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

    $ docker login -u _json_key -p "$(cat autoid_registry_key.json)" https://gcr.io/forgerock-autoid

    You should see:

    Login Succeeded
  12. Run the create-template command to generate the deployer.sh script wrapper and configuration files. Note that the command sets the configuration directory on the target node to /config. The --user parameter eliminates the need to use sudo while editing the hosts file and other configuration files.

    $ docker run --user=`id -u` -v ~/autoid-config:/config -it gcr.io/forgerock-autoid/deployer:2020.10.0 create-template
        ...
    d6c7c6f3303e: Pull complete
    Digest: sha256:15225be65417f8bfb111adea37c83eb5e0d87140ed498bfb624a358f43fb48bf
    Status: Downloaded newer image for gcr.io/forgerock-autoid/autoid/dev-compact/deployer@sha256:15225be65417f8bfb111a
    dea37c83eb5e0d87140ed498bfb624a358f43fb48bf
    Config template is copied to host machine directory mapped to /config
  13. Make the script executable.

    $ chmod +x deployer.sh
  14. Configure your upgraded system by editing the ~/autoid-config/vars.yml, ~/autoid-config/hosts, and ~/autoid-config/vault.yml files on the deployer machine.

    The key here is to keep your configuration settings consistent from one system to another. For example, if your hosts file from your 2020.6.x system is as follows:

    [docker-managers]
    34.70.190.144
    
    [docker-workers]
    34.70.190.144
    
    [docker:children]
    docker-managers
    docker-workers
    
    [cassandra-seeds]
    34.70.190.144
    
    [cassandra-workers]
    34.70.190.144
    
    [spark-master]
    34.70.190.144
    
    [spark-workers]
    34.70.190.144
    
    [analytics]
    34.70.190.144

    An example 2020.10.0 hosts file would be as follows (single-node example):

    [docker-managers]
    34.70.190.144
    
    [docker-workers]
    34.70.190.144
    
    [docker:children]
    docker-managers
    docker-workers
    
    [cassandra-seeds]
    34.70.190.144
    
    [cassandra-workers]
    34.70.190.144
    
    [spark-master]
    34.70.190.144
    
    [spark-workers]
    34.70.190.144
    
    [analytics]
    34.70.190.144
    
    [mongo_master]
    #ip#  mongodb_master=True
    
    [mongo_replicas]
    #ip-1#
    ##ip-2#
    ##...
    
    [mongo:children]
    mongo_replicas
    mongo_master
    
    # ELastic Nodes
    [odfe-master-node]
    34.70.190.144
    
    [odfe-data-nodes]
    34.70.190.144
    
    [kibana-node]
    34.70.190.144
  15. Download the images. This step downloads software dependencies needed for the deployment and places them in the autoid-packages directory. Make sure you are in the ~/autoid-config directory.

    $ ./deployer.sh download-images

    You should see:

    PLAY RECAP ***********************************************************************************************
    localhost          : ok=24   changed=17   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
  16. Run the upgrade.

    $ ./deployer.sh upgrade

    You should see:

    PLAY RECAP ********************************************************************************************************
    <Target-IP-Addr>   : ok=313  changed=153  unreachable=0    failed=0    skipped=15   rescued=0    ignored=0
    localhost          : ok=11   changed=5    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0
  17. SSH to the target node.

  18. Check your Java version. Autonomous Identity version 2020.6.x uses OpenJDK 8u252. Autonomous Identity version 2020.10.0 uses OpenJDK 8u262.

    $ java -version

    If you Java version is 8u252, reload your .bashrc file:

    $ source ~/.bashrc
  19. Stop and restart Cassandra to apply the new 2020.10.0 certificates.

    1. Search for the Cassandra PID.

      $ ps -ef | grep cassandra
    2. Kill the process.

      $ sudo kill -9 <pid>
    3. Start Cassandra.

      $ /opt/autoid/apache-cassandra-3.11.2/bin/cassandra

      You should see:

      ...
      INFO  [main] 2020-11-03 04:15:12,737 Gossiper.java:1670 - Waiting for gossip to settle...
      INFO  [main] 2020-11-03 04:15:20,738 Gossiper.java:1701 - No gossip backlog; proceeding
    4. Check the Cassandra status.

      $ /opt/autoid/apache-cassandra-3.11.2/bin/nodetool status
  20. Stop and restart Apache Spark to apply the new 2020.10.0 certificates.

    1. Stop the Spark master and workers.

      $ /opt/autoid/spark/spark-2.4.4-bin-hadoop2.7/sbin/stop-all.sh

      You should see:

      localhost: stopping org.apache.spark.deploy.worker.Worker
      stopping org.apache.spark.deploy.master.Master
    2. Start the Spark master and workers.

      $ /opt/autoid/spark/spark-2.4.4-bin-hadoop2.7/sbin/start-all.sh

      You should see:

      starting org.apache.spark.deploy.master.Master, logging to /opt/autoid/spark/spark-2.4.4-bin-hadoop2.7/logs/spark-a
      utoid-org.apache.spark.deploy.master.Master-1.out
    3. Check the Spark status.

      $ elinks http://localhost:8080
  21. On the target node, take a backup of the /data/conf directory. This directory holds the configuration files used in 2020.6.x.

    $ cp -r /data/conf <backup_directory>
  22. On the Cassandra node, do the following:

    1. Start a CQL shell.

      $ /opt/autoid/apache-cassandra-3.11.2/bin/cqlsh --username="zoran_dba" --password=<Cassandra Admin Password> --ssl
      Connected to Zoran Cluster at 10.128.0.71:9042.
      [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
      Use HELP for help.
    2. Switch to the zoran keyspace.

      zoran_dba@cqlsh> USE autoid;
    3. View your DB tables. We want to locate the user table.

      zoran_dba@cqlsh:autoid> DESCRIBE TABLES;
    4. Drop the user table.

      zoran_dba@cqlsh:autoid> DROP TABLE user;
    5. Enter EXIT to terminate the CQL shell.

  23. Run analytics. This step creates a template from the new analytics image.

    $ analytics create-template
  24. Edit /data/conf/analytics_init_config.yml file if you made changes to this file in your previous deployment.

  25. Copy the app_attributes.csv and ent_attributes.csv to the /data/input directory.

    $ cp *.csv /data/input
  26. Apply the analytics template:

    $ analytics apply-template
  27. Upgrade the analytics:

    $ analytics upgrade

    You should see:

    Script : /home/analytics/upgrade/db_migration.py is successful
  28. Edit the ingestion section in the /data/conf/analytics_init_config.yml file. Add the tables property and set it to app_attributes, ent_attributes prior to data ingestion.

    ingestion:
      drop_if_create: true
      tables: app_attributes, ent_attributes
      catalog_step: false
      staging: false
      connector:
        type: csv
        connector-oim:
          type: oim
          timeout: 15
          batchsize: 1000
          change_reconciliation:
  29. Run data ingestion:

    $ analytics ingest

    You should see:

    Script : /home/analytics/autoid-analytics/ai_ingest.py is successful
  30. Publish the data.

    $ analytics publish

    You should see:

    Script : /home/analytics/autoid-analytics/ai_load.py is successful
  31. Recreate the index:

    $ analytics create-assignment-index

    You should see:

    Script : CreateElasticIndex is successful
  32. Edit the /data/conf/analytics_init_config.yml file again to reset the tables property to all.

    ingestion:
      drop_if_create: true
      tables: all
      catalog_step: false
      staging: false
      connector:
        type: csv
        connector-oim:
          type: oim
          timeout: 15
          batchsize: 1000
          change_reconciliation:

    You have successfully upgraded your Autonomous Identity server to 2020.10.

Access the Dashboard

Access the Autonomous Identity console UI:

  1. Open a browser, and point it to https://autoid-ui.forgerock.com/ (or your customized URL: https://myid-ui.abc.com).

  2. Log in as a test user: bob.rodgers@forgerock.com. Enter the password: Welcome123.

Read a different version of :