Install a Multi-Node Deployment

For each VM, make sure that you have CentOS 7 as your operating system. Check your CentOS 7 version.

$ sudo cat /etc/centos-release

Set the user for the target node to a username of your choice:

Change to the user account.

$ su - autoid

Install yum-utils package on the deployer machine. yum-utils is a utilities manager for the Yum RPM package repository. The repository compresses software packages for Linux distributions.

$ sudo yum install -y yum-utils

Change to the user account.

$ su - autoid

Create the installation directory. Note that you can use any install directory for your system as long as your run the deployer.sh script from there. Also, the disk volume where you have the install directory must have at least 8GB free space for the installation.

$ mkdir autoid-config

Set up the Docker-CE repository.

$ sudo yum-config-manager \
     --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Install the latest version of the Docker CE, the command-line interface, and containerd.io, a containerized website.

$ sudo yum install -y docker-ce docker-ce-cli containerd.io

Enable Docker to start at boot.

$ sudo systemctl enable docker

Start Docker.

$ sudo systemctl start docker

Check that Docker is running.

$ systemctl status docker

Add the user to the Docker group.

$ sudo usermod -aG docker ${USER}

Logout of the user account.

$ logout

Re-login using created user. Login with the user created for the deployer machine. For example, autoid.

$ su - autoid

On the deployer machine, change to the ~/.ssh directory.

$ cd ~/.ssh

Run ssh-keygen to generate an RSA keypair, and then click Enter. You can use the default filename.

$ ssh-keygen -t rsa -C "autoid"

Copy the SSH key to the autoid-config directory.

$ cp id_rsa ~/autoid-config

Change the privileges to the file.

$ chmod 400 ~/autoid-config/id_rsa

Copy your public SSH key, id_rsa.pub, to each of your nodes.

$ ssh-copy-id -i id_rsa.pub autoid@<Node IP Address>

On the deployer machine, test your SSH connection to each target machine. This is a critical step. Make sure the connection works before proceeding with the installation.

$ ssh -i id_rsa autoid@<Node 1 IP Address>

Last login: Sat Oct 3 03:02:40 2020

If you can successfully SSH to each machine, set the privileges on your ~/.ssh and ~/.ssh/authorized_keys.

$ chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys

Enter Exit to end your SSH session.

Repeat steps 5–8 again for each node.

On the Analytics Spark Master node, install nfs-utils. This step may require that you run the install with root privileges, such as sudo or equivalent.

$ sudo yum install -y nfs-utils

Create the /data directory.

$ mkdir -p /data

Change the permissions on the /data directory.

$ chmod -R 755 /data
$ chown nfsnobody:nfsnobody /data

Start the services and enable them to start at boot.

$ systemctl enable rpcbind
$ systemctl enable nfs-server
$ systemctl enable nfs-lock
$ systemctl enable nfs-idmap

$ systemctl start rpcbind
$ systemctl start nfs-server
$ systemctl start nfs-lock
$ systemctl start nfs-idmap

Define the sharing points in the /etc/exports file.

$ vi /etc/exports

/data  <Remote IP Address 1>(rw,sync,no_root_squash,no_all_squash)
/data  <Remote IP Address 2>(rw,sync,no_root_squash,no_all_squash)

Start the NFS service.

$ systemctl restart nfs-server

Add the NFS service to the firewall-cmd public zone service:

$ firewall-cmd --permanent --zone=public --add-service=nfs
$ firewall-cmd --permanent --zone=public --add-service=mountd
$ firewall-cmd --permanent --zone=public --add-service=rpc-bind
$ firewall-cmd --reload

On each spark worker node, run the following:

SSH to a Cassandra seed machine.

Change to the default user. In this example, autoid.

Open a text editor and edit the /etc/sysctl.conf file. Add the following settings to the file:

net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.core.rmem_default=16777216
net.core.wmem_default=16777216
net.core.optmem_max=40960
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
vm.max_map_count=1048575

At runtime, make sure swap is not used. For a permanent change, modify /etc/fstab file.

$ sudo swapoff -a

Disable defrag of huge pages. The following command must be executed at each boot:

$ echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

Set the proper ulimits for the user running Cassandra by setting them in the /etc/security/limits.conf file. In this example, the user running Cassandra is autoid.

autoid - memlock unlimited
autoid - nofile 100000
autoid - nproc 32768
autoid - as unlimited

On the deployer machine, change to the installation directory.

$ cd ~/autoid-config/

Obtain the registry key for the ForgeRock Google Cloud Registry (gcr.io). The registry key is only available to ForgeRock Autonomous Identity customers. For specific instructions on obtaining the registry key, see How To Configure Service Credentials (Push Auth, Docker) in Backstage.

Log in to the ForgeRock Google Cloud Registry (gcr.io) using the registry key.

$ docker login -u _json_key -p "$(cat autoid_registry_key.json)" https://gcr.io/forgerock-autoid

Login Succeeded

Run the create-template command to generate the deployer.sh script wrapper. Note that the command sets the configuration directory on the target node to /config. Note that the --user parameter eliminates the need to use sudo while editing the hosts file and other configuration files.

$ docker run --user=$(id -u) -v ~/autoid-config:/config -it gcr.io/forgerock-autoid/deployer:2021.8.6 create-template

On the deployer node, change to the autoid-config/ directory.

Check the ansible.cfg file for the remote user and SSH private key file location. If you followed these instruction, the default settings should be in place, which will not require any edits to the ansible.cfg file.

[defaults]
host_key_checking = False
remote_user = autoid
private_key_file = id_rsa

On the deployer machine, open a text editor and edit the ~/autoid-config/vars.yml file to configure specific settings for your deployment:

Open a text editor and enter the private IP addresses of the target machines in the ~/autoid-config/hosts file. Make sure the target host IP addresses are accessible from the deployer machine. The following is an example of the ~/autoid-config/hosts file. NOTE: [notebook] is not used in Autonomous Identity.

[docker-managers]
10.128.0.90

[docker-workers]
10.128.0.170

[docker:children]
docker-managers
docker-workers

[cassandra-seeds]
10.128.0.175
10.128.0.34

[spark-master]
10.128.0.180

[spark-workers]
10.128.0.176
10.128.0.177

[spark:children]
spark-master
spark-workers

[mongo_master]
#ip#  mongodb_master=True

[mongo_replicas]
#ip-1#
##ip-2#
##…​

[mongo:children]
mongo_replicas
mongo_master

# ELastic Nodes
[odfe-master-node]
10.128.0.178

[odfe-data-nodes]
10.128.0.184

[kibana-node]
10.128.0.184

[notebook]
#ip#

Open a text editor and set the Autonomous Identity passwords for the configuration service, elasticsearch backend, and Cassandra or MongoDB database. The vault passwords file is located at ~/autoid-config/vault.yml.

configuration_service_vault:
    basic_auth_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&

cassandra_vault:
    cassandra_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    cassandra_admin_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    keystore_password: Acc#1234
    truststore_password: Acc#1234

mongo_vault:
    mongo_admin_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    mongo_root_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    keystore_password: Acc#1234
    truststore_password: Acc#1234

elastic_vault:
    elastic_admin_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    elasticsearch_password: ~@C~O>@%^()-_+=|<Y*$$rH&&/m#g{?-o!z/1}2??3=!*&
    keystore_password: Acc#1234
    truststore_password: Acc#1234

Encrypt the vault file that stores the Autonomous Identity passwords, located at ~/autoid-config/vault.yml. The encrypted passwords will be saved to /config/.autoid_vault_password . The /config/ mount is internal to the deployer container.

$ ./deployer.sh encrypt-vault

Download the images. This step downloads software dependencies needed for the deployment and places them in the autoid-packages directory.

$ ./deployer.sh download-images

Run the deployment.

$ ./deployer.sh run

SSH to a Cassandra seed node.

Change to the /opt/autoid/apache-cassandra-3.11.2/ directory.

Start the Cassandra sheel, cqlsh, and define the autoid keyspace. Change the replication factor to match the number of seed nodes. The default admin user for Cassandra is zoran_dba.

$ bin/cqlsh -u zoran_dba

$ zoran_dba@cqlsh> desc keyspace autoid;
CREATE KEYSPACE autoid WITH replication = {'class':'SimpleStrategy','replication_factor':'2'} AND durable_writes=true;

CREATE TABLE autoid.user_access_decisions_history(
  user text,
  entitlement text,
  date_created timestamp,
  …​

Restart Cassandra on this node.

Repeat these steps on the other Cassandra seed node(s).

Configure your DNS servers to access Autonomous Identity dashboard on the target node. The following domain names must resolve to the IP address of the target node:

<target-environment>-ui.<domain-name>

If DNS cannot resolve target node hostname, edit it locally on the machine that you want to access Autonomous Identity using a browser.

<Docker Mgr Node Public IP Address>  <target-environment>-ui.<domain-name>

<IP Address>  autoid-ui.forgerock.com

If you set up a custom domain name and target environment, add the entries in /etc/hosts. For example:

<IP Address>  myid-ui.abc.com

Open a browser. If you set up your own url, use it for your login.

$ https://autoid-ui.forgerock.com/

Log in as a test user.

test user: bob.rodgers@forgerock.com
password: <password>

On the target node, check the status of Apache Cassandra.

$ /opt/autoid/apache-cassandra-3.11.2/bin/nodetool status

An example output is as follows:

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address       Load       Tokens       Owns (effective)  Host ID                               Rack
UN  34.70.190.144 1.33 MiB   256          100.0%            a10a91a4-96e83dd-85a2-4f90d19224d9  rack1
--

On the target node, check the status of MongoDB.

$ mongo --tls \
--host <Host IP> \
--tlsCAFile /opt/autoid/mongo/certs/rootCA.pem  \
--tlsAllowInvalidCertificates  \
--tlsCertificateKeyFile /opt/autoid/mongo/certs/mongodb.pem

SSH to the target node and open Spark dashboard using the bundled text-mode web browser

$ elinks http://localhost:8080

Click to See an Example of the Spark Dashboard