Troubleshooting

Kubernetes deployments are multi-layered and often complex.

Errors and misconfigurations can crop up in a variety of places. Performing a logical, systematic search for the source of a problem can be daunting.

Here are some techniques you can use to troubleshoot problems with CDK and CDM deployments:

Problem Troubleshooting Technique

Problem	Troubleshooting Technique
Pods in the CDK or CDM don’t start up as expected.	Review pod descriptions and container logs. See if your cluster is resource-constrained. Check for underconfigured clusters by using the `kubectl describe nodes` and `kubectl get events -w` commands. Pods killed with out of memory (OOM) conditions indicate that your cluster is underconfigured. Make sure that you’re using tested versions of third-party software. Stage your deployment. Install ForgeRock Identity Platform components separately, instead of installing all the components with a single command. Staging your deployment lets you make sure each component works correctly before installing the next component.
All the pods have started, but you can’t reach the services running in them.	Make sure you don’t have any ingress issues.
AM doesn’t work as expected.	Set the AM logging level, recreate the issue, and analyze the AM log files.
IDM doesn’t work as expected.	Set the IDM logging level, recreate the issue, and analyze the AM log files.
Your JVM crashed with an out of memory error or you suspect that you have a memory leak.	Collect and analyze Java thread dumps and heap dumps.
Changes you’ve made to ForgeRock’s Kustomize files don’t work as expected.	Fully expand the Kustomize output, and then examine the output for unintended effects.
Your Minikube deployment doesn’t work.	Make sure that you don’t have a problem with virtual hardware requirements.
Skaffold doesn’t run as expected.	If Skaffold cannot push a Docker image, review your push setup. For other problems with Skaffold, you can try increasing Skaffold’s logging verbosity.
You’re having name resolution or other DNS issues.	Use diagnostic tools in the debug tools container.
You want to run DS utilities without disturbing a DS pod.	Use the bin/ds-debug.sh script or DS tools in the debug tools container.
You want to keep the `amster` pod running to diagnose AM configuration issues.	Use the amster command.
The `kubectl` command requires too much typing.	Enable kubectl tab autocompletion.

Pods in the CDK or CDM don’t start up as expected.

Review pod descriptions and container logs.

See if your cluster is resource-constrained. Check for underconfigured clusters by using the kubectl describe nodes and kubectl get events -w commands. Pods killed with out of memory (OOM) conditions indicate that your cluster is underconfigured.

Make sure that you’re using tested versions of third-party software.

Stage your deployment. Install ForgeRock Identity Platform components separately, instead of installing all the components with a single command. Staging your deployment lets you make sure each component works correctly before installing the next component.

All the pods have started, but you can’t reach the services running in them.

Make sure you don’t have any ingress issues.

AM doesn’t work as expected.

Set the AM logging level, recreate the issue, and analyze the AM log files.

IDM doesn’t work as expected.

Set the IDM logging level, recreate the issue, and analyze the AM log files.

Your JVM crashed with an out of memory error or you suspect that you have a memory leak.

Collect and analyze Java thread dumps and heap dumps.

Changes you’ve made to ForgeRock’s Kustomize files don’t work as expected.

Fully expand the Kustomize output, and then examine the output for unintended effects.

Your Minikube deployment doesn’t work.

Make sure that you don’t have a problem with virtual hardware requirements.

Skaffold doesn’t run as expected.

If Skaffold cannot push a Docker image, review your push setup.

For other problems with Skaffold, you can try increasing Skaffold’s logging verbosity.

You’re having name resolution or other DNS issues.

Use diagnostic tools in the debug tools container.

You want to run DS utilities without disturbing a DS pod.

Use the bin/ds-debug.sh script or DS tools in the debug tools container.

You want to keep the amster pod running to diagnose AM configuration issues.

Use the amster command.

The kubectl command requires too much typing.

Enable kubectl tab autocompletion.

ForgeOps

Troubleshooting