Pod Descriptions and Container Logs

Look at pod descriptions and container log files for irregularities that indicate problems.

Pod descriptions contain information about active Kubernetes pods, including their configuration, status, containers (including containers that have finished running), volume mounts, and pod-related events.

Container logs contain startup and run-time messages that might indicate problem areas. Each Kubernetes container has its own log that contains all output written to stdout by the application running in the container. The am container logs are especially important for troubleshooting AM issues in Kubernetes deployments. AM writes its debug logs to stdout. Therefore, the am container logs include all the AM debug logs.

Here’s an example of how you can use pod descriptions and container logs to troubleshoot. Events in the pod description indicate that Kubernetes was unsuccessful in pulling a Docker image required to run a container. You can review your Docker registry’s configuration to determine whether a misconfiguration caused the problem.

The debug-logs.sh script generates the following HTML-formatted output, which you can view in a browser:

  • Descriptions of all the Kubernetes pods running the ForgeRock Identity Platform in your namespace

  • Logs for all of the containers running in these pods

Perform the following steps to run the debug-logs.sh script and view the output in a browser:

  1. Make sure that your namespace is the current namespace in your Kubernetes context.

  2. Make sure you’ve checked out the release/7.1.0 branch of the forgeops repository.

  3. Change to the /path/to/forgeops/bin directory in your forgeops repository clone.

  4. Run the debug-logs.sh script:

    $ ./debug-logs.sh
    Generating debug log for namespace prod
    Generating admin-ui-5b6dcf78fd-pq4fs logs
    Generating am-695fdf5f97-ch66f logs
    Generating amster-ss9j5 logs
    Generating ds-cts-0 logs
    Generating ds-idrepo-0 logs
    Generating end-user-ui-875cc95df-9nmf4 logs
    Generating idm—​ea2281802e-892q1 logs
    Generating ldif-importer-x25x6 logs
    Generating login-ui-74b8d676-vqm6c logs
    open file:///tmp/forgeops/log.html in your browser
  5. In a browser, go to the URL shown in the debug-logs.sh output. For example, file:///tmp/forgeops/log.html. The browser displays a screen with a link for each ForgeRock Identity Platform pod in your namespace:

    Screen shot of debug-logs.sh output.
  6. (Optional) To access the information for a pod, select its link from the start of the debug-logs.sh output.

    Selecting the link takes you to the pod’s description. Logs for each of the pod’s containers follow the pod’s description.

  7. (Optional) To modify the output to contain the latest updates to the pod descriptions and container logs, run the debug-logs.sh script again, and then refresh your browser.

After you’ve obtained the pod descriptions and container logs, here are some things to look for:

  • Examine each pod’s event log for failures.

  • If a Docker image could not be pulled, verify that the Docker image name and tag are correct. If you are using a private registry, verify that your image pull secret is correct.

  • Examine the init containers. Did each init container complete with a zero (success) exit code? If not, examine the logs from that failed init container using the kubectl logs pod-xxx -c init-container-name command.

  • Look at the pods' logs to see if the main container entered a crashloop.