IDM 7.3.1

Load testing

Load testing can help you get the most out of IDM and other ForgeRock products. The benefits load testing provides include:

  • Reducing the chance that unexpected spikes in system activity will cause the system to become unstable

  • Allowing developers and system administrators to reason more accurately and be more confident in release cycle timelines

  • Providing baseline statistics which can be used to identify and investigate unexpected behavior

Load testing is a complex subject that requires knowledge of your system and a disciplined approach. There is no "one-size-fits-all" solution that applies in all circumstances. However, there are some basic principles to keep in mind while planning, executing, and evaluating load tests.

Planning tests

The first step is to determine what metrics need to be examined, what components are going to be tested, what levels of load are going to be used, and what response ranges are acceptable. Answering these questions requires:

  • Service-level Agreements (SLAs)

  • Understanding of your use case

  • Baseline knowledge of your system

SLAs provide a stationary, business-based target to aim for in testing. An example SLA appears as follows:

Service/Endpoint Sustained load Peak load Required response time

Customer auth against LDAP repo

50,000 over 16 hours

4,000 per second three times in a 16-hour period

200ms

Employee auth against AD repo

4,000 over 10 hours

100/second

400ms

Customer registration

1,000 over 24 hours

10/second

500ms

Employee password reset

10 over 24 hours

1/second

500ms
Sample SLA warnings and details:

  • Response times are between load generator and ForgeRock platform and do not account for latency between client devices and architecture.

  • IDM must support four writes and 45 read transactions per second for 12 hours using DS as the repository.

  • IDM must support 2,000 changes from HR service.

  • Measuring response times occurs after establishing 10,000 active, concurrent stateful sessions with 10,000 unique identities.

Details will vary depending on your use case and application flow, present usage patterns, full load profile, and environment. To get the most benefit, collect this information.

The system’s full load profile depends on how it is designed and used. For example, some systems have thousands of clients each using a small slice of bandwidth, while others have only a few high-bandwidth connections. Understanding these nuances helps determine an appropriate number of connections and threads of execution to use to generate a test load.

If you have trouble determining which systems and components are being used at various points during your application flow, consider modeling your application using a sequence diagram.

Understanding resource use

Understanding what resources are heavily consumed by ForgeRock products will help you with your test planning. The following chart details some products and their consumed resources:

Product Consumed resources

AM with external stores

CPU, memory

DS as a user repository

I/O, memory

DS as a token store

I/O, memory (if high token count)

IDM

I/O; CPU and memory play an important role in provisioning, sync, and user self-service

IG

CPU

All of the above depends on network performance, including name resolution and proper load balancing when required.

Executing tests

When it comes to executing tests, these are the basic principles to keep in mind:

  1. Every system is different; "it depends" is the cardinal rule.

  2. Testing scenarios that don’t happen in reality gives you test results that don’t happen in reality.

  3. System performance is constrained by the scarcest resource.

One way to ensure that your tests reflect real use patterns is to begin with a load generator that creates periods of consistent use and periods of random spikes in activity. During the consistent periods, gradually add load until you exceed your SLAs and baselines. By using that data and the data from the periods of spiking activity, you can determine how your system handles spikes in activity in many different scenarios.

Your load generator should be located on separate hardware/instances from your production systems. It should have adequate resources to generate the expected load.

When testing systems with many components, begin by testing the most basic things — I/O, CPU, and memory use. IDM provides insight into these by exposing JVM Metrics.

Once you have an understanding of the basic elements of your system, introduce new components into the tests. Keep a record of each test’s environment and the components which were under test. These components may include:

  • Hardware/Hypervisor/Container platform

  • Hosting OS/VM/Container environment

  • Hosted OS

  • Java Virtual Machine (JVM)

  • Web/J2EE Container (if used to host ForgeRock AM/IG or ForgeRock AM Agent)

  • Databases, repositories, and directory servers used with ForgeRock

  • Networking, load balancers, and firewalls between instances

  • SSL, termination points, and other communications

  • Points of integration, if any

  • Other applications and services that utilize ForgeRock components

  • Load generation configuration

  • Sample data, logs from test runs, and other generated files

While there are many tools that can help you monitor your system, a thorough understanding of your system logs is the best path to understanding its behavior.

To keep your results clear and focused, only add or adjust one variable at a time.

Do not run tests designed to stress the system to its theoretical limit. The results you get from these stress tests rarely provide actionable insights.
Copyright © 2010-2024 ForgeRock, all rights reserved.