Load testing
Load testing can help you get the most out of IDM and other ForgeRock products. The benefits load testing provides include:
-
Reducing the chance that unexpected spikes in system activity will cause the system to become unstable
-
Allowing developers and system administrators to reason more accurately and be more confident in release cycle timelines
-
Providing baseline statistics which can be used to identify and investigate unexpected behavior
Load testing is a complex subject that requires knowledge of your system and a disciplined approach. There is no "one-size-fits-all" solution that applies in all circumstances. However, there are some basic principles to keep in mind while planning, executing, and evaluating load tests.
Plan tests
The first step is to determine what metrics need to be examined, what components are going to be tested, what levels of load are going to be used, and what response ranges are acceptable. Answering these questions requires:
-
Service-level Agreements (SLAs)
-
Understanding of your use case
-
Baseline knowledge of your system
SLAs provide a stationary, business-based target to aim for in testing. An example SLA appears as follows:
| Service/Endpoint | Sustained load | Peak load | Required response time |
|---|---|---|---|
Customer auth against LDAP repo |
50,000 over 16 hours |
4,000 per second three times in a 16-hour period |
200ms |
Employee auth against AD repo |
4,000 over 10 hours |
100/second |
400ms |
Customer registration |
1,000 over 24 hours |
10/second |
500ms |
Employee password reset |
10 over 24 hours |
1/second |
500ms |
Sample SLA warnings and details:
|
|||
Details will vary depending on your use case and application flow, present usage patterns, full load profile, and environment. To get the most benefit, collect this information.
The system’s full load profile depends on how it is designed and used. For example, some systems have thousands of clients each using a small slice of bandwidth, while others have only a few high-bandwidth connections. Understanding these nuances helps determine an appropriate number of connections and threads of execution to use to generate a test load.
| If you have trouble determining which systems and components are being used at various points during your application flow, consider modeling your application using a sequence diagram. |
Understand resource usage
Understanding what resources are heavily consumed by ForgeRock products will help you with your test planning. The following chart details some products and their consumed resources:
| Product | Consumed resources |
|---|---|
AM with external stores |
CPU, memory |
DS as a user repository |
I/O, memory |
DS as a token store |
I/O, memory (if high token count) |
IDM |
I/O; CPU and memory play an important role in provisioning, sync, and user self-service |
PingGateway |
CPU |
All of the above depends on network performance, including name resolution and proper load balancing when required. |
|
Execute tests
When it comes to executing tests, these are the basic principles to keep in mind:
-
Every system is different; "it depends" is the cardinal rule.
-
Testing scenarios that don’t happen in reality gives you test results that don’t happen in reality.
-
System performance is constrained by the scarcest resource.
One way to ensure that your tests reflect real use patterns is to begin with a load generator that creates periods of consistent use and periods of random spikes in activity. During the consistent periods, gradually add load until you exceed your SLAs and baselines. By using that data and the data from the periods of spiking activity, you can determine how your system handles spikes in activity in many different scenarios.
| Your load generator should be located on separate hardware/instances from your production systems. It should have adequate resources to generate the expected load. |
When testing systems with many components, begin by testing the most basic things — I/O, CPU, and memory use. IDM provides insight into these by exposing JVM Metrics.
Once you have an understanding of the basic elements of your system, introduce new components into the tests. Keep a record of each test’s environment and the components which were under test. These components may include:
-
Hardware/Hypervisor/Container platform
-
Hosting OS/VM/Container environment
-
Hosted OS
-
Java Virtual Machine (JVM)
-
Web/J2EE Container (if used to host ForgeRock AM/IG or ForgeRock AM Agent)
-
Databases, repositories, and directory servers used with ForgeRock
-
Networking, load balancers, and firewalls between instances
-
SSL, termination points, and other communications
-
Points of integration, if any
-
Other applications and services that utilize ForgeRock components
-
Load generation configuration
-
Sample data, logs from test runs, and other generated files
| While there are many tools that can help you monitor your system, a thorough understanding of your system logs is the best path to understanding its behavior. |
|
To keep your results clear and focused, only add or adjust one variable at a time. Do not run tests designed to stress the system to its theoretical limit. The results you get from these stress tests rarely provide actionable insights. |
Change the JVM heap size
Changing the JVM heap size can improve performance and reduce the time it takes to run reconciliations.
You can set the JVM heap size via the OPENIDM_OPTS environment variable. If OPENIDM_OPTS is undefined, the JVM maximum heap size defaults to 2GB. For example, to set the minimum and maximum heap sizes to 4GB, enter the following before starting IDM:
cd /path/to/openidm/ export OPENIDM_OPTS="-Xms4096m -Xmx4096m" ./startup.sh Using OPENIDM_HOME: /path/to/openidm Using PROJECT_HOME: /path/to/openidm Using OPENIDM_OPTS: -Xms4096m -Xmx4096m ... OpenIDM ready
cd \path\to\openidm set OPENIDM_OPTS=-Xms4096m -Xmx4096m startup.bat "Using OPENIDM_HOME: \path\to\openidm" "Using PROJECT_HOME: \path\to\openidm" "Using OPENIDM_OPTS: -Xms4096m -Xmx4096m -Dfile.encoding=UTF-8" ... OpenIDM ready
You can also edit the OPENIDM_OPTS values in startup.sh or startup.bat.