DS release notes

What’s new

DS 7.4.0

If you use data encryption with default settings, avoid deploying DS 7.4.0. Use DS 7.4.1 or later instead, as these releases include a fix for OPENDJ-10211.

If you deployed DS 7.4.0 with data encryption (confidentiality) using default settings, do not use the upgrade command to upgrade in-place.

Instead, refer to Upgrade from DS 7.4.0.

DS 7.3.0

If you deployed DS 7.3.0 with static groups, important upgrade actions may be required.

Contact ForgeRock Support for details.

DS 7.4

DS 7.4.1

DS 7.4.1 is the latest release targeted for DS 7.4 deployments and can be downloaded from the ForgeRock Backstage website.

The release can be deployed as an initial deployment or updated from an existing DS 7.4.x or earlier deployment.

Security

  • The DS Crypto Manager configuration now has an advanced property, key-wrapping-mode, to set the key wrapping mode for protecting symmetric keys.

    When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set key-wrapping-mode: WRAP.

Tools

  • The supportextract command now includes the system hostname in the name of the archive file.

DS 7.4.0

HDAP

This release introduces the HTTP Directory Access Protocol (HDAP) API for HTTP access to LDAP data.

Interface stability: Technology Preview

HDAP enables web-based HTTP applications and services to interact with LDAP directories. It uses HTTP as the transport protocol and JSON as the data format, making directory data easy to access and use in web applications. With HDAP, web-based systems and LDAP directories communicate and integrate simply and easily.

HDAP exposes the full hierarchical object-oriented data model of Directory Services with the following benefits:

  • Supports all features of the LDAP protocol and many extensions

  • Simplifies configuration; no complex data mappings

  • Manages LDAP schema as data

  • Lets applications validate resources using their JSON schema

  • Opens access to advanced administrative features such as collective attributes, password policy sub-entries, and access controls

  • Lets applications perform subtree searches and rename resources

For details, read Use HDAP.

Access logs

  • DS log messages now include elapsedQueueingTime, the time the request waited in the queue, and elapsedProcessingTime, the time actively processing the request. For details, refer to About logs.

    Use the following new access log filtering criteria for logs targeting outliers:

    • response-etime-queueing-greater-than

    • response-etime-queueing-less-than

    • response-etime-processing-greater-than

    • response-etime-processing-less-than

  • DS log messages with additionalItems fields now set their additional items to true instead of null.

    For example, the message for an unindexed search now includes "additionalItems":{"unindexed":true}.

  • DS can now record the attributes targeted by modification requests.

    For details, refer to Log modifications.

  • DS now logs security information about TLS handshake operations, including the negotiated protocol version and cipher, and the resulting security strength factor (SSF).

    A new tls setting for access log filtering criteria lets you filter on TLS handshakes.

    For details, refer to About logs.

  • DS access logs messages now include the user-friendly name and criticality for LDAP controls. For request controls, the messages also include the values.

    For details, refer to About logs.

Debug and error logs

  • Configuring debug-level logging is simpler with predefined logging categories.

    For details, refer to Debug-level logging.

Monitoring

  • DS monitoring metrics now include information to help troubleshoot changelog purging:

    • The LDAP attributes under cn=monitor are:
      ds-mon-changelog-file-count
      ds-mon-purge-waiting-for-change-number-indexing

    • The Prometheus metrics are:
      ds_replication_changelog_purge_waiting_for_change_number_indexing
      ds_replication_changelog_replica_dbs_changelog_file_count

Password policy

  • DS attribute value password validators with ds-pwp-attribute-value-check-substrings:true or check-substrings:true now check whether the password contains portions of attribute values and whether the attribute values contain portions of the password.

Collective attributes

  • DS servers now let you rename collective attributes.

    For details, refer to Rename an attribute.

Schema

  • New DS servers now check certificate lists, certificate pairs, and postal address syntax attributes for validity. The change affects attributes such as certificateRevocationList, crossCertificatePair, and postalAddress.

    The change doesn’t apply to DS servers you upgrade in place.

Replication

  • DS servers now log a warning message such as the following when replication status is Bad data and the problem seems to be the fractional replication configuration:

    Replication server RS([.var]##<server-id>##) ignoring update [.var]##<csn>## for domain "[.var]##<domain>##"
    from directory server DS([.var]##<server-id>##) at [.var]##<server-address>## because the peer DS
    reported to be in BAD_DATA status. The generation ID matches, (DS is [.var]##<generation-id>##,
    RS is [.var]##<generation-id>##), there may be a problem with fractional replication configuration.
    Check the DS error logs for more details

Performance

  • DS servers now more efficiently process OR filters composed of equality filters using the same attribute types.

    An example LDAP search filter of this type is (|(uid=uid1)(uid=uid2)…​(uid=uidN)), where N can be large.

Security

Native packages

Tools

  • The dsconfig command online help and Configuration reference now label deprecated and legacy configuration objects and properties.

  • Many commands with the --usePkcs11KeyStore option now also support the following options:

    • --providerArg to specify the provider configuration.

    • --providerClass or --providerName to specify the implementation.

  • All backendstat list-* subcommands now display indexes ordered by name.

DS 7.3

DS 7.3.4

DS 7.3.4 is the latest release targeted for DS 7.3 deployments and can be downloaded from the ForgeRock Backstage website.

The release can be deployed as an initial deployment or updated from an existing DS 7.3.x or earlier deployment.

Security

  • The DS Crypto Manager configuration now has an advanced property, key-wrapping-mode, to set the key wrapping mode for protecting symmetric keys.

    When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set key-wrapping-mode: WRAP.

Tools

  • The supportextract command now includes the system hostname in the name of the archive file.

DS 7.3.3

DS 7.3.3 is a maintenance release that does not include new features.

DS 7.3.2

DS 7.3.2 is a maintenance release that does not include new features.

DS 7.3.1

DS 7.3.1 is a maintenance release that does not include new features.

DS 7.3.0

Replication

  • DS servers now send data more efficiently when initializing a replica online, improving the speed of online initialization. This improvement requires that both servers run DS 7.3 or later.

  • DS replication now distinguishes when a replica requires reinitialization because it has fallen further behind the replication server than allowed by the replication-purge-delay. DS sets the ds-mon-status attribute for LDAP or ds_replication_replica_status{status} for Prometheus to Too late. Earlier versions of DS assigned Bad generation id status to such replicas.

    The dsrepl status output changed to take advantage of the new status. It now distinguishes the following states for a replicated data set:

    BAD - DATA MISMATCH

    Requires reinitialization; verify the replication configuration

    BAD - TOO LATE

    Requires reinitialization

    GOOD

    Normal operation; nothing to do

    SLOW

    Replication delay greater than five seconds

Groups

  • DS now uses significantly less memory for the group cache and for entry caches.

    Revisit Java heap size and database cache settings after upgrading. For details on setting heap and cache sizes, refer to Performance tuning.

  • DS now more effectively reads and updates entries with attributes having many values, such as LDAP and POSIX group entries.

    DS entry caches are no longer necessarily required for these entries. If you have enabled entry caches for large groups, consider removing them after upgrade.

  • DS monitoring metrics now include counts of static, dynamic, and virtual static groups, and statistics on the distribution of group sizes.

    For details, refer to Groups (Prometheus) and Groups (LDAP).

Indexing

When you upgrade DS directory servers in place, you must rebuild all indexes. The rebuilt indexes reflect required string normalization fixes.

If possible, trigger this rebuild during in place upgrade. Normal operations can result in degraded index errors until you rebuild the indexes.

  • DS servers now let you monitor index cost, which enables you to determine which indexes are causing write contention.

    For details, refer to Index cost.

  • DS servers now support a new matching rule, making it easier to monitor progress when migrating passwords to a new storage scheme.

    For an example, refer to Eliminate reversible password storage.

  • DS servers now display a message when you configure an unnecessary presence index for an attribute that already has an equality index DS can use for presence searches.

    DS servers also display the message at startup.

Monitoring

  • DS monitoring for Prometheus now includes a metric for replica status.

    For details, refer to Replication status (Prometheus).

  • DS monitoring metrics now include counts global and entry ACIs, and of the number of entries with ACIs.

    For details, refer to ACIs (Prometheus) and ACIs (LDAP).

  • DS monitoring metrics now include counts of the memory allocated to entry caches.

  • DS monitoring metrics now include a count of LDAP subentries.

    For details, refer to Subentries (Prometheus) and Subentries (LDAP).

  • DS monitoring metrics now include information to help troubleshoot change number indexing:

    • The Prometheus metrics are:
      ds_change_number_indexing_state
      ds_change_number_time_since_last_indexing_seconds

    • The LDAP attributes under cn=monitor are:
      ds-mon-indexing-state
      ds-mon-replicas-preventing-indexing
      ds-mon-time-since-last-indexing

    When the indexing state is not INDEXING, also read the replication logs for warning messages with additional details.

  • The metrics counting the number of indexed and unindexed searches have been renamed ds-mon-backend-filter-indexed and ds-mon-backend-filter-unindexed, and are now always maintained, even when index filter analysis is disabled.

Logging

LDAP

  • DS servers now compare userCertificate attribute values more efficiently.

Schema

  • DS servers now allow mail addresses to include UTF-8 characters, not just ASCII.

    The change does not affect directory data but does affect mail indexes which may become degraded. When upgrading, you may need to rebuild degraded indexes. For details, refer to When adding new servers and Update LDAP schema.

Tools

  • The modrate command now includes new options for adding and replacing values of multivalued attributes. This lets you use the command to simulate more realistic workloads. Previously, modrate only supported single-valued replace updates.

    The new options are:

    --strategy (modify|read_modify)

    Set this option to --strategy read_modify to use the new feature, reading an entry, then modifying it by deleting and adding attribute values.

    The default, --strategy modify, uses single-valued replace updates as before.

    --valueCount number

    Set this option to specify how many attribute values the multivalued attribute should contain following the modify operation. The number default is 1.

    --mvcc attribute

    Optionally set this option to specify the attribute to use for MVCC. Default: --mvcc eTag.

    For details, refer to the modrate reference page.

DS 7.2

DS 7.2.4

DS 7.2.4 is the latest release targeted for DS 7.2 deployments and can be downloaded from the ForgeRock Backstage website.

You can deploy this release as an initial deployment or use it to update an existing DS 7.2.x deployment.

Security

  • The DS Crypto Manager configuration now has an advanced property, key-wrapping-mode, to set the key wrapping mode for protecting symmetric keys.

    When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set key-wrapping-mode: WRAP.

DS 7.2.3

DS 7.2.3 is a maintenance release that doesn’t include new features.

DS 7.2.2

DS 7.2.2 is a maintenance release that doesn’t include new features.

DS 7.2.1

DS 7.2.1 is a maintenance release that does not include new features.

DS 7.2.0

Backup

  • DS servers now support Amazon AWS temporary credentials when backing up and restoring data using S3.

    You set the AWS session token using the s3.sessionToken.env.var storage property. For example, first set the session token as the value of the AWS_SESSION_TOKEN environment variable, then use --storageProperty s3.sessionToken.env.var:AWS_SESSION_TOKEN in the dsbackup commands.

    For additional examples, refer to Cloud storage.

  • DS servers now send an alert notification when backup task completes.

    The new alert types are org.opends.server.BackupSuccess and org.opends.server.BackupFailure, and are documented in Alert types.

Indexing

  • DS servers now support big indexes. A big index is a new kind of index optimized for attributes with few unique values. Big indexes let users more easily page through all the users in a US state, for example.

  • DS servers now let you monitor index use, so you can determine which indexes are unused.

    For details, refer to Unused indexes.

  • DS servers now support a DN pattern matching rule that lets you index an attribute with DN values, and search with wildcard characters, so you can find matches for specific RDNs in the DN, for example.

    For details, refer to DN patterns.

  • DS servers have improved output for debugging search indexes.

    For examples, refer to Debug search indexes. (As explained there, the format of debugsearchindex values is not a stable public interface, because it is intended for human beings, not scripts.)

  • The output for the backendstat list-indexes and backendstat show-index-status commands is easier to read and to understand.

  • DS servers now optimize searches for unresolved conflicts.

  • DS servers now more efficiently optimize searches for initial substrings.

Logging

  • DS servers now include entrySize in access log messages. You can filter access logs based on minimum entry size with the log filtering criteria setting, response-entry-size-greater-than.

    For details, refer to About logs.

  • By default, DS servers are configured to manage log file retention and rotation. For details on configuring this, refer to Rotate and retain logs.

    When an external program is also configured to manage DS log files, and moves or deletes log files in a way that a DS server does not expect, the DS now detects the change and logs an error message.

    Either let the DS server manage its log files, or configure an external program to do so, not both.

Monitoring

  • DS monitoring now takes replication listener threads into account when calculating whether a server is healthy. Monitoring shows a server to be in a healthy state if the server is alive, the replication server is accepting connections on the configured port, and any replication delays are below the configured threshold.

  • DS servers now support histogram metrics, as described in Metric types reference.

    As indicated in LDAP metrics reference and Prometheus metrics reference, DS servers expose the following histogram monitoring metrics:

    LDAP

    ds-mon-backend-entry-size-read
    ds-mon-backend-entry-size-written

    Prometheus

    ds_backend_entry_size_read_bucket{backend,type,le}
    ds_backend_entry_size_written_bucket{backend,type,le}

  • DS servers now let the monitor user read monitoring information over HTTP when some backends are offline, as long as backend with the monitor user entry remains online.

Password storage

  • DS servers now support hashing passwords with Argon2 for password storage.

    For details, refer to Password Storage.

Performance

  • DS servers now generate ETag attribute values more efficiently.

    This improves the performance of REST to LDAP applications that use ETags for MVCC. The plugin generates real ETag attributes for adds and updates. The server relies on the existing virtual attribute implementation only when a real ETag is not available.

    The implementation depends on a server plugin that is only configured for new servers. After upgrading all servers, configure the plugin on each server to use the new feature. For details, refer to Use the entity tag plugin for ETags.

  • DS servers now more efficiently verify passwords stored with PKCS5S2.

  • DS servers now run the rebuild-index command more efficiently when you identify specific indexes to rebuild.

    They also now run the rebuild-index --rebuildDegraded command more efficiently when there are no indexes to rebuild.

  • DS servers now start up more quickly when there are large numbers of groups.

    When the server starts, it runs an internal search to find all groups. DS servers now maintain a big index for objectClass that is specific to groups.

    In previous versions, the search for groups at startup could be unindexed. The workaround was to raise the index entry limit for the objectClass index, with the tradeoff of maintaining indexes for more object classes, and impacting write performance. The workaround is no longer necessary for new servers.

    Upgrading does not change the server configuration, however, so the index is not present after you upgrade. If you have applied the workaround of raising index-entry-limit for objectClass, and have upgraded your servers:

    1. Install a new, throwaway server with the evaluation profile, as described in Install DS for evaluation.

    2. Review the configuration for the big-equality index for objectClass.

      For example:

      dsconfig get-backend-index-prop --backend-name dsEvaluation --index-name objectClass --offline
    3. For your upgraded servers, consider adding a big-equality index for the groups, lowering index-entry-limit for objectClass, and rebuilding the objectClass indexes.

      Server startup time should be just as good, and write performance might improve.

Proxy

  • DS servers now support the Proxy Protocol from HAProxy.

    For details, refer to Proxy protocol.

  • The proxy backend settings to regularly contact remote LDAP servers now offer additional configuration for more fine-grained control when keeping connections alive and checking remote server availability.

    For details, refer to Proxy backend.

Replication

  • DS replication servers now check that the port is available when you change the configuration.

REST to LDAP

  • When you perform a paged results query whose corresponding LDAP search is indexed, the response now contains an estimated number of "totalPagedResults", and "totalPagedResultsPolicy" : "ESTIMATE".

    For an example, refer to Paged results.

  • When you perform a query, you can now request the resource count only, using the new _countOnly query string parameter. REST to LDAP returns the count, and not the resources.

    This parameter requires protocol version 2.2 or later. Use a header like Accept-API-Version: protocol=2.2,resource=1.0, for example.

    For details, refer to Query.

  • When converting JSON values, REST to LDAP now coerces:

    • Strings to booleans, integers, or JSON where possible.

    • Whole floating point numbers to integers.

    REST to LDAP also returns helpful errors when coercion fails. This improves interoperability with client applications that do not or cannot perform the conversions before adding or updating resources.

  • The REST to LDAP gateway settings now let you configure:

    • Availability checks for load balancing.

      The default heartbeat check settings have also been changed to check that pooled connections are alive every five minutes with a three-second keep-alive heartbeat timeout.

    • As many pools of failover servers as needed.

      You specify the pools using the "failoverLdapServers" field. The gateway still accepts "primaryLdapServers" and "secondaryLdapServers" settings for compatibility.

    • A connection timeout.

    For details regarding these new settings, refer to LDAP connection factories.

  • Internally, REST to LDAP now simplifies search filters when possible. This can improve search performance in some cases.

    REST to LDAP removes redundant objectClass assertions from search filters, retaining specific classes, but removing the superclasses they inherit from. For example:

    (&(objectClass=top)(objectClass=person)(objectClass=organizationalPerson)

    Becomes:

    (objectClass=organizationalPerson)
  • REST to LDAP now updates single-valued LDAP attributes by replacing the value, which reduces the network bandwidth and historical change data needed to replicate the update.

Schema

  • The schema definitions in the db/schema/04-rfc2307bis.ldif file now align with those of the latest RFC 2703bis Internet-Draft, An Approach for Using LDAP as a Network Information Service.

    The change does not affect directory data, but when upgrading you may need to rebuild degraded indexes. For details, refer to When adding new servers and Update LDAP schema.

Security

  • PKCS#11 hardware security module now explains how to use an HSM for all asymmetric keys, including the shared master key, for data that is not (yet) encrypted.

    If you plan to use an HSM for the shared master key, read the documentation carefully before you install DS. When you set up the server, you must avoid accidentally encrypting data while using the wrong shared master key.

    For details, refer to Store the shared master key.

Tools

  • A new DS bash-completion command generates a completion script for the Bash shell that makes it easier to write other DS commands.

    The completion script depends on support for bash-completion, which is not included by default on macOS.

    To set up Bash completion for DS commands, source the output of the script:

    • Bash 4

    • Bash 3.2 macOS

    source <(/path/to/opendj/bin/bash-completion)
    # First, install bash-completion support.
    # Next:
    eval "$( /path/to/opendj/bin/bash-completion )"

    You can make completion available in any new interactive shell by adding it to your ~/.bash_profile file, or ~/.bashrc file if it is loaded by the new shell.

  • The new dskeymgr show-deployment-id command displays key information about a given deployment ID—​formerly known as a deployment key—​such as the expiration date for the derived CA certificate.

    For details, refer to Show deployment ID information.

  • The dsrepl status --showReplicas command now displays an Entry count column.

    The entry counts in each row reflect the number of entries in the specified replica under the specified base DN.

  • The supportextract command now collects additional system information, including data to indicate whether the system is running in a virtual machine.

  • When collecting environment variable values, the supportextract command now excludes environment variables whose names contain PASS, PWD, and _PW.

Virtual attributes

DS 7.1

DS 7.1.7

DS 7.1.7 is the latest release targeted for DS 7.1 deployments and can be downloaded from the ForgeRock Backstage website.

The release can be deployed as an initial deployment or updated from an existing DS 7.1.x or earlier deployment.

DS 7.1.6

DS 7.1.6 is a maintenance release and does not include new features.

DS 7.1.5

DS 7.1.5 is a maintenance release and does not include new features.

DS 7.1.4

DS 7.1.4 is a maintenance release and does not include new features.

DS 7.1.3

REST to LDAP

  • REST to LDAP now updates single-valued LDAP attributes by replacing the value, which reduces the network bandwidth and historical change data needed to replicate the update.

DS 7.1.2

Performance

  • DS servers now run the rebuild-index --rebuildDegraded command more efficiently when there are no indexes to rebuild.

Tools

  • The supportextract command now collects additional system information, including data to indicate whether the system is running in a virtual machine.

  • When collecting environment variable values, the supportextract command now excludes environment variables whose names contain PASS, PWD, and _PW.

DS 7.1.1

DS 7.1.1 is a maintenance release that does not include new features.

Java

DS 7.1.1 introduces support for Java 17 (17.0.3 or later) in addition to Java 11:

  • In Java 17, the PCKS#12 keystore encryption/Mac algorithm has been upgraded to HmacPBESHA256. Update to at least Java 11.0.12 if you have an application that runs Java 11 and must read the keystore.

  • Use G1 GC (the default) instead of parallel GC. The setting is shown in Java Settings. Use of ZGC or Shenandoah is not recommended for production deployments at this stage.

For details, refer to Java.

If you are upgrading, refer to Supported Java.

DS 7.1.0

Backup

  • The dsbackup command now lets you set a non-default storage provider endpoint.

    For details, refer to Cloud Storage.

Indexing

  • The online rebuild index process is now less intrusive, more effective, and more robust. When you run a rebuild-index command while the server is online, the backend database remains available for directory operations during the rebuild.

    Individual indexes do appear as degraded and unavailable while the server rebuilds them. A search request that relies on an index in this state may temporarily fail as an unindexed search.

Logging

  • DS log messages now include the authorization ID for every request.

  • DS servers now support logging the internal delete operations triggered by entry expiration.

    If you have set the ttl-age and ttl-enabled properties for a backend, use this feature by configuring an access log publisher to record messages about internal operations. When the server deletes an expired entry, it logs a message with "additionalItems":{"ttl": null} in the response.

    For background, refer to Entry Expiration.

Passwords

  • Password quality checks using the password quality advice control now ensure that proposed passwords are not in the password history.

    If the server finds the proposed password in the password history, this appears in the failing criteria returned with the advice response control.

    In addition, the REST to LDAP response over HTTP now includes the password attribute type.

Replication

  • DS servers now let you restrict which replicas you trust to send updates.

    By default, all directory servers in a replication topology trust all replicas. If a replica allows an update, then other servers relay and replay the update without further verification. This simplifies deployments where you control all the replicas.

    In deployments where you do not control all the replicas, you can configure replication servers to accept updates only from trusted replicas. The trust depends on the certificate that a replica presents to the replication server when connecting.

    For details, refer to Trusted Replicas.

  • DS servers now let you define replication group failover. This determines how a directory server selects the next group with replication servers to connect to when no replication server is available in the directory server’s own group.

    To activate replication group failover, set the global configuration property, group-id-failover-order.

    For details on how a directory server chooses a replication server, refer to Replication Connection Selection.

  • When a replica’s last change is older than the oldest change recorded in the replication server’s changelog, the replication server now records the problem in its log, and sends a message to the replica. When it receives the message, a 7.1 replica remains connected to the replication server, but refuses update operations, effectively becoming read-only. A pre-7.1 replica closes the connection.

    In any case, the replica no longer applies replication updates. Its data diverges more and more from other replicas' data.

    Should this happen in your deployment, reinitialize the replica. For details, refer to Manual Initialization.

  • DS servers now log more explicit messages when they discover duplicate server IDs.

REST to LDAP

  • This release introduces support for querying fields of reference or reverseReference resources that are subtypes of the resources you are searching.

    As an example, suppose that devices and users are both subtypes of a "managed object" type. Also, suppose that devices have a deviceType field, that users have a surname field, and that a basic managed object has neither of these fields. Now, your queries on a collection of managed objects can match properties of the referenced subtypes, such as /managedObjects?_queryFilter=deviceType+eq+phone, or /managedObjects?_queryFilter=surname+eq+Jensen.

Samples

  • The sample for building custom DS Docker images now has a USE_DEMO_KEYSTORE_AND_PASSWORDS setting that simplifies getting started with a basic Docker image on your computer.

    For details, refer to opendj/samples/docker/README.md.

Security

  • DS directory and proxy servers now allow access to the root DSE operational attribute subSchemaSubEntry. This attribute indicates the entry holding the LDAP schema definitions.

    Many applications retrieve this attribute, and the associated schema, to properly display or validate attribute values. If you cannot upgrade yet, update the configuration of your DS server to grant all users read access to subSchemaSubEntry at least on the root DSE:

    • For DS 7 directory servers, add subSchemaSubEntry to the attribute list in the "User-Visible Root DSE Operational Attributes" global ACI.

    • For DS 7 directory proxy servers, add allowed-attribute:subSchemaSubEntry on the Root DSE access configuration object.

    For details on granting access to subSchemaSubEntry on entries in directory data, refer to ACI: Access SubSchemaSubEntry Attribute.

  • DS servers now support text-based Privacy-Enhanced Mail (PEM) keys and certificates for server key pairs, master keys, and trusted certificates.

    For details, refer to Use PEM-Format Keys.

  • The DS fingerprint-certificate-mapper now also supports fingerprints without colons.

    For example, the following SHA-256 fingerprints are equivalent:

    • 0555BDA5E14C35A6A54E78DD3EFDEA5A665DE0DC9CC5187EE9CAA91ECD874B78

    • 05:55:BD:A5:E1:4C:35:A6:A5:4E:78:DD:3E:FD:EA:5A:66:5D:E0:DC:9C:C5:18:7E:E9:CA:A9:1E:CD:87:4B:78

Tools

  • DS command options that have secrets as arguments now support :env and :file modifier suffixes.

    For example, if the bind password is stored in a ~/.pass file, use --bindPassword:file ~/.pass. If the password is stored in the environment variable PASS, use --bindPassword:env PASS.

    Use the modifiers with the following options to provide the secret in an environment variable (:env), or in a file (:file):

    • --bindPassword[:env|:file]

    • --deploymentKeyPassword[:env|:file]

    • --keyStorePassword[:env|:file]

    • --monitorUserPassword[:env|:file]

    • --rootUserPassword[:env|:file]

    • --set[:env|:file] (for setup profile parameters)

    • --trustStorePassword[:env|:file]

  • The supportextract command now uses the jcmd command, if available, for heap dumps. Otherwise, it uses the jmap command.

  • The addrate, authrate, modrate, and searchrate commands now include connection time as part of the response time for a request.

DS 7.0

DS 7.0.2

There are no new features in DS 7.0.2, only bug fixes.

DS 7.0.2 is the latest release targeted for DS 7.0.x deployments and can be downloaded from the ForgeRock Backstage website.

The release can be deployed as an initial deployment or updated from an existing DS 7.0.x deployment.

DS 7.0.1

  • The DS password synchronization plugin for IDM now supports OAuth 2.0 access token bearer authentication.

    For details, refer to Synchronize Passwords With ForgeRock Directory Services (DS) in the IDM documentation.

  • DS command options that have secrets as arguments now support :env and :file modifier suffixes. Use these with the following options to provide the secret in an environment variable (:env), or in a file (:file):

    --bindPassword[:env|:file]
    --deploymentKeyPassword[:env|:file]
    --keyStorePassword[:env|:file]
    --monitorUserPassword[:env|:file]
    --rootUserPassword[:env|:file]
    --set[:env|:file] (for setup profile parameters)
    --trustStorePassword[:env|:file]

    For example, if the bind password is stored in a ~/.pass file, use --bindPassword:file ~/.pass. If the password is stored in the environment variable PASS, use --bindPassword:env PASS.

  • The supportextract command now uses the jcmd command, if available, for heap dumps. Otherwise, it uses the jmap command. (Issue: OPENDJ-7662)

DS 7.0.0

Access control

Aliases for controls and extended operations

DS 7.0.0 supports use of aliases in addition to OIDs for LDAP controls and extended operations in ACIs, making those ACIs significantly more human-readable. For details, refer to Directory server ACIs.

Since previous releases support only OIDs, only use aliases in ACIs after upgrading all directory servers. Otherwise, older servers will log warning messages for the unrecognized aliases, such as the following:

Access Control Instruction (ACI) targetcontrol expression value "value" is invalid.
 A valid targetcontrol keyword expression value requires one or more valid control OID strings in the following format:
 oid [|| oid1] ... [|| oidN]

Authentication

Multiple identity mappers

The following configuration objects can now reference multiple identity mappers:

When resolving the identity, the server uses the first identity mapper that finds a match. If multiple identity mappers match different entries, however, then the server returns LDAP error code 19, Constraint Violation.

For background information, refer to Identity mappers.

Backup and restore

New, simplified implementation with cloud storage support

The release provides a new, simplified implementation for DS backup and restore operations:

  • The new implementation replaces backup archives with collections of backup files.

    The collection includes backend files and backup metadata. The files always follow the same layout, regardless of what you back up.

    You manage backup files by retaining an entire backup directory. You are no longer required to use a separate backup directory for each backend.

  • You can now stream backup files directly to cloud storage, and restore directly from cloud storage.

  • You no longer have to make a choice between full and incremental backup operations. Backup operations are incremental by design. When you reuse the same backup directory, the process only backs up new data.

  • The new implementation includes a purge subcommand for removing old backup files. You can purge old files either as an external command, or as a server task.

  • In the event of a disaster, you can restore from a backup directory stored off-site using only the deployment key and password, and a backup copy of the server configurations.

    The new implementation protects (encrypts) the backup encryption keys with the shared master key. It stores the encrypted encryption keys in the backup files.

    You no longer need to configure replication between new replicas and a server from the existing topology. Instead, you first set up replacement replicas with the deployment key and password, restoring the backed up server configurations to match those servers lost in the disaster. You then restore the data using the off-site backup directory.

  • The new implementation always signs and verifies the integrity of backups, and always encrypts backup files.

    The new implementation encrypts the keys used for signing and encryption with the shared master key. It stores the encrypted keys in the backup files.

    You can verify the integrity and ability to decrypt backups before restoring a backend.

  • The new implementation makes it possible to list and verify backups while the server is online.

  • The new implementation improves restore performance compared to restore of incremental backups in previous versions.

    The previous implementation restored files from the full backup archive, and then restored files from each incremental backup archive. Files could be restored and then removed, or restored multiple times.

    The new implementation only restores one version of each file in the backup directory.

  • A new command, dsbackup, replaces the backup and restore commands.

    The dsbackup command performs operations formerly performed using separate commands:

    • dsbackup create performs backup operations.

    • dsbackup list displays a summary of available backups, and lets you verify them.

    • dsbackup purge removes old backup files.

    • dsbackup restore performs restore operations.

  • The new dsbackup restore command has a --backendName option, which lets you restore only the specified backend.

For examples, refer to Backup and restore.

Cloud deployments

Ready for Docker and Kubernetes

DS 7.0.0 lifts restrictions on running DS servers in Docker and Kubernetes deployments. Many individual improvements make this possible, including the following:

  • Replication improvements let you scale the number of DS replicas in your stateful sets up and down.

  • The new dsrepl command runs well in Docker containers.

ForgeRock supports customers deploying DS in Docker containers and Kubernetes platforms.

To get started, try the following:

  • Use the forgeops repository and the unsupported, evaluation-only base images for the ForgeRock Identity Platform. The images are available in ForgeRock’s public Docker registry.

  • Build your own sample DS Docker image.

    Unpack the .zip distribution, then refer to the opendj/samples/docker/README.md file for instructions.

Collective attributes

Relative parent support

DS servers now support specifying the relative parent in collective attribute subentries.

For details, refer to Inherit from a parent entry.

Data storage

Shared database cache by default

By default, DS servers now share cache memory among JE database backends. The server keeps JE database internal and leaf nodes in the database heap cache.

For existing servers, the upgrade command does not change the database cache behavior. Consider setting the global property je-backend-shared-cache-enabled:true, and the JE backends' properties db-cache-mode:cache-ln after upgrade.

For details, read the following documentation:

Newer JE

DS 7.0.0 upgrades JE backend databases to Berkeley DB Java Edition 18.3.12.

Different DS server versions continue to replicate data during the upgrade process. However, the JE upgrade has the following implications for the portability of local DS data. Once you upgrade the data in a JE backend database:

  • You cannot downgrade a directory server without also restoring JE backend data from a pre-upgrade server.

  • You cannot restore backups of an upgraded JE backend on a pre-upgrade directory server.

In addition, several JE backend properties that affect cache sizing and database maintenance can now be changed at runtime without restarting the backend. For details, refer to JE backend.

Data encryption

Portable encrypted data

DS servers now store symmetric keys, encrypted with the shared master key, with the data they encrypt.

It is no longer necessary for disaster recovery to maintain a file system backup of a server from each replication topology in your deployment. It is now sufficient to keep the backup directory and a means to recover the shared master key. As long as a server has the same shared master key as the server that encrypted the data, it can recover symmetric keys needed to decrypt data.

Be aware that this feature is new, and not provided in previous versions of DS software. Replication is fully compatible with previous server versions, but backup files are not. For this feature to work, you must use a backup from an upgraded or new server.

GCM with AES

DS directory servers now support Galois/Counter Mode (GCM) with AES for encrypted data confidentiality. GCM is efficient and improves integrity protection for encrypted backend data.

Set the data encryption cipher transformation, as described in Data encryption. The default setting for the backend property, cipher-transformation, is now AES/GCM/NoPadding.

Email notifications

Secure, authenticated connections

Email notifications now support SMTP authentication and use of TLS.

For details, refer to Send account status mail, and Mail server.

Interoperability

Microsoft AD range retrieval support

DS software now supports the * character in malformed attribute options for interoperability with the Microsoft Active Directory "range retrieval" mechanism.

Logging

Field whitelisting

ForgeRock Common Audit loggers now whitelist all fields that are safe to log by default. The whitelist is processed before the blacklist, so blacklist settings overwrite the whitelist defaults.

For details, refer to Allow log message fields.

Error messages to standard output

DS servers can now send error messages to standard output.

For details, refer to Log errors to standard output.

More information about operations in access logs

DS servers now record additional information about LDAP operations in access log messages:

  • For LDAP bind operations, the security strength factor (SSF) negotiated for secure client connections appears in the response field of the access log message:

    {..."request":{"protocol":"LDAPS","operation":"BIND"...}..."response":{..."additionalItems":"ssf=128"},...}
  • For persistent searches, the log messages include "additionalItems":"persistent".

Details when a connection handler fails to start

When a connection handler fails to start, DS servers now log an error message indicating the cause.

Monitoring

Monitor account replicated by default

DS servers now replicate the monitor user created at setup time (default DN: uid=monitor).

This lets commands like dsrepl status use the same account credentials to retrieve monitoring information from all servers. You can use the account in the same way for multi-server monitoring operations.

Networking

Advertised listen address

DS servers now have a new, required, global property, advertised-listen-address. This setting specifies the hostname or IP address that clients should use for connecting to the server. The advertised-listen-address can be multi-valued in systems with multiple network interfaces. DS servers also now have a global property, listen-address.

The listen-address property can be set to the wildcard IP address, 0.0.0.0, but the advertised-listen-address property cannot. By default, replication and connection handlers inherit their settings for listen addresses from these global properties.

This improvement lets DS servers make fewer DNS requests than before.

When setting up a new server, the setup command sets the advertised-listen-address property to the IP address or the FQDN provided as the --hostname argument.

During upgrade, the value for the advertised-listen-address property is assigned using the hostname derived from administrative data under cn=admin data. If any listen-address properties are set to the same value, then those settings are removed during upgrade, and the values are inherited instead.

Passwords

SCRAM SASL support

DS software now supports Salted Challenge Response Authentication Mechanism (SCRAM) SASL binds.

A SASL SCRAM mechanism provides a secure alternative to transmitting plaintext passwords during binds. It is an appropriate replacement for DIGEST-MD5 and CRAM-MD5.

With a SCRAM SASL bind, the client must demonstrate proof that it has the original plaintext password. During the SASL bind, the client must perform computationally intensive processing to prove that it has the plaintext password. This computation is like what the server performs for PBKDF2, but the password is not communicated during the bind.

Once the server has stored the password, the client pays the computational cost to perform the bind. The server only pays a high computational cost when the password is updated, for example, when an entry with a password is added or during a password modify operation. A SASL SCRAM mechanism therefore offers a way to offload the high computational cost of secure password storage to client applications during authentication.

Passwords storage using a SCRAM storage scheme is compatible with simple binds and SASL PLAIN binds. When a password is stored using a SCRAM storage scheme, the server pays the computational cost to perform the bind during a simple bind or SASL PLAIN bind.

The SCRAM password storage scheme must match the SASL SCRAM mechanism used for authentication. In other words, SASL SCRAM-SHA-256 requires a SCRAM-SHA-256 password storage scheme. SASL SCRAM-SHA-512 requires a SCRAM-SHA-512 password storage scheme.

DS software offers the following in the configuration for new servers:

Password Storage Scheme SASL Mechanism

SCRAM-SHA-256

SCRAM-SHA-256

SCRAM-SHA-512

SCRAM-SHA-512

For additional information, refer to Password storage for the server, and LDAP connection factories for the REST to LDAP gateway.

DS servers now support LDAP subentry password policies that match all features available in per-server password policies.

Servers store subentry policies in the directory data, and therefore replicate them. This improvement significantly simplifies password policy management across multiple replicas.

For details, refer to DS subentry password policies. Many samples in the documentation now demonstrate features of the improved subentry password policies.

Stronger password storage schemes

DS servers now support additional password storage schemes, PBKDF2-HMAC-SHA256 and PBKDF2-HMAC-SHA512.

The new password storage schemes use SHA-256 and SHA-512 hash-based message authentication code settings. The PBKDF2 password storage scheme uses SHA-1.

To migrate passwords to a new storage scheme, refer to Deprecate a password storage scheme.

128-bit salt

Salted hashed password storage schemes now use 128-bit salt when generating a hash.

This change applies to the following password storage schemes:

PBKDF2
PBKDF2-HMAC-SHA256
PBKDF2-HMAC-SHA512
Salted MD5
Salted SHA-1
Salted SHA-256
Salted SHA-384
Salted SHA-512

Rehash passwords

You can now configure BCrypt and PBKDF2-based password storage schemes to recalculate password hashes after the iterations settings are changed. DS servers recalculate and store an account’s password hash when the user binds successfully with their password.

For details regarding BCrypt, refer to the reference for the property rehash-policy. For details regarding PBKDF2-based schemes, refer to the reference for the property rehash-policy.

Password quality advice

DS servers support a new control to request password quality advice when changing a password. Should the request fail due to low password quality, the response control indicates which password validator settings led to the failure.

The ldappasswordmodify and ldapmodify commands support the new control. Use them to test and debug password policy validation settings.

The new LDAP control has interface stability Evolving. It may be removed in a future release, or replaced with a more general mechanism.

For details, refer to Check password quality (LDAP), and Check password quality (REST).

Performance

Faster export

The export-ldif command can now complete an export up to twice as fast as before. This improvement is particularly useful with large data sets including tens or hundreds of millions of entries.

Faster REST to LDAP performance

DS servers now perform better for REST to LDAP searches, and operations that rely on ETags for MVCC.

Proxy

Mutual TLS with LDAP servers

The setup command now lets a proxy backend bind to remote servers with mutual TLS. The setup profile for a proxy server configures the server to use mutual TLS to authenticate when binding to backend servers. As a result, you must provision the key manager for the proxy with the proxy service account keys, and include the certificate in the proxy user account when using the DS proxy server setup profile.

For details, refer to Install a directory proxy.

DS proxied server

When setting up new DS replicas, use the ds-proxied-server setup profile to prepare the replicas for use with new DS proxy servers.

For details, refer to Install DS for use with DS proxy.

REST

Path references

REST to LDAP mappings now support references by resource paths, simplifying access to all resource fields. REST clients can use this to issue graph-like queries. For example, the following path and query filter returns the groups that Babs Jensen’s manager belongs to:

/users/bjensen?_fields=/manager/group

For an example, refer to Graph-like queries.

To demonstrate this feature, the sample REST to LDAP mapping now uses resource paths. The configuration is simpler than the configuration with base DN references.

For example, this excerpt shows a manager reference from the version that uses a base DN:

{
  "manager": {
    "type": "reference",
    "ldapAttribute": "manager",
    "baseDn": "..",
    "primaryKey": "uid",
    "mapper": {
      "type": "object",
      "properties": {
        "_id": {
          "type": "simple",
          "ldapAttribute": "uid",
          "isRequired": true
        },
        "displayName": {
          "type": "simple",
          "ldapAttribute": "cn",
          "writability": "readOnlyDiscardWrites"
        }
      }
    }
  }
}

The same manager reference using a resource path now looks like this:

{
  "manager": {
    "type": "reference",
    "resourcePath": ".."
  }
}

The latter definition ensures access to all fields defined for the referenced resource.

Reverse references

REST to LDAP mappings now support reverse references.

Reverse references are similar to the isMemberOf LDAP attribute used for groups. For example, use a reference mapping to list a user’s devices, or to list a manager’s reports:

{
  "reports": {
    "type": "reverseReference",
    "resourcePath": "..",
    "propertyName": "manager"
  }
}

For an example in context, refer to Reverse references.

Password quality advice

REST to LDAP now supports passwordQualityAdvice and dryRun query string parameters.

The passwordQualityAdvice parameter relies on the DS LDAP password quality advice control, OID 1.3.6.1.4.1.36733.2.1.5.5, which users must have access to request. The dryRun parameter relies on the LDAP no-op control, OID 1.3.6.1.4.1.4203.1.10.2.

The password quality advice control and the passwordQualityAdvice parameter have interface stability: Evolving. They may be removed in a future release, or replaced with a more general mechanism.

For details, refer to Check password quality.

Account usability support

REST to LDAP now includes an accountUsability action.

For details, refer to Account usability action.

SASL EXTERNAL and SASL SCRAM support

The REST to LDAP gateway now supports SASL EXTERNAL and SASL SCRAM binds.

For details, refer to LDAP connection factories.

Per-Server password policies over REST

DS servers now let you create per-server (configuration-based) password policies over REST.

For an example, refer to Per-server password policies.

Replication

Replication at setup time

The setup command now lets you configure replication at setup time.

You no longer need to get all peer servers running before configuring replication. The server begins replicating with peer servers when it comes online, and when it can contact the peers. For this reason, the setup command no longer starts the server by default. To ensure replication proceeds smoothly from the beginning, finish configuring the server before starting it for the first time.

These new setup command options enable replication:

  • When you set the -r, --replicationPort option, the server runs a replication service and maintains a changelog.

    If you add local application data at setup time, the server replicates the data with other replicas. There is no need to configure and initialize replication separately.

  • When you set the --bootstrapReplicationServer option, the server contacts the specified replication server(s) to discover peer replicas and replication servers. This option is required when replicating between multiple servers.

    Use this option multiple times to specify redundant bootstrap servers for availability. Specify the same list of bootstrap servers each time you set up a replica.

    Your first bootstrap server(s) must have replication ports, because the first bootstrap server(s) must play the replication server role.

For examples, refer to Installation.

New command to manage replication

After configuring servers to replicate as part of the setup process, use the new dsrepl command to manage replication.

For details, refer to Replication.

String-based server IDs

DS 7.0.0 lets you set server IDs to alphanumeric strings, such as ds1-us-west.

When you set a server ID, take care to choose a relatively short string.

The server ID appears in historical data values that include a change sequence number. For example, it shows up in monitoring metrics, and in the values of ds-sync-state and ds-sync-hist attributes in application data on DS replicas. As a result, historical data is potentially easier to interpret, but larger than in previous versions where server IDs were numbers.

One server ID per server

Servers are now identified by a single, global server ID. For details, refer to server-id.

For new servers, use the setup command to specify the server ID, or accept the generated default string.

For existing servers, the upgrade command derives the ID in the following way:

The command the existing global server ID, if available.
Otherwise, the command uses the first server ID found in cn=admin data.

Other server ID values are no longer used.

If replication has not yet been configured, the command generates a new ID for the server.
One group ID per server

Servers now have a single, global group ID. For details, refer to group-id.

For existing servers with group IDs, the upgrade command determines which ID is used most, and uses that ID as the single, global ID.

Replication delay metrics

DS 7.0.0 introduces replication receive delay and replay delay monitoring metrics. These metrics provide the best means yet to help you estimate whether the data in your directory server replicas is converging toward a consistent state.

Replay performance

DS 7.0.0 improves replication replay performance, reduces disk space used by the replication changelog database, and reduces replication delay in deployments under extreme load.

Replication of offline LDIF changes

Servers now replicate changes made offline to an LDIF backend. The server replicates the offline changes once it starts again.

Automatic purge of stale replicas

DS 7.0.0 purges out-of-date replicas from the changelog. The replica is purged when it has been out of contact for longer than the replication purge delay.

This enables DS servers to eventually discard information about replicas that you have removed from service, for example.

You can also use the dsrepl purge-meta-data to eliminate stale historical data. For details, refer to Manual purge.

Exclude domains from changelog indexes

DS 7.0.0 introduces a new replication server property to exclude domains from the changelog indexes, changelog-enabled-excluded-domains. Use this to prevent applications that read the external change log from having to process update notifications for entries that are not relevant to them.

This property eliminates the need for a separate external changelog domain configuration.

For an example, refer to Exclude a domain.

CTS excluded from changelog indexing

The am-cts setup profile now excludes the CTS base DN from change number indexing.

There is no need to update the changelog configuration manually after installing a new DS replica for as a CTS store.

More details about unresolved conflicts

DS servers now log additional information about naming conflicts, which helps you identify the server that generated the conflicting operation.

Samples

Sample for building custom Docker images

The DS server distribution now includes a sample Dockerfile and related files for building custom DS Docker images.

Updated sample for Grafana and Prometheus

The DS server distribution now includes an updated sample monitoring dashboard for use with Grafana and Prometheus.

Schema

Require TRUE or FALSE boolean values

DS servers now support an option to require strict compliance for boolean attribute values.

By default, DS servers accept a range of values for boolean attributes. For details, refer to strict-format-boolean.

Security

Secure by default

Default settings for new DS servers are more secure than before.

The explicit --productionMode option has been removed, as server configurations and profiles are now secure by default. New server installations require:

Secure connections

All operations except bind requests and StartTLS requests, and base object searches on the root DSE, require secure connections.

This behavior is governed by the global configuration property, unauthenticated-requests-policy, which is now set to allow-discovery, instead of allow, unless the last setup profile applied is the ds-evaluation profile.

For details on securing connections, refer to Secure connections.

Authentication

By default, servers deny anonymous access to most LDAP operations, controls, and extended operations.

For details on access control, refer to Access control.

Additional access policies

By default, servers deny access to directory data. You must configure access policies to grant access to directory data. For details on granting access, refer to Access control.

Only the evaluation setup profile is more lenient. It grants global permission to perform operations over insecure connections, and open access to sample Example.com data. For details, refer to Install DS for evaluation.

Stronger passwords

Passwords must have at least 8 characters. Common passwords are rejected.

For details on changing password policy, refer to Passwords.

Permission to read log files

Log files are now read/write only by the DS server user.

For details on log file permissions, refer to File permissions.

As the upgrade process preserves the existing configuration, upgraded servers are not affected.

Simple private PKI

The setup and dskeymgr commands simplify creation and management of a public key infrastructure (PKI).

DS 7.0.0 introduces the concept of a deployment ID and deployment ID password. The deployment ID and password serve as an alternative to a private CA, simplifying evaluation, development, and testing, and managing directory services. They also serve to derive a shared master key to protect secret keys. The deployment ID and password are required as part of the setup process. For details, refer to Cryptographic keys.

When you use an existing CA, you can continue to use key pairs with CA-signed certificates.

For public-facing directory services, you can continue to configure connection handlers with additional key and trust manager providers using certificates signed by a well-known CA. For details, refer to Key management.

To manage deployment keys, key pairs, CA certificates, and master keys after setting up a server, use the dskeymgr command.

Many examples in the documentation now demonstrate use of deployment IDs and passwords.

Keystores reload without restart

DS servers now reload file-based keystores and truststores when their contents change.

This lets you rotate certificates and keys without restarting the key manager or trust manager components.

Simple key rotation

DS 7.0.0 greatly simplifies rotating the key pairs used to secure replication connections. By default, replication now uses the same keys as the other connection handlers.

For details on changing key pairs, refer to Key management.

Alternative PKCS#11 types

PKCS#11 key managers and trust managers now let you set the keystore or truststore type. The default type is PKCS11.

If your JVM supports other types, set the keystore or truststore type with one of the following properties:

Multiple trust managers

The following configuration objects can now reference multiple objects:

Use this feature to allow trust for both well-known CAs whose certificates are stored in the JVM truststore, and internal or deployment-specific CAs whose certificates are stored in a separate truststore.

Multiple certificate mappers

An external SASL mechanism handler can now reference multiple certificate-mapper configurations.

The server uses the first certificate mapper that finds a successful match.

Indexes for certificate attributes

When you create a user data backend using the ds-user-data setup profile, the setup process now configures equality indexes for the ds-certificate-fingerprint and ds-certificate-subject-dn attributes.

Certificate mappers use these indexes during certificate-based authentication.

Richer access log messages

DS servers now record additional items in access log messages when multiple password policy subentries apply to a user. The messages are logged only for bind, add, and modify operations. The messages show the DN of the user having more than one applicable policy, and the DN of the policy the server actually used for the operation.

The server logs a message such as the following for a bind request with two conflicting policies:

"additionalItems":{"pwdpolicywarning":"Found 2 conflicting password policy subentries for user <user-dn>,
used <policy-dn>","ssf":"0"}

As described in Assign password policies, you must not assign more than one password policy to the same account.

Tools

New setup-profile command

A new command, setup-profile, enables configuration of setup profiles following initial installation. Use the setup-profile command when the server is offline.

This command is intended for use in DevOps deployments where you apply additional configuration to a base image that is the same for all deployments.

If you have changes that apply to each server you set up, you can create and maintain your own setup profile. For details, refer to Setup profiles.

Configurable domains and base DNs

All setup command profiles, except the ds-evaluation profile, now allow you to set the domain or the base DN.

For details, refer to Setup profiles.

Generate systemd service files

The create-rc-script command now produces a systemd service file when you use the --systemdService option.

Generate user entries for evaluation

The ds-evaluation setup profile now lets you generate an arbitrarily large number of similar user entries. By default, the profile adds 100,000 generated users in addition to users previously included, such as Babs Jensen and Kirsten Vaughan.

Each user entry has a uid RDN like user.number. Each user entry’s password is password.

The capability replaces the setup command option -d, --sampleData.

Proxied authorization for rate tools

The addrate, modrate, and searchrate commands now support proxied authorization with the -Y, --proxyAs {authzID} option.

Support for formatted integers

Formatted integers can now be supplied to some integer arguments, making commands easier to read.

When setting the number of generated sample entries as an argument to the setup command, and when setting integer arguments for the addrate, authrate, modrate, and searchrate commands, you can now use formatted integers.

For example, the following are equivalent to 10000000:

10,000,000
10.000.000
"10'000'000"
10_000_000
"10 000 000"

Templates for the makeldif command can also accept formatted integers for numbers declared in a subordinate template.

Task management

The manage-tasks command now has --status and --type options.

When used with the --summary option, these options filter the list to include only tasks of the specified type and status. The option arguments are case-insensitive, and must be provided in the JVM locale.

For example, to list only unscheduled tasks on a JVM with the French locale, use --status "non planifié" instead of --status unscheduled.

Set task IDs and descriptions

When you schedule a task, you can now set its identifier with the --taskId option, and its description with the --description option. The identifiers and descriptions appear in output and messages that describe the task.

These new options are especially useful for recurring tasks. Use the task identifier when managing the task in subsequent commands, for example.

No changes when reading JE backends offline

The following tools now never write to JE backend databases when reading JE information:

  • backendstat

  • export-ldif

  • verify-index

Status output improvements

The status command now displays the same types of information independently of the server configuration, and regardless of whether the command runs in online or offline mode.

The command still displays more detailed information in online mode than in offline mode.

Supportextract improvements
  • The supportextract command now also collects:

    • The directory superuser and monitor user account files.

    • The archived configuration files.

    • The profile and backend database version files.

    • Information about the changelog database.

    • The server.out log file before capturing stack traces.

    • The server PID in a message in the tool’s log.

    • The cpuinfo, meminfo, slabinfo, and buddyinfo files on Linux systems.

    • Stack traces with jcmd tool, falling back to the jstack tool, and then to sigquit (or kill -3 on Linux) as necessary.

    • Environment variables used in configuration expressions.

  • The extract generated by the tool is now compatible with the Java 11 JVM unified logging framework.

Byte-by-byte LDIF comparisons

The ldifdiff command now supports a new -x, --exactMatch option for byte-by-byte LDIF comparisons.

This is useful for comparing LDAP schema files, for example.

DS 6.5.6

DS 6.5.6 is the final release targeted for DS 6.5.x deployments and can be downloaded from the ForgeRock Backstage website.

DS 6.5.6 can be deployed as an initial deployment or updated from an existing DS 6.5.x deployment.

  • The supportextract command now collects environment variables used in configuration expressions.

DS 6.5.5

  • DS servers now more effectively calculate reservable memory when using G1 garbage collection, and reduce the risk of long fsync pauses.

    This change introduces a ds-mon-db-cache-size-total metric to track the maximum size of the database cache. It also changes the ds-mon-db-log-size-active metric to reflect only live data.

  • The supportextract command now uses the jcmd command, if available, for heap dumps. Otherwise, it uses the jmap command.

DS 6.5.4

There are no new features introduced in DS 6.5.4, only bug fixes.

DS 6.5.3

DS servers now support storing the ads-certificate key pair and peer DS servers' trusted public keys in a PKCS#11 module, such as a hardware security module (HSM). This means you can store the keys used to secure replication traffic and to protect symmetric keys in an HSM. Previously, DS servers supported use of a PKCS#11 module only to store the keys used to secure other communications.

DS servers support PKCS#11 modules through the JVM. How to configure the JVM to allow DS servers to use your module, how to generate keys in the module, and how to export and import public key certificates all depend on your specific HSM/PKCS#11 module.

For details on how to perform such actions, refer to your PKCS#11 module’s documentation.

Before trying this feature, perform these tests:

  • Test that the PKCS#11 module supports multiple aliases for the same certificate.

  • Test that the PKCS#11 module supports generating a key pair with an RSA self-signed certificate and a key size of 2048 bits.

  • Test the replication topology using the default JKS ads-truststore implementation.

    Verify that replication functions properly before using the PKCS#11 module.

After validating the results of the tests, use the new feature by performing these steps for each server:

  1. Configure JVM security to enable use of the PKCS#11 module for the server.

  2. Generate a key pair using an RSA self-signed certificate and a key size of 2048 bits with the alias ads-certificate on the PKCS#11 module.

  3. After generating the key pair:

    1. Export the ads-certificate certificate, and reimport it on the PKCS#11 module using the MD5 fingerprint as the certificate alias.

      For example, if the ads-certificate certificate MD5 fingerprint is 07:35:80:D8:F3:CE:E1:39:9C:D0:73:DB:6C:FA:CC:1C, reimport the certificate with the alias 073580D8F3CEE1399CD073DB6CFACC1C.

    2. Prepare LDIF to update cn=admin data for the new certificate.

      The LDIF adds the new certificate as an instance key, and sets the key ID for the current server to use the key. In the following example LDIF, the certificate alias is 073580D8F3CEE1399CD073DB6CFACC1C, and the server hostname:_admin-port_ combination is opendj.example.com:4444:

      dn: ds-cfg-key-id=073580D8F3CEE1399CD073DB6CFACC1C,cn=instance keys, cn=admin data
      changetype: add
      ds-cfg-key-id: 073580D8F3CEE1399CD073DB6CFACC1C
      ds-cfg-public-key-certificate;binary:: MIIB6zCCAVSgAwIBAgIEDKSUFjANBgkqhkiG9w0BA
      QUFADA6MRswGQYDVQQKExJPcGVuREogQ2VydGlmaWNhdGUxGzAZBgNVBAMTEm9wZW5hbS5leGFtcGxl
      LmNvbTAeFw0xMzAyMDcxMDMwMzNaFw0zMzAyMDIxMDMwMzNaMDoxGzAZBgNVBAoTEk9wZW5ESiBDZXJ
      0aWZpY2F0ZTEbMBkGA1UEAxMSb3BlbmFtLmV4YW1wbGUuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNAD
      CBiQKBgQCfGLAiUOz4sC8CM9T5DPTk9V9ErNC8N59XwBt1aN7UjhQl4/JZZsetubtUrZBLS9cRrnYdZ
      cpFgLQNEmXifS+PdZ0DJkaLNFmd8ZX0spX8++fb4SkkggkmNRmi1fccDQ/DHMlwl7kk884lXummrzcD
      GbZ7p4vnY7y7GmD1vZSP+wIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAJciUzUP8T8A9VV6dQB0SYCNG1o
      7IvpE7jGVZh6KvM0m5sBNX3wPbTVJQNij3TDm8nx6yhi6DUkpiAZfz/OBL5k+WSw80TjpIZ2+klhP1s
      srsST4Um4fHzDZXOXHR6NM83XxZBsR6MazYecL8CiGwnYW2AeBapzbAnGn1J831q1q
      objectClass: top
      objectClass: ds-cfg-instance-key
      
      dn: cn=opendj.example.com:4444,cn=Servers,cn=admin data
      changetype: modify
      replace: ds-cfg-key-id
      ds-cfg-key-id: 073580D8F3CEE1399CD073DB6CFACC1C

      Do not yet update cn=admin data. You must not change the server key ID until the server is ready to use the PKCS#11 module.

  4. Edit the ads-truststore trust manager provider configuration to access the PKCS#11 module with the following properties:

    trust-store-file

    Leave this unchanged.

    trust-store-pin

    Set this to the PIN for the PKCS#11 module.

    You can avoid exposing secrets in the configuration by using expressions. For details, refer to Property value substitution.

    trust-store-type

    Set this to PKCS11.

  5. Stop the DS server.

  6. Using the ldifmodify command while the server is stopped, update cn=admin data for the server with the LDIF you prepared.

    This step changes the key ID for the server, letting it use its keys held in the PKCS#11 module.

  7. On another, running server replica, use the ldapmodify command to update cn=admin data for the server with the LDIF you prepared.

    Replication propagates the change to the other running server replicas.

    This step changes the key ID for the server, letting other servers trust its new certificate once it restarts.

  8. Restart the DS server.

DS 6.5.2

There are no new features introduced in DS 6.5.2, only bug fixes.

DS 6.5.1

There are no new features introduced in DS 6.5.1, only bug fixes.

DS 6.5.0

Connection limiting

DS servers now allow you to limit the number of concurrent connections per client.

For details, refer to Connection limits.

Data distribution

DS proxy servers now support simple, non-elastic data distribution.

You can configure a proxy server to equitably distribute LDAP write requests across multiple replication partitions to scale the directory service horizontally. As the present implementation does not permit elastic scaling or data redistribution, make sure that you understand the documented constraints of the present implementation before deploying it in production.

For an example, refer to Data distribution.

Database caching

A new DS directory server uses shared cache by default for all JE database backends. As a result, you are no longer required to set the database cache size using the db-cache-percent or db-cache-size setting for each backend.

It remains possible to use these settings if necessary by configuring them appropriately.

Logging

  • DS servers now support sending access log messages to standard output.

    For details, refer to Log access to standard output.

    When the new handler is used, standard output from the server includes a mix of JSON for access messages and non-JSON DS-format server event messages.

  • Common audit logging now supports denying log message fields to prevent them from showing up in log messages.

    For an example, refer to Deny log message fields.

  • Common audit logging now supports writing multiple file-based logs to the same directory by setting a different log-file-name-prefix for each file-based log.

Monitoring

DS servers now provide health status checks for anonymous requests over HTTP and LDAP. This allows a remote application to check that a server is "alive" and "healthy".

Anonymous HTTP requests can retrieve "alive" and "healthy" status codes. Anonymous LDAP requests can retrieve "alive" and "healthy" boolean values.

The "alive" and "healthy" status indicates that the server has passed its own internal tests. It is not, however, a guarantee that the server is free from other errors. If a server is not "alive," it requires administrative intervention. If a server is not "healthy," temporarily route requests to another server.

When a server is not "alive" or "healthy," a user with privileges to read monitoring information receives health status error messages in the body of the HTTP response, and can obtain health status error messages over LDAP. No error messages are returned in response to anonymous requests.

For examples demonstrating how to use this feature, refer to monitoring.

When you upgrade DS servers to 6.5.0, the anonymously accessible HTTP endpoints are not configured. To add the endpoints on an upgraded server that lacks them, use the dsconfig command:

$ dsconfig \
create-http-endpoint \
--endpoint-name /alive \
--type alive-endpoint \
--set enabled:true \
--set authorization-mechanism:HTTP Anonymous \
--set authorization-mechanism:HTTP Basic \
--hostname opendj.example.com \
--port 4444 \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--trustAll \
--no-prompt

$ dsconfig \
create-http-endpoint \
--endpoint-name /healthy \
--type healthy-endpoint \
--set enabled:true \
--set authorization-mechanism:HTTP Anonymous \
--set authorization-mechanism:HTTP Basic \
--hostname opendj.example.com \
--port 4444 \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--trustAll \
--no-prompt

Platform Integration

When setting up a directory server for use with other ForgeRock Identity Platform component products, you can use available setup profiles to greatly simplify initial configuration.

For details, refer to Setup profiles.

Replication

When configuring a replication server on a multi-homed system with multiple IP addresses, you can now specify which listen addresses to use.

Set the property, listen-address, as shown in Listen addresses.

Support

The DS support extract command, supportextract, now ships with DS server software, making it easier to capture troubleshooting information.

The command works on all supported platforms.

For details, refer to supportextract.

DS 6.0.0

Backend database storage

  • Directory servers now use an optimized JE backend implementation whose dependencies are distributed under the Apache License, Version 2.0. This license is suitable for all deployments, including OEM deployments.

    Support for PDB backend databases has been removed in 6.0.0.

    Before upgrading a directory server using any PDB backends, take one of the following actions:

    • Move the data to JE backend databases before upgrading.

    • Export all data in PDB backend databases to LDIF before upgrading, and import the data into the JE backend databases with the same names after upgrading.

  • Backend indexes now have new time to live (TTL) properties to configure automated, optimized entry expiration and removal:

    For details on how to use this feature, refer to Entry expiration.

  • DS 6.0.0 upgrades JE backend databases to Berkeley DB Java Edition 7.5.11.

  • DS 6.0.0 improves many JE backend settings:

    • A new advanced property, db-durability, makes the backend durability setting easier to configure and to read.

    • New defaults for disk space thresholds better fit modern deployments.

      The property, disk-low-threshold, defaults to 5% of the filesystem size, plus 5 GB.

      The property, disk-full-threshold, defaults to 5% of the filesystem size, plus 1 GB.

    • Default limits for backend database log files better fit larger deployments.

      The property, db-log-file-max, defaults to 1 GB instead of 100 MB.

      The property, db-log-filecache-size, defaults to 200 instead of 100.

      As a result, the database can grow to 200 GB instead of 10 GB on disk before the file cache begins to close database log files in order to open others.

    • The new properties, db-run-log-verifier, and db-log-verifier-schedule, make it possible to configure whether and when the server runs checksum verification on backend database logs.

Configuration expressions

  • Server configuration expressions have been reimplemented to align with other ForgeRock Identity Platform software. Configuration expressions make it possible to substitute configuration property values with variables that you can set before starting DS servers.

    For details, refer to Property value substitution.

Faster bulk updates

  • The ldapmodify and ldapdelete commands now offer a --numConnections option to perform updates in parallel on multiple LDAP connections.

    This feature enables faster bulk updates, and provides an alternative to the import-ldif --append option removed in version 3.0.

JSON support

  • In addition to optimized indexes for JSON attribute values that are queried with common REST query filters, DS directory servers now also support optimizations for JSON with optional fields.

Monitoring

  • DS server monitoring has been reimplemented to align with other ForgeRock Identity Platform software.

    This change affects LDAP and HTTP monitoring interfaces, but not SNMP interfaces. The SNMP interface continues to use standard metrics.

    Monitoring capabilities now have interface stability Stable.

    For documentation about available interfaces and metrics, refer to Monitoring.

    DS servers now feature the following monitoring capabilities:

    • Support for pulling monitoring data to Prometheus monitoring software.

    • Support for pushing monitoring data to a Graphite service.

    • Support for creating a directory monitoring account during setup.

      When using JMX to monitor the server with this account, be aware that in 6.0.0 the account does not have JMX-related privileges. Instead, you must add the required JMX-related privileges, as described in JMX-based monitoring.

  • DS servers support a new privilege, monitor-read. This prevents unauthorized users from reading monitoring metrics unless they have the privilege.

    Assign this privilege to users who read monitoring metrics over LDAP or HTTP.

    When upgrading, add the missing privilege to the global administrator account. These privileges are required when using the dsreplication status command.

Performance

  • DS 6.0.0 includes many performance improvements.

    No DS server performance improvements require action on your part, though optimal tuning settings including JVM settings may now be different for specific scenarios.

    When upgrading to this release, be aware that the command-line performance tools have a new template value syntax.

Replication

  • DS servers now allow you to choose a single, global server ID used when configuring replication, rather than letting replication configuration randomly assign server IDs.

    Before configuring replication, you can set the global configuration property server-id.

    This makes it easier to keep track of server IDs when reviewing replication configuration, and to parameterize replication configuration in DevOps deployments.

  • DS directory servers store historical replication information for internal use in entries' ds-sync-hist attributes. This release introduces a new encoding that significantly reduces the space required to store ds-sync-hist data.

    The space reduction trades a smaller footprint for increased CPU use when preparing to write ds-sync-hist values. Read and search operations should not be negatively impacted, however. Indeed, read and search performance should improve to the extent that reduced entry size means more efficient use of backend database and CPU caches.

  • DS replication domains can now be disabled by setting enabled:false for the domain. If a replication domain is disabled, its contents are not replicated.

    This change facilitates parameterizing whether backends and associated replication domains are enabled, which is useful in DevOps deployments where not all environments replicate the same data on each replica.

Security

  • DS certificate mappers now support certificate issuer verification. Use this to verify the certificate issuer whenever multiple CAs are trusted in order to prevent impersonation. Different CAs can issue certificates with the same subject DN, but not with the same issuer DN.

    This feature relies on the certificate mapper property, issuer-attribute, to identify the attribute in the user entry that holds the issuer DN. For this purpose, DS servers define the attribute, ds-certificate-issuer-dn, as an optional attribute of the ds-certificate-user object class.

    For an example using this attribute, refer to Certificate-based authentication.

Server configuration

  • The dsconfig command now allows you to configure DS servers that are not running.

    DS 6.0.0 introduces an --offline option and a --configFile option. When you use the dsconfig --offline command with a DS server that is stopped, you change the server configuration file in the default location, such as /path/to/opendj/config/config.ldif. The --configFile option allows you to specify an alternative configuration file in offline mode.

Server-side sorting

  • JSON ordering matching rules for JSON attributes.

    DS servers can implement JSON ordering matching rules on demand. This enables REST to LDAP to support for _sortKeys where a field is inside the value of a JSON attribute.

  • An extension of the server-side sort request sort order specification.

  • A means for REST to LDAP to communicate the _sortKeys field that is inside the value of a JSON attribute to the DS server.

    This functionality is part of REST to LDAP. By using the useServerSideSortForJson boolean configuration parameter, you can configure whether to sort results in the REST to LDAP gateway.

  • A means for the REST to LDAP gateway to limit the maximum number of entries supported by the local sort mechanism when sorting results based on JSON attributes.

    The setting is localSortMaxEntries.

Unindexed Searches

DS directory servers now support the following improvements for unindexed searches. These improvements are designed to help applications that store data in the directory as arbitrary JSON objects, and that provide a graphical UI for browsing directory data accessed over REST. With these improvements, users can page through directory data, sorting on whichever JSON field they choose without initially specifying any filter.

As with any unindexed search that you allow, the tradeoff is inefficient use of system resources and less performance. This is not, therefore, a general capability that should be provided to all applications without taking the impact into consideration. It is intended for use by a directory data administrator who is browsing data without knowing in advance what they are looking for:

  • DS directory servers can now use an appropriately configured VLV index to sort results for an unindexed search.

  • DS directory servers sort unindexed search results as long as they are paged.

    This improvement has the following limitations:

    • The simple paged results control must specify a page size that is less than or equal to the index-entry-limit (default: 4000).

    • For each page, the server reads the entire backend database, retaining page size number of sorted entries.

DS 5.5.3

DS 5.5.3 is the final release targeted for DS 5.5.x deployments and can be downloaded from the ForgeRock Backstage website.

DS 5.5.3 can be deployed as an initial deployment or updated from an existing DS 5.5.x deployment.

There are no new features introduced in DS 5.5.3, only bug fixes.

DS 5.5.2

There are no new features introduced in DS 5.5.2, only bug fixes.

DS 5.5.1

There are no new features introduced in DS 5.5.1, only bug fixes.

DS 5.5.0

Backend database storage

  • JE backend databases are upgraded to Berkeley DB Java Edition 7.4.5 in this release.

Directory proxy

  • Directory proxy servers now automatically retry operations when they detect a temporary failure on the remote directory server.

    For details, refer to About failures.

  • The new proxy backend property, heartbeat-search-request-base-dn, lets you configure proxy backend heartbeat requests to target an entry under a base DN of interest rather than targeting the root DSE.

  • When the global configuration property, trust-transaction-ids, is set to true, the proxy backend now adds a ForgeRock transaction ID control before forwarding the request, even if the incoming request did not include the control.

    As a result, all proxied requests have ForgeRock transaction IDs when you configure the server to trust transaction IDs.

Monitoring

  • Replication server configurations now include these advanced properties for monitoring disk space use and stopping operations when the disk is full:

    disk-low-threshold

    When this threshold is reached, the server logs warnings and sends warnings to the disk space monitoring subsystem.

    The directory administrator must take action to provide more disk space.

    disk-full-threshold

    When this threshold is reached, the server stops operations and lets connected directory servers fail over to another replication server. The replication server can resume operations once free disk space rises above the disk-low-threshold setting.

  • Servers and the REST to LDAP gateway now include a ForgeRock transaction ID with each request.

    If you do not configure the server or gateway to trust transaction IDs in client application requests, then they ignore incoming transaction IDs, and instead generate a transaction ID for each request.

    If you configure the server or gateway to trust transaction IDs in client application requests, then outgoing requests reuse the incoming transaction ID. For each outgoing request in the transaction, the request’s transaction ID has the form original-transaction-id/sequence-number, where sequence-number reflects the position of the request in the series of requests for this transaction. For example, if the original-transaction-id is abc123, the first outgoing request has transaction ID abc123/0, the second abc123/1, the third abc123/2, and so on. This helps you to distinguish specific requests within a transaction when correlating audit events from multiple services.

    To configure a server to trust transaction IDs in client application requests, set the global configuration property, trust-transaction-ids, to true.

    To configure the REST to LDAP gateway to trust transaction IDs in client application requests, set the JVM system property, org.forgerock.http.TrustTransactionHeader, to true in the web application container where the gateway runs.

  • When an internal search is unindexed, a directory server now logs a message.

REST to LDAP

  • REST APIs now support the _sortKeys parameter to request that the server sort the query results it returns.

  • REST to LDAP now uses the affinity load balancer.

Security

  • DS servers use SHA-256 as the signature algorithm when generating key pairs, as an attacker with sufficient computing power could break SHA-1.

    NIST Special Publication 800-131A, Transitions: Recommendation for Transitioning the Use of Cryptographic Algorithms and Key Lengths, generally disallows use of SHA-1 for digital signature generation.

  • DS servers now support using a PKCS#11 device, such as a hardware security module (HSM), as a truststore.

    When you use a PKCS#11 device as a truststore, all trusted certificates must be present on the device. No CA certificates are available by default. Import all the signing certificates required for your deployment before configuring the device for use as a truststore.

    To use an HSM as a truststore:

    1. Configure the JVM to allow access to the PKCS#11 device.

    2. Using the dsconfig command, create a PKCS#11 trust manager provider configuration to access the PKCS#11 device as a truststore.

Simplified replication server setup

  • Use the new setup replication-server command to set up a server as a standalone replication server.

    You can use this command to install standalone replication servers without specifying the base DNs to replicate.

Tools

  • The dsconfig command now allows you to switch to advanced mode while using the command interactively.

  • The setup --help command now presents options in sorted order.

Copyright © 2010-2024 ForgeRock, all rights reserved.