What’s new
|
DS 7.5.1
DS 7.5.1 is the latest release targeted for DS 7.5 deployments and can be downloaded from the download page.
The release can be deployed as an initial deployment or updated from an existing DS 7.5.x or earlier deployment.
DS 7.5.0
Disaster recovery
-
DS servers now support safer disaster recovery procedures you can apply one server at a time.
The procedures hinge on the new
dsrepl disaster-recovery
command. The new subcommand replaces the previous subcommands, which have been removed. For details, refer to the disaster recovery documentation.
HDAP
-
HDAP now supports HTTP Bearer authorization with an access token.
This is especially useful when performing multiple operations as a user with a strong password storage scheme. The computational cost to validate the password is high, but you only pay the cost once. The cost to validate a JWT token is low for each subsequent operation. For details, refer to Bearer auth and to the code samples in the HDAP documentation.
Indexing
-
DS servers no longer require rebuilding indexes for attributes that have never been used before.
-
DS servers are now more efficient in using equality indexes as a replacement for presence indexes when processing intersection search filters.
-
DS servers can now fall back to VLV indexes for some true and equality searches with appropriate sort keys.
For details, refer to Index use by matching rule.
Access logs
-
DS servers now log search index diagnostics for unindexed searches.
When the
response
>additionalItems
>unindexed
field of an access log message istrue
, review theresponse
>additionalItems
>debugSearchIndex
object to diagnose why the search was not indexed. -
Log messages now list LDAP control aliases instead of OIDs in the response details.
Monitoring
-
DS monitoring metrics now include specific metrics for persistent search etimes:
-
The LDAP attribute on an LDAP connection handler monitoring entry is
ds-mon-requests-psearch
. -
The Prometheus metric is
ds_connection_handlers_ldap_requests_count{ldap_handler,type="psearch"}
.
-
-
DS Prometheus monitoring output now complies better with third-party applications. It follows the Prometheus text format.
To continue using the previous format for now, set
legacy-format:true
in the Prometheus endpoint configuration. This setting is deprecated and likely to be removed in a future release.
Resource limits
-
DS resource limits now depend on the proxy authorization identity rather than the bind DN.
Java
-
DS now supports Java 17 and 21.
With Java 21 you can try experimental virtual thread support, a DS Technology Preview (not for production use).
When trying this experimental feature, run DS with a Project Loom early-access build of Java. At the time of this writing, the early-access builds are based on JDK 23. The early-access builds include fixes preventing thread starvation and deadlocks in DS under extreme write loads.
DS uses virtual threads for core processing, not for network I/O, replication, or other features. The use of virtual threads is expected to expand to other features in future releases. For details, refer to Java settings.
Tools
-
The new
dsrepl decode-csn
command helps debug replication issues. It displays the components of one or more valid CSNs to show when they were generated and which server generated them. -
The
supportextract
command now includes the system hostname in the name of the archive file. -
The
upgrade
command now stops the upgrade process if the previous version is DS 7.4.0 and the server uses data encryption (confidentiality) with the default AES/GCM cipher.For instructions, refer to Upgrade from DS 7.4.0.
DS 7.4
DS 7.4.3
DS 7.4.3 is the latest release targeted for DS 7.4 deployments and can be downloaded from the ForgeRock Backstage website.
The release can be deployed as an initial deployment or updated from an existing DS 7.4.x or earlier deployment.
DS 7.4.2
DS 7.4.2 is a maintenance release with the following improvements.
Disaster recovery
-
DS servers now support safer disaster recovery procedures you can apply one server at a time.
The procedures hinge on the new
dsrepl disaster-recovery
command. For details, refer to the disaster recovery documentation.
DS 7.4.1
DS 7.4.1 is a maintenance release with the following improvements.
Security
-
The DS Crypto Manager configuration now has an advanced property,
key-wrapping-mode
, to set the key wrapping mode for protecting symmetric keys.When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set
key-wrapping-mode: WRAP
.
DS 7.4.0
HDAP
This release introduces the HTTP Directory Access Protocol (HDAP) API for HTTP access to LDAP data.
Interface stability: Technology Preview
HDAP enables web-based HTTP applications and services to interact with LDAP directories. It uses HTTP as the transport protocol and JSON as the data format, making directory data easy to access and use in web applications. With HDAP, web-based systems and LDAP directories communicate and integrate simply and easily.
HDAP exposes the full hierarchical object-oriented data model of DS with the following benefits:
-
Supports all features of the LDAP protocol and many extensions
-
Simplifies configuration; no complex data mappings
-
Manages LDAP schema as data
-
Lets applications validate resources using their JSON schema
-
Opens access to advanced administrative features such as collective attributes, password policy sub-entries, and access controls
-
Lets applications perform subtree searches and rename resources
For details, read Use HDAP.
Access logs
-
DS log messages now include
elapsedQueueingTime
, the time the request waited in the queue, andelapsedProcessingTime
, the time actively processing the request. For details, refer to About logs.Use the following new access log filtering criteria for logs targeting outliers:
-
response-etime-queueing-greater-than
-
response-etime-queueing-less-than
-
response-etime-processing-greater-than
-
response-etime-processing-less-than
-
-
DS log messages with
additionalItems
fields now set their additional items totrue
instead ofnull
.For example, the message for an unindexed search now includes
"additionalItems":{"unindexed":true}
. -
DS can now record the attributes targeted by modification requests.
For details, refer to Log modifications.
-
DS now logs security information about TLS handshake operations, including the negotiated protocol version and cipher, and the resulting security strength factor (SSF).
A new
tls
setting for access log filtering criteria lets you filter on TLS handshakes.For details, refer to About logs.
-
DS access logs messages now include the user-friendly name and criticality for LDAP controls. For request controls, the messages also include the values.
For details, refer to About logs.
Debug and error logs
-
Configuring debug-level logging is simpler with predefined logging categories.
For details, refer to Debug-level logging.
Monitoring
-
DS monitoring metrics now include information to help troubleshoot changelog purging:
-
The LDAP attributes under
cn=monitor
are:
ds-mon-changelog-file-count
ds-mon-purge-waiting-for-change-number-indexing
-
The Prometheus metrics are:
ds_replication_changelog_purge_waiting_for_change_number_indexing
ds_replication_changelog_replica_dbs_changelog_file_count
-
Password policy
-
DS attribute value password validators with
ds-pwp-attribute-value-check-substrings:true
orcheck-substrings:true
now check whether the password contains portions of attribute values and whether the attribute values contain portions of the password.
Collective attributes
-
DS servers now let you rename collective attributes.
For details, refer to Rename an attribute.
Schema
-
New DS servers now check certificate lists, certificate pairs, and postal address syntax attributes for validity. The change affects attributes such as
certificateRevocationList
,crossCertificatePair
, andpostalAddress
.The change doesn’t apply to DS servers you upgrade in place.
Replication
-
DS servers now log a warning message such as the following when replication status is
Bad data
and the problem seems to be the fractional replication configuration:Replication server RS([.var]##<server-id>##) ignoring update [.var]##<csn>## for domain "[.var]##<domain>##" from directory server DS([.var]##<server-id>##) at [.var]##<server-address>## because the peer DS reported to be in BAD_DATA status. The generation ID matches, (DS is [.var]##<generation-id>##, RS is [.var]##<generation-id>##), there may be a problem with fractional replication configuration. Check the DS error logs for more details
Performance
-
DS servers now more efficiently process OR filters composed of equality filters using the same attribute types.
An example LDAP search filter of this type is
(|(uid=uid1)(uid=uid2)…(uid=uidN))
, where N can be large.
Security
-
DS now supports accepting Kerberos authentication requests for more than one service principal. For details, refer to Authenticate with Kerberos and the GSSAPI SASL mechanism handler configuration.
-
DS makes configuring a PKCS#11 HSM key manager easier than before. For details, refer to the example in PKCS#11 hardware security module.
Native packages
-
DS Debian and Red Hat packages now use systemd instead of System V init files.
For details, refer to Use the Debian package or Use the RPM package.
Tools
-
The
dsconfig
command online help and Configuration reference now label deprecated and legacy configuration objects and properties. -
Many commands with the
--usePkcs11KeyStore
option now also support the following options:-
--providerArg
to specify the provider configuration. -
--providerClass
or--providerName
to specify the implementation.
-
-
All
backendstat list-*
subcommands now display indexes ordered by name.
DS 7.3
DS 7.3.5
DS 7.3.5 is the latest release targeted for DS 7.3 deployments and can be downloaded from the download page.
The release can be deployed as an initial deployment or updated from an existing DS 7.3.x or earlier deployment.
Disaster recovery
-
DS servers now support safer disaster recovery procedures you can apply one server at a time.
The procedures hinge on the new
dsrepl disaster-recovery
command. For details, refer to the disaster recovery documentation.
DS 7.3.4
Security
-
The DS Crypto Manager configuration now has an advanced property,
key-wrapping-mode
, to set the key wrapping mode for protecting symmetric keys.When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set
key-wrapping-mode: WRAP
.
DS 7.3.0
Replication
-
DS servers now send data more efficiently when initializing a replica online, improving the speed of online initialization. This improvement requires that both servers run DS 7.3 or later.
-
DS replication now distinguishes when a replica requires reinitialization because it has fallen further behind the replication server than allowed by the
replication-purge-delay
. DS sets theds-mon-status
attribute for LDAP ords_replication_replica_status{status}
for Prometheus toToo late
. Earlier versions of DS assignedBad generation id
status to such replicas.The
dsrepl status
output changed to take advantage of the new status. It now distinguishes the following states for a replicated data set:BAD - DATA MISMATCH
-
Requires reinitialization; verify the replication configuration
BAD - TOO LATE
-
Requires reinitialization
GOOD
-
Normal operation; nothing to do
SLOW
-
Replication delay greater than five seconds
Groups
-
DS now uses significantly less memory for the group cache and for entry caches.
Revisit Java heap size and database cache settings after upgrading. For details on setting heap and cache sizes, refer to Performance tuning.
-
DS now more effectively reads and updates entries with attributes having many values, such as LDAP and POSIX group entries.
DS entry caches are no longer necessarily required for these entries. If you have enabled entry caches for large groups, consider removing them after upgrade.
-
DS monitoring metrics now include counts of static, dynamic, and virtual static groups, and statistics on the distribution of group sizes.
For details, refer to Groups (Prometheus) and Groups (LDAP).
Indexing
When you upgrade DS directory servers in place, you must rebuild all indexes. The rebuilt indexes reflect required string normalization fixes. If possible, trigger this rebuild during in place upgrade. Normal operations can result in degraded index errors until you rebuild the indexes. |
-
DS servers now let you monitor index cost, which enables you to determine which indexes are causing write contention.
For details, refer to Index cost.
-
DS servers now support a new matching rule, making it easier to monitor progress when migrating passwords to a new storage scheme.
For an example, refer to Eliminate reversible password storage.
-
DS servers now display a message when you configure an unnecessary presence index for an attribute that already has an equality index DS can use for presence searches.
DS servers also display the message at startup.
Monitoring
-
DS monitoring for Prometheus now includes a metric for replica status.
For details, refer to Replication status (Prometheus).
-
DS monitoring metrics now include counts global and entry ACIs, and of the number of entries with ACIs.
For details, refer to ACIs (Prometheus) and ACIs (LDAP).
-
DS monitoring metrics now include counts of the memory allocated to entry caches.
For details, refer to Entry cache size (Prometheus) and Entry cache size (LDAP).
-
DS monitoring metrics now include a count of LDAP subentries.
For details, refer to Subentries (Prometheus) and Subentries (LDAP).
-
DS monitoring metrics now include information to help troubleshoot change number indexing:
-
The Prometheus metrics are:
ds_change_number_indexing_state
ds_change_number_time_since_last_indexing_seconds
-
The LDAP attributes under
cn=monitor
are:
ds-mon-indexing-state
ds-mon-replicas-preventing-indexing
ds-mon-time-since-last-indexing
When the indexing state is not
INDEXING
, also read the replication logs for warning messages with additional details. -
-
The metrics counting the number of indexed and unindexed searches have been renamed
ds-mon-backend-filter-indexed
andds-mon-backend-filter-unindexed
, and are now always maintained, even when index filter analysis is disabled.
Logging
-
DS servers can now produce error messages in JSON format.
For details, refer to Log error messages as JSON.
Schema
-
DS servers now allow
mail
addresses to include UTF-8 characters, not just ASCII.The change does not affect directory data but does affect
mail
indexes which may become degraded. When upgrading, you may need to rebuild degraded indexes. For details, refer to When adding new servers and Update LDAP schema.
Tools
-
The
modrate
command now includes new options for adding and replacing values of multivalued attributes. This lets you use the command to simulate more realistic workloads. Previously,modrate
only supported single-valued replace updates.The new options are:
--strategy (modify|read_modify)
-
Set this option to
--strategy read_modify
to use the new feature, reading an entry, then modifying it by deleting and adding attribute values.The default,
--strategy modify
, uses single-valued replace updates as before. --valueCount number
-
Set this option to specify how many attribute values the multivalued attribute should contain following the modify operation. The number default is 1.
--mvcc attribute
-
Optionally set this option to specify the attribute to use for MVCC. Default:
--mvcc eTag
.
For details, refer to the modrate reference page.
DS 7.2
DS 7.2.5
DS 7.2.5 is the latest release targeted for DS 7.2 deployments and can be downloaded from the download page.
You can deploy this release as an initial deployment or use it to update an existing DS 7.2.x deployment.
Disaster recovery
-
DS servers now support safer disaster recovery procedures you can apply one server at a time.
The procedures hinge on the new
dsrepl disaster-recovery
command. For details, refer to the disaster recovery documentation.
DS 7.2.4
Security
-
The DS Crypto Manager configuration now has an advanced property,
key-wrapping-mode
, to set the key wrapping mode for protecting symmetric keys.When using a FIPS-compliant security provider that doesn’t allow direct encryption, change the Crypto Manager configuration to set
key-wrapping-mode: WRAP
.
DS 7.2.0
Backup
-
DS servers now support Amazon AWS temporary credentials when backing up and restoring data using S3.
You set the AWS session token using the
s3.sessionToken.env.var
storage property. For example, first set the session token as the value of theAWS_SESSION_TOKEN
environment variable, then use--storageProperty s3.sessionToken.env.var:AWS_SESSION_TOKEN
in thedsbackup
commands.For additional examples, refer to Cloud storage.
-
DS servers now send an alert notification when backup task completes.
The new alert types are
org.opends.server.BackupSuccess
andorg.opends.server.BackupFailure
, and are documented in Alert types.
Indexing
-
DS servers now support big indexes. A big index is a new kind of index optimized for attributes with few unique values. Big indexes let users more easily page through all the users in a US state, for example.
For details, refer to Big index and Indexes for attributes with few unique values.
-
DS servers now let you monitor index use, so you can determine which indexes are unused.
For details, refer to Unused indexes.
-
DS servers now support a DN pattern matching rule that lets you index an attribute with DN values, and search with wildcard characters, so you can find matches for specific RDNs in the DN, for example.
For details, refer to DN patterns.
-
DS servers have improved output for debugging search indexes.
For examples, refer to Debug search indexes. (As explained there, the format of
debugsearchindex
values is not a stable public interface, because it is intended for human beings, not scripts.) -
The output for the
backendstat list-indexes
andbackendstat show-index-status
commands is easier to read and to understand. -
DS servers now optimize searches for unresolved conflicts.
-
DS servers now more efficiently optimize searches for initial substrings.
Logging
-
DS servers now include
entrySize
in access log messages. You can filter access logs based on minimum entry size with the log filtering criteria setting,response-entry-size-greater-than
.For details, refer to About logs.
-
By default, DS servers are configured to manage log file retention and rotation. For details on configuring this, refer to Rotate and retain logs.
When an external program is also configured to manage DS log files, and moves or deletes log files in a way that a DS server does not expect, the DS now detects the change and logs an error message.
Either let the DS server manage its log files, or configure an external program to do so, not both.
Monitoring
-
DS monitoring now takes replication listener threads into account when calculating whether a server is healthy. Monitoring shows a server to be in a healthy state if the server is alive, the replication server is accepting connections on the configured port, and any replication delays are below the configured threshold.
-
DS servers now support histogram metrics, as described in Metric types reference.
As indicated in LDAP metrics reference and Prometheus metrics reference, DS servers expose the following histogram monitoring metrics:
- LDAP
-
ds-mon-backend-entry-size-read
ds-mon-backend-entry-size-written
- Prometheus
-
ds_backend_entry_size_read_bucket{backend,type,le}
ds_backend_entry_size_written_bucket{backend,type,le}
-
DS servers now let the monitor user read monitoring information over HTTP when some backends are offline, as long as backend with the monitor user entry remains online.
Password storage
-
DS servers now support hashing passwords with Argon2 for password storage.
For details, refer to Password Storage.
Performance
-
DS servers now generate ETag attribute values more efficiently.
This improves the performance of REST to LDAP applications that use ETags for MVCC. The plugin generates real ETag attributes for adds and updates. The server relies on the existing virtual attribute implementation only when a real ETag is not available.
The implementation depends on a server plugin that is only configured for new servers. After upgrading all servers, configure the plugin on each server to use the new feature. For details, refer to Use the entity tag plugin for ETags.
-
DS servers now more efficiently verify passwords stored with PKCS5S2.
-
DS servers now run the
rebuild-index
command more efficiently when you identify specific indexes to rebuild.They also now run the
rebuild-index --rebuildDegraded
command more efficiently when there are no indexes to rebuild. -
DS servers now start up more quickly when there are large numbers of groups.
When the server starts, it runs an internal search to find all groups. DS servers now maintain a big index for
objectClass
that is specific to groups.In previous versions, the search for groups at startup could be unindexed. The workaround was to raise the index entry limit for the
objectClass
index, with the tradeoff of maintaining indexes for more object classes, and impacting write performance. The workaround is no longer necessary for new servers.Upgrading does not change the server configuration, however, so the index is not present after you upgrade. If you have applied the workaround of raising
index-entry-limit
forobjectClass
, and have upgraded your servers:-
Install a new, throwaway server with the evaluation profile, as described in Install DS for evaluation.
-
Review the configuration for the
big-equality
index forobjectClass
.For example:
dsconfig get-backend-index-prop --backend-name dsEvaluation --index-name objectClass --offline
-
For your upgraded servers, consider adding a
big-equality
index for the groups, loweringindex-entry-limit
forobjectClass
, and rebuilding theobjectClass
indexes.Server startup time should be just as good, and write performance might improve.
-
Proxy
-
DS servers now support the Proxy Protocol from HAProxy.
For details, refer to Proxy protocol.
-
The proxy backend settings to regularly contact remote LDAP servers now offer additional configuration for more fine-grained control when keeping connections alive and checking remote server availability.
For details, refer to Proxy backend.
Replication
-
DS replication servers now check that the port is available when you change the configuration.
REST to LDAP
-
When you perform a paged results query whose corresponding LDAP search is indexed, the response now contains an estimated number of
"totalPagedResults"
, and"totalPagedResultsPolicy" : "ESTIMATE"
.For an example, refer to Paged results.
-
When you perform a query, you can now request the resource count only, using the new
_countOnly
query string parameter. REST to LDAP returns the count, and not the resources.This parameter requires protocol version 2.2 or later. Use a header like
Accept-API-Version: protocol=2.2,resource=1.0
, for example.For details, refer to Query.
-
When converting JSON values, REST to LDAP now coerces:
-
Strings to booleans, integers, or JSON where possible.
-
Whole floating point numbers to integers.
REST to LDAP also returns helpful errors when coercion fails. This improves interoperability with client applications that do not or cannot perform the conversions before adding or updating resources.
-
-
The REST to LDAP gateway settings now let you configure:
-
Availability checks for load balancing.
The default heartbeat check settings have also been changed to check that pooled connections are alive every five minutes with a three-second keep-alive heartbeat timeout.
-
As many pools of failover servers as needed.
You specify the pools using the
"failoverLdapServers"
field. The gateway still accepts"primaryLdapServers"
and"secondaryLdapServers"
settings for compatibility. -
A connection timeout.
For details regarding these new settings, refer to LDAP connection factories.
-
-
Internally, REST to LDAP now simplifies search filters when possible. This can improve search performance in some cases.
REST to LDAP removes redundant
objectClass
assertions from search filters, retaining specific classes, but removing the superclasses they inherit from. For example:(&(objectClass=top)(objectClass=person)(objectClass=organizationalPerson)
Becomes:
(objectClass=organizationalPerson)
-
REST to LDAP now updates single-valued LDAP attributes by replacing the value, which reduces the network bandwidth and historical change data needed to replicate the update.
Schema
-
The schema definitions in the
db/schema/04-rfc2307bis.ldif
file now align with those of the latest RFC 2703bis Internet-Draft, An Approach for Using LDAP as a Network Information Service.The change does not affect directory data, but when upgrading you may need to rebuild degraded indexes. For details, refer to When adding new servers and Update LDAP schema.
Security
-
PKCS#11 hardware security module now explains how to use an HSM for all asymmetric keys, including the shared master key, for data that is not (yet) encrypted.
If you plan to use an HSM for the shared master key, read the documentation carefully before you install DS. When you set up the server, you must avoid accidentally encrypting data while using the wrong shared master key.
For details, refer to Store the shared master key.
Tools
-
A new DS
bash-completion
command generates a completion script for the Bash shell that makes it easier to write other DS commands.The completion script depends on support for
bash-completion
, which is not included by default on macOS.To set up Bash completion for DS commands, source the output of the script:
-
Bash 4
-
Bash 3.2 macOS
source <(/path/to/opendj/bin/bash-completion)
# First, install bash-completion support. # Next: eval "$( /path/to/opendj/bin/bash-completion )"
You can make completion available in any new interactive shell by adding it to your
~/.bash_profile
file, or~/.bashrc
file if it is loaded by the new shell. -
-
The new
dskeymgr show-deployment-id
command displays key information about a given deployment ID—formerly known as a deployment key—such as the expiration date for the derived CA certificate.For details, refer to Show deployment ID information.
-
The
dsrepl status --showReplicas
command now displays anEntry count
column.The entry counts in each row reflect the number of entries in the specified replica under the specified base DN.
-
The
supportextract
command now collects additional system information, including data to indicate whether the system is running in a virtual machine. -
When collecting environment variable values, the
supportextract
command now excludes environment variables whose names containPASS
,PWD
, and_PW
.
Virtual attributes
-
DS now lets you create virtual attributes based on the values of non-virtual attributes.
For details, refer to Template-based virtual attributes.
DS 7.1
DS 7.1.8
DS 7.1.8 is the latest release targeted for DS 7.1 deployments and can be downloaded from the download page.
The release can be deployed as an initial deployment or updated from an existing DS 7.1.x or earlier deployment.
DS 7.1.2
Performance
-
DS servers now run the
rebuild-index --rebuildDegraded
command more efficiently when there are no indexes to rebuild.
Tools
-
The
supportextract
command now collects additional system information, including data to indicate whether the system is running in a virtual machine. -
When collecting environment variable values, the
supportextract
command now excludes environment variables whose names containPASS
,PWD
, and_PW
.
DS 7.1.1
DS 7.1.1 is a maintenance release that does not include new features.
Java
DS 7.1.1 introduces support for Java 17 (17.0.3 or later) in addition to Java 11:
-
In Java 17, the PCKS#12 keystore encryption/Mac algorithm has been upgraded to
HmacPBESHA256
. Update to at least Java 11.0.12 if you have an application that runs Java 11 and must read the keystore. -
Use G1 GC (the default) instead of parallel GC. The setting is shown in Java Settings. Use of ZGC or Shenandoah is not recommended for production deployments at this stage.
For details, refer to Java.
If you are upgrading, refer to Supported Java.
DS 7.1.0
Backup
-
The
dsbackup
command now lets you set a non-default storage provider endpoint.For details, refer to Cloud Storage.
Indexing
-
The online rebuild index process is now less intrusive, more effective, and more robust. When you run a
rebuild-index
command while the server is online, the backend database remains available for directory operations during the rebuild.Individual indexes do appear as degraded and unavailable while the server rebuilds them. A search request that relies on an index in this state may temporarily fail as an unindexed search.
Logging
-
DS log messages now include the authorization ID for every request.
-
DS servers now support logging the internal delete operations triggered by entry expiration.
If you have set the
ttl-age
andttl-enabled
properties for a backend, use this feature by configuring an access log publisher to record messages about internal operations. When the server deletes an expired entry, it logs a message with"additionalItems":{"ttl": null}
in the response.For background, refer to Entry Expiration.
Passwords
-
Password quality checks using the password quality advice control now ensure that proposed passwords are not in the password history.
If the server finds the proposed password in the password history, this appears in the failing criteria returned with the advice response control.
In addition, the REST to LDAP response over HTTP now includes the password attribute type.
Replication
-
DS servers now let you restrict which replicas you trust to send updates.
By default, all directory servers in a replication topology trust all replicas. If a replica allows an update, then other servers relay and replay the update without further verification. This simplifies deployments where you control all the replicas.
In deployments where you do not control all the replicas, you can configure replication servers to accept updates only from trusted replicas. The trust depends on the certificate that a replica presents to the replication server when connecting.
For details, refer to Trusted Replicas.
-
DS servers now let you define replication group failover. This determines how a directory server selects the next group with replication servers to connect to when no replication server is available in the directory server’s own group.
To activate replication group failover, set the global configuration property,
group-id-failover-order
.For details on how a directory server chooses a replication server, refer to Replication Connection Selection.
-
When a replica’s last change is older than the oldest change recorded in the replication server’s changelog, the replication server now records the problem in its log, and sends a message to the replica. When it receives the message, a 7.1 replica remains connected to the replication server, but refuses update operations, effectively becoming read-only. A pre-7.1 replica closes the connection.
In any case, the replica no longer applies replication updates. Its data diverges more and more from other replicas' data.
Should this happen in your deployment, reinitialize the replica. For details, refer to Manual Initialization.
-
DS servers now log more explicit messages when they discover duplicate server IDs.
REST to LDAP
-
This release introduces support for querying fields of
reference
orreverseReference
resources that are subtypes of the resources you are searching.As an example, suppose that devices and users are both subtypes of a "managed object" type. Also, suppose that devices have a
deviceType
field, that users have asurname
field, and that a basic managed object has neither of these fields. Now, your queries on a collection of managed objects can match properties of the referenced subtypes, such as/managedObjects?_queryFilter=deviceType+eq+phone
, or/managedObjects?_queryFilter=surname+eq+Jensen
.
Samples
-
The sample for building custom DS Docker images now has a
USE_DEMO_KEYSTORE_AND_PASSWORDS
setting that simplifies getting started with a basic Docker image on your computer.For details, refer to
opendj/samples/docker/README.md
.
Security
-
DS directory and proxy servers now allow access to the root DSE operational attribute
subSchemaSubEntry
. This attribute indicates the entry holding the LDAP schema definitions.Many applications retrieve this attribute, and the associated schema, to properly display or validate attribute values. If you cannot upgrade yet, update the configuration of your DS server to grant all users read access to
subSchemaSubEntry
at least on the root DSE:-
For DS 7 directory servers, add
subSchemaSubEntry
to the attribute list in the "User-Visible Root DSE Operational Attributes" global ACI. -
For DS 7 directory proxy servers, add
allowed-attribute:subSchemaSubEntry
on theRoot DSE access
configuration object.
For details on granting access to
subSchemaSubEntry
on entries in directory data, refer to ACI: Access SubSchemaSubEntry Attribute. -
-
DS servers now support text-based Privacy-Enhanced Mail (PEM) keys and certificates for server key pairs, master keys, and trusted certificates.
For details, refer to Use PEM-Format Keys.
-
The DS
fingerprint-certificate-mapper
now also supports fingerprints without colons.For example, the following SHA-256 fingerprints are equivalent:
-
0555BDA5E14C35A6A54E78DD3EFDEA5A665DE0DC9CC5187EE9CAA91ECD874B78
-
05:55:BD:A5:E1:4C:35:A6:A5:4E:78:DD:3E:FD:EA:5A:66:5D:E0:DC:9C:C5:18:7E:E9:CA:A9:1E:CD:87:4B:78
-
Tools
-
DS command options that have secrets as arguments now support
:env
and:file
modifier suffixes.For example, if the bind password is stored in a
~/.pass
file, use--bindPassword:file ~/.pass
. If the password is stored in the environment variablePASS
, use--bindPassword:env PASS
.Use the modifiers with the following options to provide the secret in an environment variable (
:env
), or in a file (:file
):-
--bindPassword[:env|:file]
-
--deploymentKeyPassword[:env|:file]
-
--keyStorePassword[:env|:file]
-
--monitorUserPassword[:env|:file]
-
--rootUserPassword[:env|:file]
-
--set[:env|:file]
(for setup profile parameters) -
--trustStorePassword[:env|:file]
-
-
The
supportextract
command now uses thejcmd
command, if available, for heap dumps. Otherwise, it uses thejmap
command. -
The
addrate
,authrate
,modrate
, andsearchrate
commands now include connection time as part of the response time for a request.
DS 7.0
DS 7.0.2
There are no new features in DS 7.0.2, only bug fixes.
DS 7.0.2 is the latest release targeted for DS 7.0.x deployments and can be downloaded from the download page.
The release can be deployed as an initial deployment or updated from an existing DS 7.0.x deployment.
DS 7.0.1
-
The DS password synchronization plugin for IDM now supports OAuth 2.0 access token bearer authentication.
For details, refer to Synchronize Passwords With ForgeRock Directory Services (DS) in the IDM documentation.
-
DS command options that have secrets as arguments now support
:env
and:file
modifier suffixes. Use these with the following options to provide the secret in an environment variable (:env
), or in a file (:file
):--bindPassword[:env|:file]
--deploymentKeyPassword[:env|:file]
--keyStorePassword[:env|:file]
--monitorUserPassword[:env|:file]
--rootUserPassword[:env|:file]
--set[:env|:file]
(for setup profile parameters)
--trustStorePassword[:env|:file]
For example, if the bind password is stored in a
~/.pass
file, use--bindPassword:file ~/.pass
. If the password is stored in the environment variablePASS
, use--bindPassword:env PASS
. -
The
supportextract
command now uses thejcmd
command, if available, for heap dumps. Otherwise, it uses thejmap
command. (Issue: OPENDJ-7662)
DS 7.0.0
Access control
Aliases for controls and extended operations
DS 7.0.0 supports use of aliases in addition to OIDs for LDAP controls and extended operations in ACIs, making those ACIs significantly more human-readable. For details, refer to Directory server ACIs.
Since previous releases support only OIDs, only use aliases in ACIs after upgrading all directory servers. Otherwise, older servers will log warning messages for the unrecognized aliases, such as the following:
Access Control Instruction (ACI) targetcontrol expression value "value" is invalid.
A valid targetcontrol keyword expression value requires one or more valid control OID strings in the following format:
oid [|| oid1] ... [|| oidN]
Authentication
Multiple identity mappers
The following configuration objects can now reference multiple identity mappers:
-
CRAM-MD5 SASL mechanism handler
-
DIGEST-MD5 SASL mechanism handler
-
GSSAPI SASL mechanism handler
-
HTTP Basic authorization mechanism
-
HTTP OAuth2 authorization mechanism
-
Password modify extended operation handlers
-
PLAIN SASL mechanism handler
-
Global configuration,
proxied-authorization-identity-mapper
When resolving the identity, the server uses the first identity mapper that finds a match. If multiple identity mappers match different entries, however, then the server returns LDAP error code 19, Constraint Violation.
For background information, refer to Identity mappers.
Backup and restore
New, simplified implementation with cloud storage support
The release provides a new, simplified implementation for DS backup and restore operations:
-
The new implementation replaces backup archives with collections of backup files.
The collection includes backend files and backup metadata. The files always follow the same layout, regardless of what you back up.
You manage backup files by retaining an entire backup directory. You are no longer required to use a separate backup directory for each backend.
-
You can now stream backup files directly to cloud storage, and restore directly from cloud storage.
-
You no longer have to make a choice between full and incremental backup operations. Backup operations are incremental by design. When you reuse the same backup directory, the process only backs up new data.
-
The new implementation includes a
purge
subcommand for removing old backup files. You can purge old files either as an external command, or as a server task. -
In the event of a disaster, you can restore from a backup directory stored off-site using only the deployment key and password, and a backup copy of the server configurations.
The new implementation protects (encrypts) the backup encryption keys with the shared master key. It stores the encrypted encryption keys in the backup files.
You no longer need to configure replication between new replicas and a server from the existing topology. Instead, you first set up replacement replicas with the deployment key and password, restoring the backed up server configurations to match those servers lost in the disaster. You then restore the data using the off-site backup directory.
-
The new implementation always signs and verifies the integrity of backups, and always encrypts backup files.
The new implementation encrypts the keys used for signing and encryption with the shared master key. It stores the encrypted keys in the backup files.
You can verify the integrity and ability to decrypt backups before restoring a backend.
-
The new implementation makes it possible to list and verify backups while the server is online.
-
The new implementation improves restore performance compared to restore of incremental backups in previous versions.
The previous implementation restored files from the full backup archive, and then restored files from each incremental backup archive. Files could be restored and then removed, or restored multiple times.
The new implementation only restores one version of each file in the backup directory.
-
A new command,
dsbackup
, replaces thebackup
andrestore
commands.The
dsbackup
command performs operations formerly performed using separate commands:-
dsbackup create
performs backup operations. -
dsbackup list
displays a summary of available backups, and lets you verify them. -
dsbackup purge
removes old backup files. -
dsbackup restore
performs restore operations.
-
-
The new
dsbackup restore
command has a--backendName
option, which lets you restore only the specified backend.
For examples, refer to Backup and restore.
Cloud deployments
Ready for Docker and Kubernetes
DS 7.0.0 lifts restrictions on running DS servers in Docker and Kubernetes deployments. Many individual improvements make this possible, including the following:
-
Replication improvements let you scale the number of DS replicas in your stateful sets up and down.
-
The new
dsrepl
command runs well in Docker containers.
ForgeRock supports customers deploying DS in Docker containers and Kubernetes platforms.
To get started, try the following:
-
Use the
forgeops
repository and the unsupported, evaluation-only base images for the Ping Identity Platform. The images are available in ForgeRock’s public Docker registry. -
Build your own sample DS Docker image.
Unpack the .zip distribution, then refer to the
opendj/samples/docker/README.md
file for instructions.
Data storage
Shared database cache by default
By default, DS servers now share cache memory among JE database backends. The server keeps JE database internal and leaf nodes in the database heap cache.
For existing servers, the upgrade
command does not change the database cache behavior.
Consider setting the global property je-backend-shared-cache-enabled:true
,
and the JE backends' properties db-cache-mode:cache-ln
after upgrade.
For details, read about the following:
-
Database cache settings
-
Java settings
-
je-backend-shared-cache-enabled
-
db-cache-mode
Newer JE
DS 7.0.0 upgrades JE backend databases to Berkeley DB Java Edition 18.3.12.
Different DS server versions continue to replicate data during the upgrade process. However, the JE upgrade has the following implications for the portability of local DS data. Once you upgrade the data in a JE backend database:
|
In addition, several JE backend properties that affect cache sizing and database maintenance can now be changed at runtime without restarting the backend. For details, refer to JE backend.
Data encryption
Portable encrypted data
DS servers now store symmetric keys, encrypted with the shared master key, with the data they encrypt.
It is no longer necessary for disaster recovery to maintain a file system backup of a server from each replication topology in your deployment. It is now sufficient to keep the backup directory and a means to recover the shared master key. As long as a server has the same shared master key as the server that encrypted the data, it can recover symmetric keys needed to decrypt data.
Be aware that this feature is new, and not provided in previous versions of DS software. Replication is fully compatible with previous server versions, but backup files are not. For this feature to work, you must use a backup from an upgraded or new server.
GCM with AES
DS directory servers now support Galois/Counter Mode (GCM) with AES for encrypted data confidentiality. GCM is efficient and improves integrity protection for encrypted backend data.
Set the data encryption cipher transformation, as described in Data encryption.
The default setting for the backend property cipher-transformation
is now AES/GCM/NoPadding
.
Logging
Field whitelisting
ForgeRock Common Audit loggers now whitelist all fields that are safe to log by default. The whitelist is processed before the blacklist, so blacklist settings overwrite the whitelist defaults.
For details, refer to Allow log message fields.
Error messages to standard output
DS servers can now send error messages to standard output.
For details, refer to Log errors to standard output.
More information about operations in access logs
DS servers now record additional information about LDAP operations in access log messages:
-
For LDAP bind operations, the security strength factor (SSF) negotiated for secure client connections appears in the
response
field of the access log message:{..."request":{"protocol":"LDAPS","operation":"BIND"...}..."response":{..."additionalItems":"ssf=128"},...}
-
For persistent searches, the log messages include
"additionalItems":"persistent"
.
Monitoring
Monitor account replicated by default
DS servers now replicate the monitor user created at setup time (default DN: uid=monitor
).
This lets commands like dsrepl status
use the same account credentials
to retrieve monitoring information from all servers.
You can use the account in the same way for multi-server monitoring operations.
Networking
Advertised listen address
DS servers now have a new, required, global property,
advertised-listen-address.
This setting specifies the hostname or IP address that clients should use for connecting to the server.
The advertised-listen-address
can be multi-valued in systems with multiple network interfaces.
DS servers also now have a global property,
listen-address.
The listen-address
property can be set to the wildcard IP address, 0.0.0.0
,
but the advertised-listen-address
property cannot.
By default, replication and connection handlers inherit their settings for listen addresses
from these global properties.
This improvement lets DS servers make fewer DNS requests than before.
When setting up a new server, the setup
command sets the advertised-listen-address
property
to the IP address or the FQDN provided as the --hostname
argument.
During upgrade, the value for the advertised-listen-address
property is assigned
using the hostname derived from administrative data under cn=admin data
.
If any listen-address
properties are set to the same value, then those settings are removed during upgrade,
and the values are inherited instead.
Passwords
SCRAM SASL support
DS software now supports Salted Challenge Response Authentication Mechanism (SCRAM) SASL binds.
A SASL SCRAM mechanism provides a secure alternative to transmitting plaintext passwords during binds. It is an appropriate replacement for DIGEST-MD5 and CRAM-MD5.
With a SCRAM SASL bind, the client must demonstrate proof that it has the original plaintext password. During the SASL bind, the client must perform computationally intensive processing to prove that it has the plaintext password. This computation is like what the server performs for PBKDF2, but the password is not communicated during the bind.
Once the server has stored the password, the client pays the computational cost to perform the bind. The server only pays a high computational cost when the password is updated, for example, when an entry with a password is added or during a password modify operation. A SASL SCRAM mechanism therefore offers a way to offload the high computational cost of secure password storage to client applications during authentication.
Passwords storage using a SCRAM storage scheme is compatible with simple binds and SASL PLAIN binds. When a password is stored using a SCRAM storage scheme, the server pays the computational cost to perform the bind during a simple bind or SASL PLAIN bind.
The SCRAM password storage scheme must match the SASL SCRAM mechanism used for authentication. In other words, SASL SCRAM-SHA-256 requires a SCRAM-SHA-256 password storage scheme. SASL SCRAM-SHA-512 requires a SCRAM-SHA-512 password storage scheme.
DS software offers the following in the configuration for new servers:
Password Storage Scheme | SASL Mechanism |
---|---|
|
|
|
|
For additional information, refer to Password storage for the server, and LDAP connection factories for the REST to LDAP gateway.
Full-featured, replicated password policies
DS servers now support LDAP subentry password policies that match all features available in per-server password policies.
Servers store subentry policies in the directory data, and therefore replicate them. This improvement significantly simplifies password policy management across multiple replicas.
For details, refer to DS subentry password policies. Many samples in the documentation now demonstrate features of the improved subentry password policies.
Stronger password storage schemes
DS servers now support additional password storage schemes, PBKDF2-HMAC-SHA256
and PBKDF2-HMAC-SHA512
.
The new password storage schemes use SHA-256 and SHA-512 hash-based message authentication code settings.
The PBKDF2
password storage scheme uses SHA-1.
To migrate passwords to a new storage scheme, refer to Deprecate a password storage scheme.
128-bit salt
Salted hashed password storage schemes now use 128-bit salt when generating a hash.
This change applies to the following password storage schemes:
PBKDF2
PBKDF2-HMAC-SHA256
PBKDF2-HMAC-SHA512
Salted MD5
Salted SHA-1
Salted SHA-256
Salted SHA-384
Salted SHA-512
Rehash passwords
You can now configure BCrypt and PBKDF2-based password storage schemes to recalculate password hashes after the iterations settings are changed. DS servers recalculate and store an account’s password hash when the user binds successfully with their password.
For details regarding BCrypt and PBKDF2-based schemes, refer to the reference for the property rehash-policy
.
Password quality advice
DS servers support a new control to request password quality advice when changing a password. Should the request fail due to low password quality, the response control indicates which password validator settings led to the failure.
The ldappasswordmodify
and ldapmodify
commands support the new control.
Use them to test and debug password policy validation settings.
The new LDAP control has interface stability Evolving. It may be removed in a future release, or replaced with a more general mechanism.
For details, refer to Check password quality (LDAP), and Check password quality (REST).
Performance
Proxy
Mutual TLS with LDAP servers
The setup
command now lets a proxy backend bind to remote servers with mutual TLS.
The setup profile for a proxy server configures the server to use mutual TLS to authenticate
when binding to backend servers.
As a result, you must provision the key manager for the proxy with the proxy service account keys,
and include the certificate in the proxy user account when using the DS proxy server setup profile.
For details, refer to Install a directory proxy.
REST
Path references
REST to LDAP mappings now support references by resource paths, simplifying access to all resource fields. REST clients can use this to issue graph-like queries. For example, the following path and query filter returns the groups that Babs Jensen’s manager belongs to:
/users/bjensen?_fields=/manager/group
For an example, refer to Graph-like queries.
To demonstrate this feature, the sample REST to LDAP mapping now uses resource paths. The configuration is simpler than the configuration with base DN references.
For example, this excerpt shows a manager reference from the version that uses a base DN:
{
"manager": {
"type": "reference",
"ldapAttribute": "manager",
"baseDn": "..",
"primaryKey": "uid",
"mapper": {
"type": "object",
"properties": {
"_id": {
"type": "simple",
"ldapAttribute": "uid",
"isRequired": true
},
"displayName": {
"type": "simple",
"ldapAttribute": "cn",
"writability": "readOnlyDiscardWrites"
}
}
}
}
}
The same manager reference using a resource path now looks like this:
{
"manager": {
"type": "reference",
"resourcePath": ".."
}
}
The latter definition ensures access to all fields defined for the referenced resource.
Reverse references
REST to LDAP mappings now support reverse references.
Reverse references are similar to the isMemberOf
LDAP attribute used for groups.
For example, use a reference mapping to list a user’s devices, or to list a manager’s reports:
{
"reports": {
"type": "reverseReference",
"resourcePath": "..",
"propertyName": "manager"
}
}
For an example in context, refer to Reverse references.
Password quality advice
REST to LDAP now supports passwordQualityAdvice
and dryRun
query string parameters.
The passwordQualityAdvice
parameter relies on the DS LDAP password quality advice control,
OID 1.3.6.1.4.1.36733.2.1.5.5
, which users must have access to request.
The dryRun
parameter relies on the LDAP no-op control, OID 1.3.6.1.4.1.4203.1.10.2
.
The password quality advice control and the passwordQualityAdvice
parameter have interface stability: Evolving.
They may be removed in a future release, or replaced with a more general mechanism.
For details, refer to Check password quality.
Account usability support
REST to LDAP now includes an accountUsability
action.
For details, refer to Account usability action.
Replication
Replication at setup time
The setup
command now lets you configure replication at setup time.
You no longer need to get all peer servers running before configuring replication.
The server begins replicating with peer servers when it comes online, and when it can contact the peers.
For this reason, the setup
command no longer starts the server by default.
To ensure replication proceeds smoothly from the beginning,
finish configuring the server before starting it for the first time.
These new setup
command options enable replication:
-
When you set the
-r, --replicationPort
option, the server runs a replication service and maintains a changelog.If you add local application data at setup time, the server replicates the data with other replicas. There is no need to configure and initialize replication separately.
-
When you set the
--bootstrapReplicationServer
option, the server contacts the specified replication server(s) to discover peer replicas and replication servers. This option is required when replicating between multiple servers.Use this option multiple times to specify redundant bootstrap servers for availability. Specify the same list of bootstrap servers each time you set up a replica.
Your first bootstrap server(s) must have replication ports, because the first bootstrap server(s) must play the replication server role.
For examples, refer to Installation.
New command to manage replication
After configuring servers to replicate as part of the setup process,
use the new dsrepl
command to manage replication.
For details, refer to Replication.
String-based server IDs
DS 7.0.0 lets you set server IDs to alphanumeric strings, such as ds1-us-west
.
When you set a server ID, take care to choose a relatively short string.
The server ID appears in historical data values that include a
change sequence number.
For example, it shows up in monitoring metrics,
and in the values of ds-sync-state
and ds-sync-hist
attributes in application data on DS replicas.
As a result, historical data is potentially easier to interpret,
but larger than in previous versions where server IDs were numbers.
One server ID per server
Servers are now identified by a single, global server ID.
For details, refer to server-id
.
For new servers, use the setup
command to specify the server ID, or accept the generated default string.
For existing servers, the upgrade
command derives the ID in the following way:
Otherwise, the command uses the first server ID found in cn=admin data
.
Other server ID values are no longer used.
One group ID per server
Servers now have a single, global group ID.
For details, refer to group-id
.
For existing servers with group IDs, the upgrade
command determines which ID is used most,
and uses that ID as the single, global ID.
Replication delay metrics
DS 7.0.0 introduces replication receive delay and replay delay monitoring metrics. These metrics provide the best means yet to help you estimate whether the data in your directory server replicas is converging toward a consistent state.
For details, refer to Replication delay (LDAP), or Replication delay (Prometheus).
Replay performance
DS 7.0.0 improves replication replay performance, reduces disk space used by the replication changelog database, and reduces replication delay in deployments under extreme load.
Replication of offline LDIF changes
Servers now replicate changes made offline to an LDIF backend. The server replicates the offline changes once it starts again.
Automatic purge of stale replicas
DS 7.0.0 purges out-of-date replicas from the changelog. The replica is purged when it has been out of contact for longer than the replication purge delay.
This enables DS servers to eventually discard information about replicas that you have removed from service, for example.
You can also use the dsrepl purge-meta-data
to eliminate stale historical data.
For details, refer to Manual purge.
Exclude domains from changelog indexes
DS 7.0.0 introduces a new replication server property to exclude domains from the changelog indexes,
changelog-enabled-excluded-domains
.
Use this to prevent applications that read the external change log
from having to process update notifications for entries that are not relevant to them.
This property eliminates the need for a separate external changelog domain configuration.
For an example, refer to Exclude a domain.
Samples
Security
Secure by default
Default settings for new DS servers are more secure than before.
The explicit --productionMode
option has been removed,
as server configurations and profiles are now secure by default.
New server installations require:
- Secure connections
-
All operations except bind requests and StartTLS requests, and base object searches on the root DSE, require secure connections.
This behavior is governed by the global configuration property,
unauthenticated-requests-policy
, which is now set toallow-discovery
, instead ofallow
, unless the last setup profile applied is theds-evaluation
profile.For details on securing connections, refer to Secure connections.
- Authentication
-
By default, servers deny anonymous access to most LDAP operations, controls, and extended operations.
For details on access control, refer to Access control.
- Additional access policies
-
By default, servers deny access to directory data. You must configure access policies to grant access to directory data. For details on granting access, refer to Access control.
Only the evaluation setup profile is more lenient. It grants global permission to perform operations over insecure connections, and open access to sample Example.com data. For details, refer to Install DS for evaluation.
- Stronger passwords
-
Passwords must have at least 8 characters. Common passwords are rejected.
For details on changing password policy, refer to Passwords.
- Permission to read log files
-
Log files are now read/write only by the DS server user.
For details on log file permissions, refer to File permissions.
As the upgrade process preserves the existing configuration, upgraded servers are not affected.
Simple private PKI
The setup
and dskeymgr
commands
simplify creation and management of a public key infrastructure (PKI).
DS 7.0.0 introduces the concept of a deployment ID and deployment ID password. The deployment ID and password serve as an alternative to a private CA, simplifying evaluation, development, and testing, and managing directory services. They also serve to derive a shared master key to protect secret keys. The deployment ID and password are required as part of the setup process. For details, refer to Cryptographic keys.
When you use an existing CA, you can continue to use key pairs with CA-signed certificates.
For public-facing directory services, you can continue to configure connection handlers with additional key and trust manager providers using certificates signed by a well-known CA. For details, refer to Key management.
To manage deployment keys, key pairs, CA certificates, and master keys after setting up a server,
use the dskeymgr
command.
Many examples in the documentation now demonstrate use of deployment IDs and passwords.
Keystores reload without restart
DS servers now reload file-based keystores and truststores when their contents change.
This lets you rotate certificates and keys without restarting the key manager or trust manager components.
Simple key rotation
DS 7.0.0 greatly simplifies rotating the key pairs used to secure replication connections. By default, replication now uses the same keys as the other connection handlers.
For details on changing key pairs, refer to Key management.
Alternative PKCS#11 types
PKCS#11 key managers and trust managers now let you set the keystore or truststore type.
The default type is PKCS11
.
If your JVM supports other types, set the keystore or truststore type with one of the following properties:
-
key-store-type
-
trust-store-type
Multiple trust managers
The following configuration objects can now reference multiple objects:
-
Administration connector
-
HTTP connection handler
-
LDAP connection handler
Use this feature to allow trust for both well-known CAs whose certificates are stored in the JVM truststore, and internal or deployment-specific CAs whose certificates are stored in a separate truststore.
Multiple certificate mappers
An external SASL mechanism handler can now reference multiple certificate-mapper
configurations.
The server uses the first certificate mapper that finds a successful match.
Indexes for certificate attributes
When you create a user data backend using the ds-user-data
setup profile,
the setup process now configures equality indexes
for the ds-certificate-fingerprint
and ds-certificate-subject-dn
attributes.
Certificate mappers use these indexes during certificate-based authentication.
Richer access log messages
DS servers now record additional items in access log messages when multiple password policy subentries apply to a user. The messages are logged only for bind, add, and modify operations. The messages show the DN of the user having more than one applicable policy, and the DN of the policy the server actually used for the operation.
The server logs a message such as the following for a bind request with two conflicting policies:
"additionalItems":{"pwdpolicywarning":"Found 2 conflicting password policy subentries for user <user-dn>, used <policy-dn>","ssf":"0"}
As described in Assign password policies, you must not assign more than one password policy to the same account.
Tools
New setup-profile command
A new command, setup-profile
, enables configuration of setup profiles following initial installation.
Use the setup-profile
command when the server is offline.
This command is intended for use in DevOps deployments where you apply additional configuration to a base image that is the same for all deployments.
If you have changes that apply to each server you set up, you can create and maintain your own setup profile. For details, refer to Setup profiles.
Configurable domains and base DNs
All setup
command profiles, except the ds-evaluation
profile,
now allow you to set the domain or the base DN.
For details, refer to Setup profiles.
Generate systemd service files
The create-rc-script
command now produces a systemd
service file
when you use the --systemdService
option.
Generate user entries for evaluation
The ds-evaluation
setup profile now lets you generate an arbitrarily large number of similar user entries.
By default, the profile adds 100,000 generated users in addition to users previously included,
such as Babs Jensen and Kirsten Vaughan.
Each user entry has a uid
RDN like user.number
. Each user entry’s password is password
.
The capability replaces the setup
command option -d, --sampleData
.
Proxied authorization for rate tools
The addrate
, modrate
, and searchrate
commands now support
proxied authorization with the -Y, --proxyAs {authzID}
option.
Support for formatted integers
Formatted integers can now be supplied to some integer arguments, making commands easier to read.
When setting the number of generated sample entries as an argument to the setup
command,
and when setting integer arguments for the addrate
, authrate
, modrate
,
and searchrate
commands,
you can now use formatted integers.
For example, the following are equivalent to 10000000
:
10,000,000
10.000.000
"10'000'000"
10_000_000
"10 000 000"
Templates for the makeldif
command can also accept formatted integers
for numbers declared in a subordinate template.
Task management
The manage-tasks
command now has --status
and --type
options.
When used with the --summary
option,
these options filter the list to include only tasks of the specified type and status.
The option arguments are case-insensitive, and must be provided in the JVM locale.
For example, to list only unscheduled tasks on a JVM with the French locale,
use --status "non planifié"
instead of --status unscheduled
.
Set task IDs and descriptions
When you schedule a task, you can now set its identifier with the --taskId
option,
and its description with the --description
option.
The identifiers and descriptions appear in output and messages that describe the task.
These new options are especially useful for recurring tasks. Use the task identifier when managing the task in subsequent commands, for example.
No changes when reading JE backends offline
The following tools now never write to JE backend databases when reading JE information:
-
backendstat
-
export-ldif
-
verify-index
Status output improvements
The status
command now displays the same types of information independently of the server configuration,
and regardless of whether the command runs in online or offline mode.
The command still displays more detailed information in online mode than in offline mode.
Supportextract improvements
-
The
supportextract
command now also collects:-
The directory superuser and monitor user account files.
-
The archived configuration files.
-
The profile and backend database version files.
-
Information about the changelog database.
-
The
server.out
log file before capturing stack traces. -
The server PID in a message in the tool’s log.
-
The
cpuinfo
,meminfo
,slabinfo
, andbuddyinfo
files on Linux systems. -
Stack traces with
jcmd
tool, falling back to thejstack
tool, and then tosigquit
(orkill -3
on Linux) as necessary. -
Environment variables used in configuration expressions.
-
-
The extract generated by the tool is now compatible with the Java 11 JVM unified logging framework.
DS 6.5.6
DS 6.5.6 is the final release targeted for DS 6.5.x deployments and can be downloaded from the download page.
DS 6.5.6 can be deployed as an initial deployment or updated from an existing DS 6.5.x deployment.
-
The
supportextract
command now collects environment variables used in configuration expressions.
DS 6.5.5
-
DS servers now more effectively calculate reservable memory when using G1 garbage collection, and reduce the risk of long
fsync
pauses.This change introduces a
ds-mon-db-cache-size-total
metric to track the maximum size of the database cache. It also changes theds-mon-db-log-size-active
metric to reflect only live data. -
The
supportextract
command now uses thejcmd
command, if available, for heap dumps. Otherwise, it uses thejmap
command.
DS 6.5.3
DS servers now support storing the ads-certificate
key pair
and peer DS servers' trusted public keys in a PKCS#11 module, such as a hardware security module (HSM).
This means you can store the keys used to secure replication traffic and to protect symmetric keys in an HSM.
Previously, DS servers supported use of a PKCS#11 module
only to store the keys used to secure other communications.
DS servers support PKCS#11 modules through the JVM. How to configure the JVM to allow DS servers to use your module, how to generate keys in the module, and how to export and import public key certificates all depend on your specific HSM/PKCS#11 module. For details on how to perform such actions, refer to your PKCS#11 module’s documentation. |
Before trying this feature, perform these tests:
-
Test that the PKCS#11 module supports multiple aliases for the same certificate.
-
Test that the PKCS#11 module supports generating a key pair with an RSA self-signed certificate and a key size of 2048 bits.
-
Test the replication topology using the default JKS
ads-truststore
implementation.Verify that replication functions properly before using the PKCS#11 module.
After validating the results of the tests, use the new feature by performing these steps for each server:
-
Configure JVM security to enable use of the PKCS#11 module for the server.
-
Generate a key pair using an RSA self-signed certificate and a key size of 2048 bits with the alias
ads-certificate
on the PKCS#11 module. -
After generating the key pair:
-
Export the
ads-certificate
certificate, and reimport it on the PKCS#11 module using the MD5 fingerprint as the certificate alias.For example, if the
ads-certificate
certificate MD5 fingerprint is07:35:80:D8:F3:CE:E1:39:9C:D0:73:DB:6C:FA:CC:1C
, reimport the certificate with the alias073580D8F3CEE1399CD073DB6CFACC1C
. -
Prepare LDIF to update
cn=admin data
for the new certificate.The LDIF adds the new certificate as an instance key, and sets the key ID for the current server to use the key. In the following example LDIF, the certificate alias is
073580D8F3CEE1399CD073DB6CFACC1C
, and the server hostname:_admin-port_ combination isopendj.example.com:4444
:dn: ds-cfg-key-id=073580D8F3CEE1399CD073DB6CFACC1C,cn=instance keys, cn=admin data changetype: add ds-cfg-key-id: 073580D8F3CEE1399CD073DB6CFACC1C ds-cfg-public-key-certificate;binary:: MIIB6zCCAVSgAwIBAgIEDKSUFjANBgkqhkiG9w0BA QUFADA6MRswGQYDVQQKExJPcGVuREogQ2VydGlmaWNhdGUxGzAZBgNVBAMTEm9wZW5hbS5leGFtcGxl LmNvbTAeFw0xMzAyMDcxMDMwMzNaFw0zMzAyMDIxMDMwMzNaMDoxGzAZBgNVBAoTEk9wZW5ESiBDZXJ 0aWZpY2F0ZTEbMBkGA1UEAxMSb3BlbmFtLmV4YW1wbGUuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNAD CBiQKBgQCfGLAiUOz4sC8CM9T5DPTk9V9ErNC8N59XwBt1aN7UjhQl4/JZZsetubtUrZBLS9cRrnYdZ cpFgLQNEmXifS+PdZ0DJkaLNFmd8ZX0spX8++fb4SkkggkmNRmi1fccDQ/DHMlwl7kk884lXummrzcD GbZ7p4vnY7y7GmD1vZSP+wIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAJciUzUP8T8A9VV6dQB0SYCNG1o 7IvpE7jGVZh6KvM0m5sBNX3wPbTVJQNij3TDm8nx6yhi6DUkpiAZfz/OBL5k+WSw80TjpIZ2+klhP1s srsST4Um4fHzDZXOXHR6NM83XxZBsR6MazYecL8CiGwnYW2AeBapzbAnGn1J831q1q objectClass: top objectClass: ds-cfg-instance-key dn: cn=opendj.example.com:4444,cn=Servers,cn=admin data changetype: modify replace: ds-cfg-key-id ds-cfg-key-id: 073580D8F3CEE1399CD073DB6CFACC1C
Do not yet update
cn=admin data
. You must not change the server key ID until the server is ready to use the PKCS#11 module.
-
-
Edit the
ads-truststore
trust manager provider configuration to access the PKCS#11 module with the following properties:trust-store-file
-
Leave this unchanged.
trust-store-pin
-
Set this to the PIN for the PKCS#11 module.
You can avoid exposing secrets in the configuration by using expressions. For details, refer to Property value substitution.
trust-store-type
-
Set this to
PKCS11
.
-
Stop the DS server.
-
Using the
ldifmodify
command while the server is stopped, updatecn=admin data
for the server with the LDIF you prepared.This step changes the key ID for the server, letting it use its keys held in the PKCS#11 module.
-
On another, running server replica, use the
ldapmodify
command to updatecn=admin data
for the server with the LDIF you prepared.Replication propagates the change to the other running server replicas.
This step changes the key ID for the server, letting other servers trust its new certificate once it restarts.
-
Restart the DS server.
DS 6.5.0
Connection limiting
DS servers now allow you to limit the number of concurrent connections per client.
For details, refer to Connection limits.
Data distribution
DS proxy servers now support simple, non-elastic data distribution.
You can configure a proxy server to equitably distribute LDAP write requests across multiple replication partitions to scale the directory service horizontally. As the present implementation does not permit elastic scaling or data redistribution, make sure that you understand the documented constraints of the present implementation before deploying it in production.
For details, refer to Data distribution.
Database caching
A new DS directory server uses shared cache by default for all JE database backends.
As a result, you are no longer required to set the database cache size
using the db-cache-percent
or db-cache-size
setting for each backend.
It remains possible to use these settings if necessary by configuring them appropriately.
Logging
-
DS servers now support sending access log messages to standard output.
For details, refer to Log access to standard output.
When the new handler is used, standard output from the server includes a mix of JSON for access messages and non-JSON DS-format server event messages.
-
Common audit logging now supports denying log message fields to prevent them from showing up in log messages.
For an example, refer to Deny log message fields.
-
Common audit logging now supports writing multiple file-based logs to the same directory by setting a different
log-file-name-prefix
for each file-based log.
Monitoring
DS servers now provide health status checks for anonymous requests over HTTP and LDAP. This allows a remote application to check that a server is "alive" and "healthy".
Anonymous HTTP requests can retrieve "alive" and "healthy" status codes. Anonymous LDAP requests can retrieve "alive" and "healthy" boolean values.
The "alive" and "healthy" status indicates that the server has passed its own internal tests. It is not, however, a guarantee that the server is free from other errors. If a server is not "alive," it requires administrative intervention. If a server is not "healthy," temporarily route requests to another server.
When a server is not "alive" or "healthy," a user with privileges to read monitoring information receives health status error messages in the body of the HTTP response, and can obtain health status error messages over LDAP. No error messages are returned in response to anonymous requests.
For examples demonstrating how to use this feature, refer to Monitoring.
When you upgrade DS servers to 6.5.0, the anonymously accessible HTTP endpoints are not configured.
To add the endpoints on an upgraded server that lacks them, use the dsconfig
command:
$ dsconfig \
create-http-endpoint \
--endpoint-name /alive \
--type alive-endpoint \
--set enabled:true \
--set authorization-mechanism:HTTP Anonymous \
--set authorization-mechanism:HTTP Basic \
--hostname opendj.example.com \
--port 4444 \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--trustAll \
--no-prompt
$ dsconfig \
create-http-endpoint \
--endpoint-name /healthy \
--type healthy-endpoint \
--set enabled:true \
--set authorization-mechanism:HTTP Anonymous \
--set authorization-mechanism:HTTP Basic \
--hostname opendj.example.com \
--port 4444 \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--trustAll \
--no-prompt
Platform Integration
When setting up a directory server for use with other Ping Identity Platform component products, you can use available setup profiles to greatly simplify initial configuration.
For details, refer to Setup profiles.
DS 6.0.0
Backend database storage
-
Directory servers now use an optimized JE backend implementation whose dependencies are distributed under the Apache License, Version 2.0. This license is suitable for all deployments, including OEM deployments.
Support for PDB backend databases has been removed in 6.0.0.
Before upgrading a directory server using any PDB backends, take one of the following actions:
-
Move the data to JE backend databases before upgrading.
-
Export all data in PDB backend databases to LDIF before upgrading, and import the data into the JE backend databases with the same names after upgrading.
-
-
Backend indexes now have new time to live (TTL) properties to configure automated, optimized entry expiration and removal:
-
ttl-enabled
-
ttl-age
For details on how to use this feature, refer to Entry expiration.
-
-
DS 6.0.0 upgrades JE backend databases to Berkeley DB Java Edition 7.5.11.
-
DS 6.0.0 improves many JE backend settings:
-
A new advanced property,
db-durability
, makes the backend durability setting easier to configure and to read. -
New defaults for disk space thresholds better fit modern deployments.
The property,
disk-low-threshold
, defaults to 5% of the filesystem size, plus 5 GB.The property,
disk-full-threshold
, defaults to 5% of the filesystem size, plus 1 GB. -
Default limits for backend database log files better fit larger deployments.
The property,
db-log-file-max
, defaults to 1 GB instead of 100 MB.The property,
db-log-filecache-size
, defaults to 200 instead of 100.As a result, the database can grow to 200 GB instead of 10 GB on disk before the file cache begins to close database log files in order to open others.
-
The new properties,
db-run-log-verifier
, anddb-log-verifier-schedule
, make it possible to configure whether and when the server runs checksum verification on backend database logs.
-
Configuration expressions
-
Server configuration expressions have been reimplemented to align with other Ping Identity Platform software. Configuration expressions make it possible to substitute configuration property values with variables that you can set before starting DS servers.
For details, refer to Property value substitution.
Faster bulk updates
-
The
ldapmodify
andldapdelete
commands now offer a--numConnections
option to perform updates in parallel on multiple LDAP connections.This feature enables faster bulk updates, and provides an alternative to the
import-ldif --append
option removed in version 3.0.
JSON support
-
In addition to optimized indexes for JSON attribute values that are queried with common REST query filters, DS directory servers now also support optimizations for JSON with optional fields.
Monitoring
-
DS server monitoring has been reimplemented to align with other Ping Identity Platform software.
This change affects LDAP and HTTP monitoring interfaces, but not SNMP interfaces. The SNMP interface continues to use standard metrics.
Monitoring capabilities now have interface stability Stable.
For documentation about available interfaces and metrics, refer to Monitoring.
DS servers now feature the following monitoring capabilities:
-
Support for pulling monitoring data to Prometheus monitoring software.
-
Support for pushing monitoring data to a Graphite service.
-
Support for creating a directory monitoring account during setup.
When using JMX to monitor the server with this account, be aware that in 6.0.0 the account does not have JMX-related privileges. Instead, you must add the required JMX-related privileges, as described in JMX-based monitoring.
-
-
DS servers support a new privilege,
monitor-read
. This prevents unauthorized users from reading monitoring metrics unless they have the privilege.Assign this privilege to users who read monitoring metrics over LDAP or HTTP.
When upgrading, add the missing privilege to the global administrator account. These privileges are required when using the
dsreplication status
command.
Performance
-
DS 6.0.0 includes many performance improvements.
No DS server performance improvements require action on your part, though optimal tuning settings including JVM settings may now be different for specific scenarios.
When upgrading to this release, be aware that the command-line performance tools have a new template value syntax.
Replication
-
DS servers now allow you to choose a single, global server ID used when configuring replication, rather than letting replication configuration randomly assign server IDs.
Before configuring replication, you can set the global configuration property
server-id
.This makes it easier to keep track of server IDs when reviewing replication configuration, and to parameterize replication configuration in DevOps deployments.
-
DS directory servers store historical replication information for internal use in entries'
ds-sync-hist
attributes. This release introduces a new encoding that significantly reduces the space required to storeds-sync-hist
data.The space reduction trades a smaller footprint for increased CPU use when preparing to write
ds-sync-hist
values. Read and search operations should not be negatively impacted, however. Indeed, read and search performance should improve to the extent that reduced entry size means more efficient use of backend database and CPU caches. -
DS replication domains can now be disabled by setting
enabled:false
for the domain. If a replication domain is disabled, its contents are not replicated.This change facilitates parameterizing whether backends and associated replication domains are enabled, which is useful in DevOps deployments where not all environments replicate the same data on each replica.
Security
-
DS certificate mappers now support certificate issuer verification. Use this to verify the certificate issuer whenever multiple CAs are trusted in order to prevent impersonation. Different CAs can issue certificates with the same subject DN, but not with the same issuer DN.
This feature relies on the certificate mapper property,
issuer-attribute
, to identify the attribute in the user entry that holds the issuer DN. For this purpose, DS servers define the attribute,ds-certificate-issuer-dn
, as an optional attribute of theds-certificate-user
object class.For an example using this attribute, refer to Certificate-based authentication.
Server configuration
-
The
dsconfig
command now allows you to configure DS servers that are not running.DS 6.0.0 introduces an
--offline
option and a--configFile
option. When you use thedsconfig --offline
command with a DS server that is stopped, you change the server configuration file in the default location, such as/path/to/opendj/config/config.ldif
. The--configFile
option allows you to specify an alternative configuration file in offline mode.
Server-side sorting
-
JSON ordering matching rules for JSON attributes.
DS servers can implement JSON ordering matching rules on demand. This enables REST to LDAP to support for
_sortKeys
where a field is inside the value of a JSON attribute. -
An extension of the server-side sort request sort order specification.
-
A means for REST to LDAP to communicate the
_sortKeys
field that is inside the value of a JSON attribute to the DS server.This functionality is part of REST to LDAP. By using the
useServerSideSortForJson
boolean configuration parameter, you can configure whether to sort results in the REST to LDAP gateway. -
A means for the REST to LDAP gateway to limit the maximum number of entries supported by the local sort mechanism when sorting results based on JSON attributes.
The setting is
localSortMaxEntries
.
Unindexed Searches
DS directory servers now support the following improvements for unindexed searches. These improvements are designed to help applications that store data in the directory as arbitrary JSON objects, and that provide a graphical UI for browsing directory data accessed over REST. With these improvements, users can page through directory data, sorting on whichever JSON field they choose without initially specifying any filter.
As with any unindexed search that you allow, the tradeoff is inefficient use of system resources and less performance. This is not, therefore, a general capability that should be provided to all applications without taking the impact into consideration. It is intended for use by a directory data administrator who is browsing data without knowing in advance what they are looking for:
-
DS directory servers can now use an appropriately configured VLV index to sort results for an unindexed search.
-
DS directory servers sort unindexed search results as long as they are paged.
This improvement has the following limitations:
-
The simple paged results control must specify a page size that is less than or equal to the
index-entry-limit
(default: 4000). -
For each page, the server reads the entire backend database, retaining page size number of sorted entries.
-
DS 5.5.3
DS 5.5.3 is the final release targeted for DS 5.5.x deployments and can be downloaded from the download page.
DS 5.5.3 can be deployed as an initial deployment or updated from an existing DS 5.5.x deployment.
There are no new features introduced in DS 5.5.3, only bug fixes.
DS 5.5.0
Backend database storage
-
JE backend databases are upgraded to Berkeley DB Java Edition 7.4.5 in this release.
Directory proxy
-
Directory proxy servers now automatically retry operations when they detect a temporary failure on the remote directory server.
For details, refer to About failures.
-
The new proxy backend property,
heartbeat-search-request-base-dn
, lets you configure proxy backend heartbeat requests to target an entry under a base DN of interest rather than targeting the root DSE. -
When the global configuration property,
trust-transaction-ids
, is set totrue
, the proxy backend now adds a ForgeRock transaction ID control before forwarding the request, even if the incoming request did not include the control.As a result, all proxied requests have ForgeRock transaction IDs when you configure the server to trust transaction IDs.
Monitoring
-
Replication server configurations now include these advanced properties for monitoring disk space use and stopping operations when the disk is full:
disk-low-threshold
-
When this threshold is reached, the server logs warnings and sends warnings to the disk space monitoring subsystem.
The directory administrator must take action to provide more disk space.
disk-full-threshold
-
When this threshold is reached, the server stops operations and lets connected directory servers fail over to another replication server. The replication server can resume operations once free disk space rises above the
disk-low-threshold
setting.
-
Servers and the REST to LDAP gateway now include a ForgeRock transaction ID with each request.
If you do not configure the server or gateway to trust transaction IDs in client application requests, then they ignore incoming transaction IDs, and instead generate a transaction ID for each request.
If you configure the server or gateway to trust transaction IDs in client application requests, then outgoing requests reuse the incoming transaction ID. For each outgoing request in the transaction, the request’s transaction ID has the form original-transaction-id/sequence-number, where sequence-number reflects the position of the request in the series of requests for this transaction. For example, if the original-transaction-id is
abc123
, the first outgoing request has transaction IDabc123/0
, the secondabc123/1
, the thirdabc123/2
, and so on. This helps you to distinguish specific requests within a transaction when correlating audit events from multiple services.To configure a server to trust transaction IDs in client application requests, set the global configuration property,
trust-transaction-ids
, to true.To configure the REST to LDAP gateway to trust transaction IDs in client application requests, set the JVM system property,
org.forgerock.http.TrustTransactionHeader
, to true in the web application container where the gateway runs. -
When an internal search is unindexed, a directory server now logs a message.
REST to LDAP
-
REST APIs now support the
_sortKeys
parameter to request that the server sort the query results it returns. -
REST to LDAP now uses the affinity load balancer.
Security
-
DS servers use SHA-256 as the signature algorithm when generating key pairs, as an attacker with sufficient computing power could break SHA-1.
NIST Special Publication 800-131A, Transitions: Recommendation for Transitioning the Use of Cryptographic Algorithms and Key Lengths, generally disallows use of SHA-1 for digital signature generation.
-
DS servers now support using a PKCS#11 device, such as a hardware security module (HSM), as a truststore.
When you use a PKCS#11 device as a truststore, all trusted certificates must be present on the device. No CA certificates are available by default. Import all the signing certificates required for your deployment before configuring the device for use as a truststore.
To use an HSM as a truststore:
-
Configure the JVM to allow access to the PKCS#11 device.
-
Using the
dsconfig
command, create a PKCS#11 trust manager provider configuration to access the PKCS#11 device as a truststore.
-