DS 7.1.8

DS Software

A directory service provides LDAP and HTTP access to distributed, shared directory data. A deployed directory service consists of one or more components. Each component plays a particular role in your directory service. Before you design your deployment, you need to be familiar with the roles that each component can play:

Directory service component diagram

  • Directory servers, which maintain and serve requests for directory data.

    In most deployments, directory servers use data replication to ensure that their data sets eventually converge everywhere. This documentation refers to a replicated directory server as a replica.

  • Directory proxy servers that forward requests for directory data to directory servers, and return directory server responses to client applications.

  • Replication servers that transmit data replication messages among replicas.

    You can configure a directory server to act as a replication server as well. A standalone replication server plays only the replication server role, brokering replication change messages. It does not store directory data.

  • DSML gateways that intermediate between DSML client applications and an LDAP directory.

  • REST to LDAP gateways that intermediate between RESTful HTTP client applications and LDAP directories.

  • LDAP client tools and server administration tools for testing and managing servers.

Directory Servers

Directory servers have the following characteristics.

Roles

Directory servers provide access to their copy of the distributed directory database. A directory server usually functions as the repository of identities for users, applications, and things. They respond to requests from client applications directly or indirectly through directory proxy servers. This includes the following:

  • LDAP requests for authentication, reads, and updates.

    An LDAP client application authenticates with the directory server, and then performs one or more operations before either reauthenticating to reuse the connection or ending the session and closing the connection.

  • HTTP read and update requests, often including credentials for authentication.

    An HTTP request translates to one or more internal LDAP requests.

  • Administrative requests, such as requests to modify the server configuration or to perform a task such as backup or LDIF export.

  • JMX and SNMP requests specifically for monitoring information.

In deployments with multiple replicas, directory servers replay replicated operations. Expect each replica to replay every successful update to any replica.

Data

In addition to the libraries and tools delivered with the server distribution, a directory server is associated with the following persistent state information and local data:

User data

Directory servers store user data. The directory server stores the data in local storage, such as an internal disk or an attached disk array. The storage must keep pace with throughput for update operations.

The amount of user data depends entirely on the deployment, ranging from a few LDAP entries to more than a billion. The amount of user data grows or shrinks in deployment depending on the pattern of update operations.

The directory server stores user data in a backend database. For details, see Data Storage.

Metadata for replication

A directory server can be a replica of other directory servers, meaning it can hold an eventually consistent copy of the data on the other replicas. To avoid an individual server becoming a single point of failure, almost all real-world deployments depend on replication.

When serving a request to update directory data, the directory server modifies its data and sends a request to a replication server. The replication server, described in Replication Servers, ensures that all other replicas update their data to eventually reflect the current state of the data.

To tolerate network partitions, the directory service supports concurrent update operations on different replicas. Concurrent updates potentially cause conflicts, but directory servers can resolve most conflicts automatically. To resolve conflicts, a directory server stores historical metadata alongside user data, trading space for resilience. For details, see About Replication.

The directory server purges this historical metadata after a configurable interval. The volume of historical metadata depends on the total number of updates made to the directory service since the purge interval.

Server configuration

Each server has configuration data in its config directory. This includes the server configuration, mostly in LDIF format, LDAP schema definitions also in LDIF format, keys used to secure connections and perform encryption and decryption, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command modifies this configuration data to apply any necessary changes.

Log files

The server writes to multiple log files by default, including error and access logs.

The server writes a message to the current access log for each operation. For high-volume directory services, log file storage must keep pace with the requests to record access to the service.

Log file retention and rotation policies prevent log file data from filling the disk. For details, see Logging. As a result of default retention policies, messages can eventually be lost unless you copy old files to another system for permanent storage.

Backup files

When you export directory data to LDIF or create a backup, the directory server writes the files to the specified directory. If you never purge or move these files, they can eventually fill the disk.

For details, see Import and Export, and Backup and Restore.

System Resources

When deciding how to deploy a directory server, think of it as a copy of the database. A large, high-performance, distributed database serving lots of applications requires more system resources than a small database serving one, simple application.

A directory server requires the following system resources:

  • Sufficient RAM to cache frequently used data.

    For best read performance, cache the entire directory data set in memory.

  • Sufficient CPU to perform any required calculations.

    Authentication operations generally use more CPU than other operations. In particular, password storage schemes like PBKDF2 are designed to consume CPU resources. Calculations for transport layer security can use CPU as well, particularly if many client requests are over short-lived HTTPS connections.

  • Sufficient fast disk access to serve client applications, to replay replication operations, and to log access and errors.

    The underlying disk subsystem must serve enough input/output operations per second (IOPS) to avoid becoming a bottleneck when performing these operations. A small database that serves few client operations and changes relatively infrequently requires fewer IOPS than a large database sustaining many updates and serving many clients.

    Plan additional capacity for any backup or LDIF files stored on local partitions.

  • Sufficiently fast network access to serve client applications and relay replication traffic.

    When considering network requirements, keep the following points in mind:

    • Each LDAP search request can return multiple response messages.

    • Each request to update directory data results in corresponding replication traffic. The operation must be communicated to replication servers and replayed on each other directory server.

    • Once established, and unlike most HTTP connections, LDAP connections remain open until the client closes the connection, or until the server idles the connection. This is particularly true for applications using persistent searches, which by design are intended to be permanent.

Replication Servers

Standalone replication servers have the following characteristics. If you configure a directory server to play the role of a replication server as well, then the directory server also has these roles and characteristics.

Roles

Replication servers provide the following services:

  • Receive and transmit change messages between replicas.

    Each replica is connected to one replication server at a time. A single standalone replication server can serve 10 or more replicas.

  • Maintain information about all other replication servers and directory servers in the deployment that replicate the same data.

    Change messages travel from a connected directory server to the replication server. The replication server transmits the message to connected replicas. If there are other replication servers, the replication server transmits the message to the other replication servers, which in turn transmit the message to their connected replicas. This hub-and-spoke communication model means directory services can be composed of many individual servers.

  • Respond to administrative requests.

  • Respond to HTTP, JMX, LDAP, and SNMP requests for monitoring information.

In all deployments using replication, the replication service provides the foundation of directory service availability. This is as important to the directory service as a naming service is for a network. When deploying replicated directory services, start by installing the replication service.

To avoid a single point of failure, install two or more replication servers in each location.

Data

In addition to the libraries and tools delivered with the server distribution, a replication server is associated with the following persistent state information and local data:

Change data

When serving a request to update directory data, a directory server, described in Directory Servers, modifies its data and sends a request to a replication server. The replication server ensures that all other replicas update their data to eventually reflect the current state of the data.

The replication protocol is proprietary. Replication servers expose a public record of changes in a change log, allowing other applications to keep up to date with changes to user data. This change data is stored in change log files. For details, see Changelog for Notifications.

The replication server purges this historical metadata after a configurable interval. The volume of historical metadata depends on the updates made to the directory service since the purge interval.

Server configuration

Each server has configuration data in its config directory. This includes the server configuration, mostly in LDIF format, LDAP schema definitions also in LDIF format, keys used to secure connections and perform encryption and decryption, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command modifies this configuration data to apply any necessary changes.

Log files

The server writes to multiple log files by default, including error and access logs.

Log file retention and rotation policies prevent log file data from filling the disk. For details, see Logging. This means, however, that messages are eventually lost unless you move old files to another system for permanent storage.

System Resources

When deploying a replication server, keep its foundational role in mind. Directory servers communicate with other replicas through replication servers. Directory proxy servers rely on replication servers to find directory servers.

A replication server requires the following system resources:

  • Sufficient fast disk access to log and read change messages, and to update access and error logs.

    The underlying disk subsystem must serve enough IOPS to avoid becoming a bottleneck when performing these operations.

  • Sufficiently fast network access to receive and transmit change messages for multiple replicas and for each other replication server.

Directory Proxy Servers

Directory proxy servers have the following characteristics.

Roles

Directory proxy servers provide the following services:

  • Balance load of requests to LDAP directory servers.

  • Receive and transmit LDAP client requests to LDAP directory servers.

  • Receive and transmit LDAP directory server responses to LDAP client applications.

  • Respond to administrative requests.

  • Respond to HTTP, JMX, LDAP, and SNMP requests for monitoring information.

A directory proxy server can hide the underlying directory service architecture from client applications, enabling you to build a single point of directory service access.

A directory proxy server can discover directory servers through a replication server. This capability relies, however, on the replication server configuration. If you use the proxy server with third-party directory service components, then you must manually maintain the network locations for directory servers.

A directory proxy server provides LDAP access to remote LDAP directory servers. If you want to provide HTTP access to remote LDAP directory servers, use the REST to LDAP gateway instead. For details, see REST to LDAP Gateway.

Data

In addition to the libraries and tools delivered with the server distribution, a directory proxy server is associated with the following persistent state information and local data:

Server configuration

Each server has configuration data in its config directory. This includes the server configuration, mostly in LDIF format, LDAP schema definitions also in LDIF format, keys used to secure connections and perform encryption and decryption, and some additional data.

When installing a server, the setup command instantiates this configuration data from templates.

When upgrading a server, the upgrade command modifies this configuration data to apply any necessary changes.

Log files

The server writes to multiple log files by default, including error and access logs.

Log file retention and rotation policies prevent log file data from filling the disk. For details, see Logging. This means, however, that messages are eventually lost unless you move old files to another system for permanent storage.

System Resources

In order to route requests appropriately, a directory proxy server must decode incoming requests and encode ongoing requests. It must also decode and encode incoming and outgoing responses. When deploying a directory proxy server, keep this decoding and encoding in mind, because it explains why you might need as many proxy servers as directory servers.

A directory proxy server requires the following system resources:

  • Sufficient fast disk access to update access and error logs.

    The underlying disk subsystem must serve enough IOPS to avoid becoming a bottleneck when performing these operations.

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding and encoding consume CPU resources.

  • Sufficient RAM to maintain active connections.

Command-Line Tools

When you install the files for a server component, those files include tools for setting up, upgrading, and configuring and maintaining the server and administrative tasks. The files also include LDAP command-line tools for sending LDAP requests and measuring directory service performance.

For details, see Server Commands.

DSML Gateway

The standalone DSML gateway web application has the following characteristics.

You can install this component independently of directory services. For details, see Install a DSML Gateway.

Roles

DSML gateways provide the following services:

  • Receive HTTP DSML requests from client applications, and transmit them as LDAP requests to a directory service.

  • Receive LDAP responses from a directory service, and transmit them as HTTP DSML responses to client applications.

A DSML gateway runs in a Java web application container. It is limited to one host:port combination for the LDAP directory service.

Data

A DSML gateway maintains only its own service configuration, recorded in the web application WEB-INF/web.xml file. It depends on the host web application container for other services, such as logging.

System Resources

A DSML gateway requires the following system resources:

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding, encoding, and transformation all consume CPU resources.

    Calculations to secure network connections also consume CPU resources.

  • Sufficient RAM to maintain active connections.

REST to LDAP Gateway

The standalone REST to LDAP gateway web application has the following characteristics. REST refers to the representational state transfer architectural style. RESTful requests use the HTTP protocol.

You can install this component independently of directory services. For details, see Install a REST to LDAP Gateway.

Roles

REST to LDAP gateways provide the following services:

  • Receive HTTP requests from client applications, and transmit them as LDAP requests to a directory service.

  • Receive LDAP responses from a directory service, and transmit them as HTTP responses to client applications.

A REST to LDAP gateway runs in a Java web application container. It can be configured to contact multiple LDAP directory servers.

One RESTful HTTP request can generate multiple LDAP requests. This is particularly true if the REST to LDAP mapping configuration includes references to resolve before returning response entries. For example, an LDAP user entry can have a manager attribute that holds the DN of the user’s manager’s entry. Rather than return an LDAP-specific DN in the REST response, the REST to LDAP mapping is configured to return the manager’s name in the response. As a result, every time a user’s manager is returned in the response, the gateway must make a request for the user’s LDAP information and another request for the user’s manager’s name.

Data

A REST to LDAP gateway maintains only its own service configuration, recorded in files as described in REST to LDAP Reference. It depends on the host web application container for other services, such as logging.

System Resources

A REST to LDAP gateway requires the following system resources:

  • Sufficiently fast network access to receive and transmit client requests and server responses.

  • Sufficient CPU to perform any required calculations.

    Request and response decoding, encoding, and transformation all consume CPU resources.

    Calculations to secure network connections also consume CPU resources.

  • Sufficient RAM to maintain active connections.

Copyright © 2010-2024 ForgeRock, all rights reserved.