What do I need to consider when planning a migration from Oracle DSEE to DS?

Introduction

Parts of the ODSEE code base dates back to the original Netscape® Directory Server (DS). Over the years the directory server became a widely used, robust product. And yet one of the original assumptions built into ODSEE is that directory services are mainly for reads, not for writes. Furthermore the code base for ODSEE started out at a time when the LDAP standards were still very much in progress, and so does not always apply the standards strictly.

The code base that today backs DS started in the ODSEE team after it had already become clear that directories must handle as many writes as reads, today's directory data being more volatile than ever. In addition the LDAP standards were mature and consolidating, so for interoperability it made sense to follow them more strictly. Luckily, a primary goal of the project has always been relatively easy migration from ODSEE.

Today with Oracle holding its infrastructure products like ODSEE in maintenance mode, it makes sense to leave yesterday's architecture behind and build directory services with DS.

This article is not a step-by-step guide to making that change. Instead it highlights what you must consider when planning a migration from ODSEE to DS. We assume that you are already familiar with both products, and that you already have successfully managed similar projects in the past. If not, you can turn to one of ForgeRock’s partners, or you can complete the project yourself with the expectation that you will learn a lot along the way.

How ODSEE differs from DS

DS aims for compatibility with ODSEE, providing additional performance and features. There are nevertheless a number of important differences you must be aware of when moving from ODSEE to DS, including the following which are discussed in detail in subsequent sections:

• LDAPv3: ODSEE leniency versus DS compliance.
• Replication: ODSEE peer-to-peer versus DS hub-and-spoke.
• Operational attributes and password policy: different attributes, additional DS functionality.
• Data Storage: DS for portability and performance.
• ODSEE roles and CoS versus DS groups, virtual and collective attributes.
• ODSEE ACIs versus DS​​​​​​​ ACIs: no MACRO ACIs exist in DS​​​​​​​.
• PKI: ODSEE NSS versus DS​​​​​​​ Java®-based implementations.
• Code Base: C/C++ in ODSEE versus Java in DS.

LDAPv3: ODSEE leniency versus DS compliance

In some respects, ODSEE is more lenient in what it accepts versus DS.

Attribute Value Syntax Checking 

LDAP attribute values are supposed to respect the syntax defined for the attribute. This means that applications can expect the attribute values they read from the centralized directory service to conform to the syntax, and therefore can handle the values in uniform ways.

The tradeoff is that applications that update directory entries have to perform more checking and processing of their input before sending it to the directory. When a user updates a telephone number manually, the application might have to transform that number to fit telephone number syntax before sending a modify request to the directory.

In ODSEE, the decision was made to lighten the load on application developers by allowing non-conformant attribute values. This is the case for phone numbers and access control instructions (ACIs), for example.

In DS, the default settings limit garbage coming in so as to reduce garbage going out.

If you can, clean up the data, and warn application maintainers that non-conformant values are going to be rejected.

If you cannot avoid accepting non-conformant values, then use the dsconfig command to change the advanced Global property: invalid-attribute-syntax-behavior.

Object Class Inheritance

ODSEE does not apply this restriction from RFC 4512 section 2.4.2:

An object or alias entry is characterized by precisely one structural object class superclass chain which has a single structural object class as the most subordinate object class. This structural object class is referred to as the structural object class of the entry.

DS follows the RFC by default. This can lead to errors when you try to use object class definitions that worked with ODSEE, but do not follow the restriction.

One good way to work around the problem is to fix the schema definitions. You can often redefine some object classes as AUXILIARY rather than STRUCTURAL. Auxiliary object classes augment the characteristics of entries, letting you add attributes to entries without changing what the entries represent. For instance, the example-class-of-service-object-class near the outset of Example.ldif defines an Auxiliary object class. It lets you add classOfService, mailQuota, and diskQuota attributes to user entries. Common sense tells you it is better to employ an Auxiliary object class in the case of Class of Service. Clearly a user has a Class of Service; no user is a class of service.

Another workaround is to change DS’s configuration to make it more lenient. Use the dsconfig command to change the advanced Global property: single-structural-objectclass-behavior. This workaround is recommended only as an expedient temporary measure allowing you quickly to test with ODSEE schema definitions until you get a chance to fix the schema definitions in alignment with RFC 4512.

Attribute Names with Underscores

RFC 4512 allows attribute names to use letters, digits, and hyphens with the first character being a letter, basically: [A-Za-z][A-Za-z0-9-]*. If the schema definitions from ODSEE have attribute names with underscores and applications depend on those names, it can be tough to fix them.

In such cases you can use the dsconfig command to change the advanced Global property: allow-attribute-name-exceptions.

Replication: ODSEE peer-to-peer versus DS hub-and-spoke

Both ODSEE and DS support multi-master replication with automated conflict resolution. Multi-master means that more than one replica (often all replicas) can handle both read and write operations. Automated conflict resolution means that if the directory service can resolve two conflicting write operations—such as different attribute values being written to the same entry at the same time on two different replicas—then it performs the resolution without any manual administrator action.

This is the killer feature of LDAP directories, making it easy to set up a service that continues working fine when a server or data center goes down. (The tradeoff of eventual consistency—eventually, but not instantly, replication makes data consistent across all replicas—is perhaps the Achilles heel of LDAP directories. Eventual consistency can be counterintuitive to developers used to relational databases and synchronous operations.)

In ODSEE, replication traffic goes over LDAP. Replication is configured in peer-to-peer fashion, with each server having a replication agreement for each replica. Each ODSEE replica must keep track of its state with respect to peer replicas. In a large deployment this leads to many replication agreements and complex configuration. It can also lead to a lot of traffic, including traffic over WAN links.

In DS, replication traffic uses its own ports and its own protocol. A basic default configuration resembles the peer-to-peer arrangement used by ODSEE. And yet there is a crucial design difference in DS: the directory service and the replication service are separate, and can be set up in fact, on separate servers. In DS the directory service responds to client requests. It also publishes its changes to the replication service, and subscribes to the replication service for changes from other directory servers. A directory server needs to connect only to one replication server at a time (potentially per suffix).

In DS​​​​​​​, the replication service manages replication data only, handling replication traffic with directory servers and with other replication servers, receiving, sending, and storing only changes to directory data rather than full entry data itself. When you configure the two separately, you can significantly reduce the number of replication servers needed as described in Install Standalone Servers. This can limit the traffic over WAN links for example.

When you move from ODSEE to DS​​​​​​​, you should therefore reconsider how replication is set up. In particular if you have many servers, or if you have a lot of update traffic, DS​​​​​​​ is better designed to handle your needs tha ODSEE, and you should take advantage of that.

Operational attributes and password policy: different attributes, additional DS functionality

Operational attributes are those attributes used by the directory server for its own operation. They must be specified to be returned in an LDAP search for example.

Password policy attributes can be operational, or can be part of the user data.

Operational Attribute Differences

The following table maps ODSEE operational attributes to those of DS.

Password Policy Differences

DS supports Internet-Draft Password Policy for LDAP Directories (version 09), but also server configuration-based password policies, with additional password policy features not available in ODSEE. See Configure Password Policies for details.

The following tables map ODSEE password policy attributes to DS subentry policy attributes.

Data Storage: DS for portability and performance

Like ODSEE, DS uses Berkeley DB technology to store user data. The implementations are different, however. Even if your data is perfectly compatible between ODSEE and DS, you must still export it to LDIF from ODSEE and then import it into DS.

Moreover indexes between ODSEE and DS are not configured in the same way. For each of your ODSEE indexes that applies to user data, you must configure a corresponding index in DS​​​​​​​. See Indexes for instructions on setting up indexes in DS​​​​​​​.

ODSEE uses the C language Berkeley DB. The internal data representation is page based. ODSEE data is portable from one server to another as long as the underlying host system is identical. You therefore need one ODSEE backup server per host configuration.

DS​​​​​​​ uses the Java language Berkeley DB by default to store user data. The internal data representation is log based. The log based representation affords DS​​​​​​​ improved write performance. Throughput for write operations is often an order of magnitude better with DS​​​​​​​ than with ODSEE. Furthermore, DS​​​​​​​ backups are fully portable between servers. You therefore need only one DS​​​​​​​ backup server at a time.

ODSEE roles and CoS versus DS groups, virtual and collective attributes

Roles and Class of Service (CoS) in ODSEE were implemented when Sun and Netscape server product teams worked together in iPlanet:

• Roles are a proprietary feature of ODSEE used to associate entries with different roles; somewhat similar to groups but reversed: groups have DNs pointing at members, role members have DNs (nsRole attribute) pointing at each role entry. See Migrating Oracle DSEE roles to DS for further information.
• CoS is a proprietary feature of ODSEE used to manage attribute values that are common to many entries, in a single place. For example, the postal address might be an attribute that’s common to everyone in an organization. See Migrating Oracle DSEE CoS to DS for further information.

ODSEE ACIs versus DS ACIs: no MACRO ACIs exist in DS

DS access control instructions aim to be strictly compatible with ODSEE access control instructions, except that DS does not support ODSEE Macro ACIs. Nevertheless, test that your various ACIs produce the expected results after you migrate from ODSEE to DS. See Access Control for further information.

DS employs global-aci settings in the server configuration, for server-wide access control.

DS also has a privileges subsystem that provides access control for server administration operations independently from access control instructions.

PKI: ODSEE NSS versus DS Java-based implementations

ODSEE relies on Netscape Security Services (NSS) for its keystore and truststore implementations. Sun Microsystems inherited the Netscape Certificate Management System for managing keys, but later discontinued the product. Although it was originally designed only for testing, the command-line certutil tool can be used to manage the keys.

DS builds on Java-based PKI infrastructure for its keystore and truststore implementations. By default it uses Java Key Store (JKS), but you can also use other formats such as PKCS12 for the trust store or PKCS11 for the key store. The documentation describes how to secure the ports listening for client connections and also how to change server certificates/key pairs.

Code Base: C/C++ in ODSEE versus Java in DS

As a server implemented in the C language, ODSEE provides a C-based plugin API, described in the ODSEE Developer’s Guide. ODSEE provides a number of plugin points.

As a pure Java server, DS provides a large Java API, including a plugin API with even more plugin points than ODSEE.

If you cannot avoid writing a plugin for DS to match an existing ODSEE plugin, know that you must re-implement the plugin in the Java language. Begin with the example plugin delivered with DS, /path/to/ds/example-plugin.zip.

General ODSEE to DS migration phases

Moving from ODSEE to DS is not an upgrade. You must install DS alongside ODSEE, configure DS appropriately and move your data from ODSEE to DS.

Installing DS

This is a chance to take another look at the directory service architecture over all.

DS's hub-and-spoke replication model can work in practice like ODSEE's peer-to-peer model, so you might want the same replication topology as before. Or you can take the migration as an opportunity to split the replication service from the directory service, and reduce load over the WAN.

You can choose to retire older hardware, or even to move to a platform as a service. DS has proven to work well in virtualized production environments.

Keep in mind that DS performance tuning settings, for a variety of reasons, differ from ODSEE tuning settings, so do not assume that what you have done to prepare for ODSEE has exactly the same effect for DS. If you have had to tune ODSEE performance, subject DS to the same performance tests you used for ODSEE. When out-of-the-box performance with DS is not good enough, read the Maintenance for tuning tips.

Updating DS schema definitions

Generally DS schema definitions are compatible with those of ODSEE, and customizations that you applied in ODSEE also work in DS.

But some of your ODSEE data might not quite comply with DS's more strict interpretation of object class inheritance and of attribute syntax checking. In practice, you apply schema customizations to DS, and then try to import your ODSEE data with default (strict) settings for schema checking.

If errors arise during the import, figure out how to correct each error, and prepare automated error corrections so that you can apply them during migration.

Indexing directory data

DS​​​​​​​ index configuration is done differently from ODSEE index configuration.

For each attribute you must index, find the appropriate configuration in DS​​​​​​​ so that when you import data it is indexed properly.

Adapting ACIs

Generally DS​​​​​​​'s ACIs are compatible with those of ODSEE, except that DS​​​​​​​ does not have MACRO ACIs.

That said, test the impact of your ACIs in DS​​​​​​​, and make sure there are no surprises.

If any bugs are found during testing, figure out how to correct the offending ACIs, and prepare automated error corrections so that you can apply them during migration.

Adapting password policy and account status attributes

As described above, DS​​​​​​​ password policies and operational attributes differ from those of ODSEE.

Adapt password policies and account status attributes after you export data from ODSEE in order to achieve the desired effects in DS​​​​​​​.

Prepare and test automated adaptations to apply during migration.

Moving from Roles and CoS to Virtual and Collective attributes

DS​​​​​​​ does not use the Roles and Class of Service features provided by ODSEE, but it supports all of the same use cases.

Adapt your Roles and CoS definitions for use with DS​​​​​​​ as described in the ODSEE Roles and CoS versus DS​​​​​​​ Groups, Virtual and Collective attributes section above.

Prepare and test automated adaptations to apply during migration.

Rewriting server plugins

If you have custom ODSEE server plugins that do something DS​​​​​​​ does not do, then consider whether you still need the functionality.

Custom server plugins that you find you still need must be rewritten in Java and installed into DS​​​​​​​. See /path/to/ds/example-plugin.zip where you installed DS​​​​​​​ to get started.

Adapting PKI

When you install DS​​​​​​​ separately from ODSEE, you might also be getting new private-public key pairs to secure traffic on the DS​​​​​​​ servers as part of installation, and so you can skip this phase.

In cases where you opt to reuse key pairs, you must export them from ODSEE's NSS store and import them into DS's Java-based store. Do consider the impact of potential differences in supported protocols and cipher suites.

Adapting backup and recovery strategies

As described above DS​​​​​​​ backup is more portable than that of ODSEE. Furthermore import is generally faster.

Adapt your backup and recovery plans accordingly. 

Adapting monitoring

DS​​​​​​​ has logs, numerous monitoring options as well as cn=monitor suffix similar to ODSEE. And yet the content is not identical.

Adapt your monitoring capabilities to those of DS​​​​​​​, so that you can achieve the same aims as with ODSEE.

Adapting and importing data into DS​​​​​​​

Once you have finished the steps above, you can test automated import.

Assuming the results of automated import are what you expect, then what you do depends on whether you can do the whole migration with the directory service offline, or whether you must keep the directory service online during migration.

Offline or live migration?

If you have a large enough maintenance window, you can perform offline migration:

1. Perform the export process including all the adaptations described above.
2. Import the adapted data into DS.
3. Verify that everything works as expected with DS​​​​​​​.
4. Point applications to DS​​​​​​​.

If you must keep your directory service online during migration, then you must deal with changes that happen after the initial import to DS​​​​​​​:

1. Perform the export process including all the adaptations described above.
2. Import the adapted data into DS.
3. Replay new changes in ODSEE to DS​​​​​​​ by using the ODSEE retro changelog or audit log.
4. Verify that everything works as expected with DS​​​​​​​.
5. Point applications to DS​​​​​​​.
6. Replay new changes in DS​​​​​​​ to ODSEE by using the DS​​​​​​​ external changelog until the ODSEE-based service is retired.

See Also

Migrating Oracle DSEE roles to DS

Migrating Oracle DSEE CoS to DS

FAQ: Moving from Oracle DSEE to DS

FAQ: Installing and configuring DS

Migrating from Oracle DSEE to ForgeRock Directory Services

DS product page

DS product documentation

Related Training

ForgeRock Directory Services Core Concepts (DS-400)