What do I need to consider when planning a migration from Oracle DSEE to DS?
The purpose of this article is to provide information on what you must consider when planning a migration from Oracle® Directory Server Enterprise Edition (ODSEE).
2 readers recommend this article
Introduction
Parts of the ODSEE code base dates back to the original Netscape® Directory Server (DS). Over the years the directory server became a widely used, robust product. And yet one of the original assumptions built into ODSEE is that directory services are mainly for reads, not for writes. Furthermore the code base for ODSEE started out at a time when the LDAP standards were still very much in progress, and so does not always apply the standards strictly.
The code base that today backs DS started in the ODSEE team after it had already become clear that directories must handle as many writes as reads, today's directory data being more volatile than ever. In addition the LDAP standards were mature and consolidating, so for interoperability it made sense to follow them more strictly. Luckily, a primary goal of the project has always been relatively easy migration from ODSEE.
Today with Oracle holding its infrastructure products like ODSEE in maintenance mode, it makes sense to leave yesterday's architecture behind and build directory services with DS.
This article is not a step-by-step guide to making that change. Instead it highlights what you must consider when planning a migration from ODSEE to DS. We assume that you are already familiar with both products, and that you already have successfully managed similar projects in the past. If not, you can turn to one of ForgeRock’s partners, or you can complete the project yourself with the expectation that you will learn a lot along the way.
How ODSEE differs from DS
DS aims for compatibility with ODSEE, providing additional performance and features. There are nevertheless a number of important differences you must be aware of when moving from ODSEE to DS, including the following which are discussed in detail in subsequent sections:
- LDAPv3: ODSEE leniency versus DS compliance.
- Replication: ODSEE peer-to-peer versus DS hub-and-spoke.
- Operational attributes and password policy: different attributes, additional DS functionality.
- Data Storage: DS for portability and performance.
- ODSEE roles and CoS versus DS groups, virtual and collective attributes.
- ODSEE ACIs versus DS ACIs: no MACRO ACIs exist in DS.
- PKI: ODSEE NSS versus DS Java®-based implementations.
- Code Base: C/C++ in ODSEE versus Java in DS.
LDAPv3: ODSEE leniency versus DS compliance
In some respects, ODSEE is more lenient in what it accepts versus DS.
Attribute Value Syntax Checking
LDAP attribute values are supposed to respect the syntax defined for the attribute. This means that applications can expect the attribute values they read from the centralized directory service to conform to the syntax, and therefore can handle the values in uniform ways.
The tradeoff is that applications that update directory entries have to perform more checking and processing of their input before sending it to the directory. When a user updates a telephone number manually, the application might have to transform that number to fit telephone number syntax before sending a modify request to the directory.
In ODSEE, the decision was made to lighten the load on application developers by allowing non-conformant attribute values. This is the case for phone numbers and access control instructions (ACIs), for example.
In DS, the default settings limit garbage coming in so as to reduce garbage going out.
If you can, clean up the data, and warn application maintainers that non-conformant values are going to be rejected.
If you cannot avoid accepting non-conformant values, then use the dsconfig command to change the advanced Global property: invalid-attribute-syntax-behavior.
Object Class Inheritance
ODSEE does not apply this restriction from RFC 4512 section 2.4.2:
An object or alias entry is characterized by precisely one structural object class superclass chain which has a single structural object class as the most subordinate object class. This structural object class is referred to as the structural object class of the entry.
DS follows the RFC by default. This can lead to errors when you try to use object class definitions that worked with ODSEE, but do not follow the restriction.
One good way to work around the problem is to fix the schema definitions. You can often redefine some object classes as AUXILIARY rather than STRUCTURAL. Auxiliary object classes augment the characteristics of entries, letting you add attributes to entries without changing what the entries represent. For instance, the example-class-of-service-object-class near the outset of Example.ldif defines an Auxiliary object class. It lets you add classOfService, mailQuota, and diskQuota attributes to user entries. Common sense tells you it is better to employ an Auxiliary object class in the case of Class of Service. Clearly a user has a Class of Service; no user is a class of service.
Another workaround is to change DS’s configuration to make it more lenient. Use the dsconfig command to change the advanced Global property: single-structural-objectclass-behavior. This workaround is recommended only as an expedient temporary measure allowing you quickly to test with ODSEE schema definitions until you get a chance to fix the schema definitions in alignment with RFC 4512.
Attribute Names with Underscores
RFC 4512 allows attribute names to use letters, digits, and hyphens with the first character being a letter, basically: [A-Za-z][A-Za-z0-9-]*. If the schema definitions from ODSEE have attribute names with underscores and applications depend on those names, it can be tough to fix them.
In such cases you can use the dsconfig command to change the advanced Global property: allow-attribute-name-exceptions.
Replication: ODSEE peer-to-peer versus DS hub-and-spoke
Both ODSEE and DS support multi-master replication with automated conflict resolution. Multi-master means that more than one replica (often all replicas) can handle both read and write operations. Automated conflict resolution means that if the directory service can resolve two conflicting write operations—such as different attribute values being written to the same entry at the same time on two different replicas—then it performs the resolution without any manual administrator action.
This is the killer feature of LDAP directories, making it easy to set up a service that continues working fine when a server or data center goes down. (The tradeoff of eventual consistency—eventually, but not instantly, replication makes data consistent across all replicas—is perhaps the Achilles heel of LDAP directories. Eventual consistency can be counterintuitive to developers used to relational databases and synchronous operations.)
In ODSEE, replication traffic goes over LDAP. Replication is configured in peer-to-peer fashion, with each server having a replication agreement for each replica. Each ODSEE replica must keep track of its state with respect to peer replicas. In a large deployment this leads to many replication agreements and complex configuration. It can also lead to a lot of traffic, including traffic over WAN links.
In DS, replication traffic uses its own ports and its own protocol. A basic default configuration resembles the peer-to-peer arrangement used by ODSEE. And yet there is a crucial design difference in DS: the directory service and the replication service are separate, and can be set up in fact, on separate servers. In DS the directory service responds to client requests. It also publishes its changes to the replication service, and subscribes to the replication service for changes from other directory servers. A directory server needs to connect only to one replication server at a time (potentially per suffix).
In DS, the replication service manages replication data only, handling replication traffic with directory servers and with other replication servers, receiving, sending, and storing only changes to directory data rather than full entry data itself. When you configure the two separately, you can significantly reduce the number of replication servers needed as described in Install Standalone Servers. This can limit the traffic over WAN links for example.
When you move from ODSEE to DS, you should therefore reconsider how replication is set up. In particular if you have many servers, or if you have a lot of update traffic, DS is better designed to handle your needs tha ODSEE, and you should take advantage of that.
Operational attributes and password policy: different attributes, additional DS functionality
Operational attributes are those attributes used by the directory server for its own operation. They must be specified to be returned in an LDAP search for example.
Password policy attributes can be operational, or can be part of the user data.
Operational Attribute Differences
The following table maps ODSEE operational attributes to those of DS.
Note
This table is based on the documented ODSEE operational attributes, and on DS schema. This table does not include the DS attributes that are not in ODSEE.
ODSEE | DS |
---|---|
accountUnlockTime | N/A, calculated based on pwdAccountLockedTime |
aci | aci (compatible except for MACRO acis) |
changeHasReplFixupOp | N/A, ODSEE RetroChangelog attribute |
changeIsReplFixupOp | N/A, ODSEE RetroChangelog attribute |
copiedFrom | N/A, ODSEE RetroChangelog attribute |
copyingFrom | N/A, ODSEE RetroChangelog attribute |
deletedEntryAttrs | N/A, replication model is different |
ds-pluginDigest | N/A |
ds-pluginSignature | N/A |
isMemberOf | isMemberOf virtual attribute (configurable) |
ldapSyntaxes | ldapSyntaxes |
namingContexts | namingContexts, ds-private-naming-contexts |
nsIdleTimeout | idle-time-limit, ds-rlim-idle-time-limit |
nsLookThroughLimit | time-limit, ds-rlimit-time-limit |
nsRole | N/A, can be simulated by creating an nsRole virtual attribute |
nsds5ReplConflict | N/A, replication model is different |
nsRoleDN | N/A, can be simulated by creating an nsRole virtual attribute |
nsSizeLimit | size-limit, ds-rlim-size-limit |
nsTimeLimit | time-limit, ds-rlim-time-limit |
numSubordinates | numSubordinates |
passwordAllowChangeTime | N/A, calculated based on pwdChangedTime |
passwordExpWarned | N/A |
passwordExpirationTime | N/A, calculated based on pwdChangedTime |
passwordHistory | pwdHistory |
passwordPolicySubentry | pwdPolicySubentry |
passwordRetryCount | N/A, calculated based on pwdFailureTime |
pwdAccountLockedTime | pwdAccountLockedTime |
pwdChangedTime | pwdChangedTime |
pwdFailureTime | pwdFailureTime |
pwdGraceUseTime | pwdGraceUseTime |
pwdHistory | pwdHistory |
pwdLastAuthTime | Depends on the last-login-time-attribute setting |
pwdPolicySubentry | pwdPolicySubentry |
pwdReset | pwdReset |
replicaIdentifier | replicaIdentifier |
replicationCSN | replicationCSN |
retryCountResetTime | N/A, calculated based on pwdAccountLockedTime |
subschemaSubentry | subschemaSubentry |
supportedControl | supportedControl |
supportedExtension | supportedExtension |
supportedLDAPVersion | supportedLDAPVersion |
supportedSASLMechanisms | supportedSASLMechanisms |
targetUniqueId | targetEntryUUID (alias targetUniqueID) |
vendorName | vendorName |
vendorVersion | vendorVersion |
Password Policy Differences
DS supports Internet-Draft Password Policy for LDAP Directories (version 09), but also server configuration-based password policies, with additional password policy features not available in ODSEE. See Configure Password Policies for details.
The following tables map ODSEE password policy attributes to DS subentry policy attributes.
Note
This table is based on documented ODSEE password policy attributes for the object class pwdPolicy.
ODSEE | DS |
---|---|
pwdAllowUserChange | pwdAllowUserChange |
pwdCheckQuality | N/A, use DS password validators instead |
pwdExpireWarning | pwdExpireWarning |
pwdFailureCountInterval | pwdFailureCountInterval |
pwdGraceAuthNLimit | pwdGraceAuthNLimit |
pwdInHistory | pwdInHistory |
pwdLockout | N/A, calculated from other attributes |
pwdLockoutDuration | pwdLockoutDuration |
pwdMaxAge | pwdMaxAge |
pwdMaxFailure | pwdMaxFailure |
pwdMinAge | pwdMinAge |
pwdMinLength | N/A, use the length based password validator instead |
pwdMustChange | pwdMustChange |
pwdSafeModify | pwdSafeModify |
Note
This table is based on documented ODSEE password policy attributes for the object class passwordPolicy.
ODSEE | DS |
---|---|
passwordChange | See pwdAllowUserChange |
passwordCheckSyntax | N/A, use DS password validators instead |
passwordExp | See require-change-by-time |
passwordExpireWithoutWarning | expire-passwords-without-warning |
passwordInHistory | See pwdInHistory |
passwordLockout | N/A, calculated from other attributes |
passwordLockoutDuration | See pwdLockoutDuration |
passwordMaxAge | See pwdMaxAge |
passwordMaxFailure | See pwdMaxFailure |
passwordMinAge | See pwdMinAge |
passwordMinLength | N/A, use the length based password validator instead |
passwordMustChange | See pwdMustChange |
passwordNonRootMayResetUserpwd | N/A, see the password-reset privilege |
passwordResetDuration | See pwdLockoutDuration |
passwordResetFailureCount | See pwdMaxFailure |
passwordRootdnMayBypassModsChecks | See skip-validation-for-administrators |
passwordStorageScheme | See default-password-storage-schemes |
passwordUnlock | N/A, off could be simulated with pwdLockoutDuration |
passwordWarning | See pwdExpireWarning |
Data Storage: DS for portability and performance
Like ODSEE, DS uses Berkeley DB technology to store user data. The implementations are different, however. Even if your data is perfectly compatible between ODSEE and DS, you must still export it to LDIF from ODSEE and then import it into DS.
Moreover indexes between ODSEE and DS are not configured in the same way. For each of your ODSEE indexes that applies to user data, you must configure a corresponding index in DS. See Indexes for instructions on setting up indexes in DS.
ODSEE uses the C language Berkeley DB. The internal data representation is page based. ODSEE data is portable from one server to another as long as the underlying host system is identical. You therefore need one ODSEE backup server per host configuration.
DS uses the Java language Berkeley DB by default to store user data. The internal data representation is log based. The log based representation affords DS improved write performance. Throughput for write operations is often an order of magnitude better with DS than with ODSEE. Furthermore, DS backups are fully portable between servers. You therefore need only one DS backup server at a time.
ODSEE roles and CoS versus DS groups, virtual and collective attributes
Roles and Class of Service (CoS) in ODSEE were implemented when Sun and Netscape server product teams worked together in iPlanet:
- Roles are a proprietary feature of ODSEE used to associate entries with different roles; somewhat similar to groups but reversed: groups have DNs pointing at members, role members have DNs (nsRole attribute) pointing at each role entry. See Migrating Oracle DSEE roles to DS for further information.
- CoS is a proprietary feature of ODSEE used to manage attribute values that are common to many entries, in a single place. For example, the postal address might be an attribute that’s common to everyone in an organization. See Migrating Oracle DSEE CoS to DS for further information.
ODSEE ACIs versus DS ACIs: no MACRO ACIs exist in DS
DS access control instructions aim to be strictly compatible with ODSEE access control instructions, except that DS does not support ODSEE Macro ACIs. Nevertheless, test that your various ACIs produce the expected results after you migrate from ODSEE to DS. See Access Control for further information.
DS employs global-aci settings in the server configuration, for server-wide access control.
DS also has a privileges subsystem that provides access control for server administration operations independently from access control instructions.
PKI: ODSEE NSS versus DS Java-based implementations
ODSEE relies on Netscape Security Services (NSS) for its keystore and truststore implementations. Sun Microsystems inherited the Netscape Certificate Management System for managing keys, but later discontinued the product. Although it was originally designed only for testing, the command-line certutil tool can be used to manage the keys.
DS builds on Java-based PKI infrastructure for its keystore and truststore implementations. By default it uses Java Key Store (JKS), but you can also use other formats such as PKCS12 for the trust store or PKCS11 for the key store. The documentation describes how to secure the ports listening for client connections and also how to change server certificates/key pairs.
Code Base: C/C++ in ODSEE versus Java in DS
As a server implemented in the C language, ODSEE provides a C-based plugin API, described in the ODSEE Developer’s Guide. ODSEE provides a number of plugin points.
As a pure Java server, DS provides a large Java API, including a plugin API with even more plugin points than ODSEE.
Note
Server APIs are considered evolving. It is likely that a future version of DS will provide a more stable, but also more restricted plugin API, and that customization could be achieved through scripting languages like Groovy or JavaScript instead of compiled Java.
If you cannot avoid writing a plugin for DS to match an existing ODSEE plugin, know that you must re-implement the plugin in the Java language. Begin with the example plugin delivered with DS, /path/to/ds/example-plugin.zip.
General ODSEE to DS migration phases
Moving from ODSEE to DS is not an upgrade. You must install DS alongside ODSEE, configure DS appropriately and move your data from ODSEE to DS.
Installing DS
This is a chance to take another look at the directory service architecture over all.
DS's hub-and-spoke replication model can work in practice like ODSEE's peer-to-peer model, so you might want the same replication topology as before. Or you can take the migration as an opportunity to split the replication service from the directory service, and reduce load over the WAN.
You can choose to retire older hardware, or even to move to a platform as a service. DS has proven to work well in virtualized production environments.
Keep in mind that DS performance tuning settings, for a variety of reasons, differ from ODSEE tuning settings, so do not assume that what you have done to prepare for ODSEE has exactly the same effect for DS. If you have had to tune ODSEE performance, subject DS to the same performance tests you used for ODSEE. When out-of-the-box performance with DS is not good enough, read the Maintenance for tuning tips.
Updating DS schema definitions
Generally DS schema definitions are compatible with those of ODSEE, and customizations that you applied in ODSEE also work in DS.
But some of your ODSEE data might not quite comply with DS's more strict interpretation of object class inheritance and of attribute syntax checking. In practice, you apply schema customizations to DS, and then try to import your ODSEE data with default (strict) settings for schema checking.
If errors arise during the import, figure out how to correct each error, and prepare automated error corrections so that you can apply them during migration.
Indexing directory data
DS index configuration is done differently from ODSEE index configuration.
For each attribute you must index, find the appropriate configuration in DS so that when you import data it is indexed properly.
Adapting ACIs
Generally DS's ACIs are compatible with those of ODSEE, except that DS does not have MACRO ACIs.
That said, test the impact of your ACIs in DS, and make sure there are no surprises.
If any bugs are found during testing, figure out how to correct the offending ACIs, and prepare automated error corrections so that you can apply them during migration.
Adapting password policy and account status attributes
As described above, DS password policies and operational attributes differ from those of ODSEE.
Adapt password policies and account status attributes after you export data from ODSEE in order to achieve the desired effects in DS.
Prepare and test automated adaptations to apply during migration.
Moving from Roles and CoS to Virtual and Collective attributes
DS does not use the Roles and Class of Service features provided by ODSEE, but it supports all of the same use cases.
Adapt your Roles and CoS definitions for use with DS as described in the ODSEE Roles and CoS versus DS Groups, Virtual and Collective attributes section above.
Prepare and test automated adaptations to apply during migration.
Rewriting server plugins
If you have custom ODSEE server plugins that do something DS does not do, then consider whether you still need the functionality.
Custom server plugins that you find you still need must be rewritten in Java and installed into DS. See /path/to/ds/example-plugin.zip where you installed DS to get started.
Adapting PKI
When you install DS separately from ODSEE, you might also be getting new private-public key pairs to secure traffic on the DS servers as part of installation, and so you can skip this phase.
In cases where you opt to reuse key pairs, you must export them from ODSEE's NSS store and import them into DS's Java-based store. Do consider the impact of potential differences in supported protocols and cipher suites.
Adapting backup and recovery strategies
As described above DS backup is more portable than that of ODSEE. Furthermore import is generally faster.
Adapt your backup and recovery plans accordingly.
Adapting monitoring
DS has logs, numerous monitoring options as well as cn=monitor suffix similar to ODSEE. And yet the content is not identical.
Adapt your monitoring capabilities to those of DS, so that you can achieve the same aims as with ODSEE.
Adapting and importing data into DS
Once you have finished the steps above, you can test automated import.
Assuming the results of automated import are what you expect, then what you do depends on whether you can do the whole migration with the directory service offline, or whether you must keep the directory service online during migration.
Offline or live migration?
If you have a large enough maintenance window, you can perform offline migration:
- Perform the export process including all the adaptations described above.
- Import the adapted data into DS.
- Verify that everything works as expected with DS.
- Point applications to DS.
If you must keep your directory service online during migration, then you must deal with changes that happen after the initial import to DS:
- Perform the export process including all the adaptations described above.
- Import the adapted data into DS.
- Replay new changes in ODSEE to DS by using the ODSEE retro changelog or audit log.
- Verify that everything works as expected with DS.
- Point applications to DS.
- Replay new changes in DS to ODSEE by using the DS external changelog until the ODSEE-based service is retired.
Note
When replaying the changes, take care to apply any necessary transformations where you must adapt ODSEE data for use in DS or DS data for use in ODSEE. The necessary adaptations are those described above.
See Also
Migrating Oracle DSEE roles to DS
Migrating Oracle DSEE CoS to DS
FAQ: Moving from Oracle DSEE to DS
FAQ: Installing and configuring DS
Migrating from Oracle DSEE to ForgeRock Directory Services