Backup and Restore

Important

  • Backups are not guaranteed to be compatible across major and minor server releases. Restore backups only on directory servers of the same major or minor version.

    To share data between servers of different versions, either use replication, or use LDIF.

  • Using file system snapshots for backup is not recommended.

    Database backend cleanup operations may write data even when there are no pending client or replication operations. An ongoing file system backup operation may record database log files that are not in sync with each other. If you cannot avoid file system snapshots, you must stop the server before taking the snapshot.

  • DS servers use cryptographic keys to sign and verify the integrity of backup files, and to encrypt data. Servers protect these keys by encrypting them with the shared master key for a deployment. For portability, servers store the encrypted keys in the backup files.

    Any server can therefore restore a backup taken with the same server version, as long as it holds a copy of the shared master key used to encrypt the keys.

Back Up

When you set up a directory server, the process creates a /path/to/opendj/bak/ directory. You can use this for backups if you have enough local disk space, and when developing or testing backup processes. In deployment, store backups remotely to avoid losing your data and your backups in the same crash.

Back Up Data (Server Task)

When you schedule a backup as a server task, the DS server manages task completion. The server must be running when you schedule the task, and when the task runs:

  • Schedule the task on a running server, binding as a user with the backend-backup administrative privilege.

    The following example schedules an immediate backup task for the dsEvaluation backend:

    $ dsbackup \
     create \
     --hostname localhost \
     --port 4444 \
     --bindDN uid=admin \
     --bindPassword password \
     --usePkcs12TrustStore /path/to/opendj/config/keystore \
     --trustStorePasswordFile /path/to/opendj/config/keystore.pin \
     --backupLocation bak \
     --backendName dsEvaluation

    To back up all backends, omit the --backendName option.

    To back up more than one backend, specify the --backendName option multiple times.

    For details, see dsbackup — Backup and restore backends.

Back Up Data (Scheduled Task)

When you schedule a backup as a server task, the DS server manages task completion. The server must be running when you schedule the task, and when the task runs:

  • Schedule backups using the crontab format with the --recurringTask option.

    The following example schedules nightly online backup of all user data at 2 AM, notifying diradmin@example.com when finished, or on error:

    $ dsbackup \
     create \
     --hostname localhost \
     --port 4444 \
     --bindDN uid=admin \
     --bindPassword password \
     --usePkcs12TrustStore /path/to/opendj/config/keystore \
     --trustStorePasswordFile /path/to/opendj/config/keystore.pin \
     --backupLocation bak \
     --recurringTask "00 02 * * *" \
     --description "Nightly backup at 2 AM" \
     --taskId NightlyBackup \
     --completionNotify diradmin@example.com \
     --errorNotify diradmin@example.com

    For details, see dsbackup — Backup and restore backends.

Back Up Data (External Command)

When you back up data without contacting the server, the dsbackup create command runs as an external command, independent of the server process. It backs up the data whether the server is running or not.

Note

When you back up LDIF-based backends with this method, the command does not lock the files. To avoid corrupting the backup files, do not run the dsbackup create --offline command on an LDIF backend simultaneously with any changes to the backend.

This applies to LDIF backends, schema files, and the task backend, for example.

Use this method to schedule backup with a third-party tool, such as the cron command:

  • Back up data without contacting the server process, and use the --offline option.

    The following example backs up the dsEvaluation backend immediately:

    $ dsbackup \
     create \
     --offline \
     --backupLocation bak \
     --backendName dsEvaluation

    To back up all backends, omit the --backendName option.

    To back up more than one backend, specify the --backendName option multiple times.

    For details, see dsbackup — Backup and restore backends.

Back Up Configuration Files

When you back up directory data using the DS tools, you do not back up server configuration files. The server stores configuration files under the /path/to/opendj/config/ directory.

The server records snapshots of its configuration under the /path/to/opendj/var/ directory. You can use snapshots to recover from misconfiguration performed with the dsconfig command. Snapshots only reflect the main configuration file, config.ldif.

  1. Stop the server:

    $ stop-ds
  2. Back up the configuration files:

    $ tar -zcvf backup-config-$(date +%s).tar.gz config

    By default, this backup includes the server keystore, so store it securely.

  3. Start the server:

    $ start-ds

Restore

Important

After you restore a replicated backend, replication brings it up to date with changes newer than the backup. Replication uses internal change log records to determine which changes to apply.

Replication purges internal change log records, however, to prevent the change log from growing indefinitely. Replication can only bring the backend up to date if the change log still includes the last change backed up.

For this reason, when you restore a replicated backend from backup, the backup must be newer than the last purge of the replication change log (default: 3 days).

If no backups are newer than the replication purge delay, do not restore from a backup. Initialize the replica instead, without using a backup. For details, see "Manual Initialization".

Restore Data (Server Task)
  1. (Optional) Verify the backup you intend to restore.

    The following example verifies the most recent backup of the dsEvaluation backend:

    $ dsbackup \
     list \
     --backupLocation bak \
     --backendName dsEvaluation \
     --last \
     --verify
  2. Schedule the restore operation as a task, binding as a user with the backend-restore administrative privilege.

    The following example schedules an immediate restore task for the dsEvaluation backend:

    $ dsbackup \
     restore \
     --hostname localhost \
     --port 4444 \
     --bindDN uid=admin \
     --bindPassword password \
     --usePkcs12TrustStore /path/to/opendj/config/keystore \
     --trustStorePasswordFile /path/to/opendj/config/keystore.pin \
     --backupLocation bak \
     --backendName dsEvaluation

    To restore the latest backups of more than one backend, specify the --backendName option multiple times.

    To restore a specific backup, specify the --backupId option. To restore multiple specific backups of different backends, specify the --backupId option multiple times.

    To list backup information without performing verification, use the dsbackup list command without the --verify option. The output includes backup IDs for use with the --backupId option.

    For details, see dsbackup — Backup and restore backends.

Restore Data (External Command)
  1. Stop the server if it is running:

    $ stop-ds --quiet
  2. (Optional) Verify the backup you intend to restore.

    The following example verifies the most recent backup of the dsEvaluation backend:

    $ dsbackup \
     list \
     --backupLocation bak \
     --backendName dsEvaluation \
     --last \
     --verify
  3. Restore using the --offline option.

    The following example restores the dsEvaluation backend:

    $ dsbackup \
     restore \
     --offline \
     --backupLocation bak \
     --backendName dsEvaluation

    To restore the latest backups of more than one backend, specify the --backendName option multiple times.

    To restore a specific backup, specify the --backupId option. To restore multiple specific backups of different backends, specify the --backupId option multiple times.

    To list backup information without performing verification, use the dsbackup list command without the --verify option. The output includes backup IDs for use with the --backupId option.

    For details, see dsbackup — Backup and restore backends.

  4. Start the server:

    $ start-ds --quiet
Restore Configuration Files
  1. Stop the server:

    $ stop-ds --quiet
  2. Restore the configuration files from the backup, overwriting existing files:

    $ tar -zxvf backup-config-<date>.tar.gz
  3. Start the server:

    $ start-ds --quiet

Purge Old Files

Periodically purge old backup files with the dsbackup purge command. The following example removes all backup files older than the default replication purge delay:

$ dsbackup \
 purge \
 --offline \
 --backupLocation bak \
 --olderThan 3d

This example runs the external command without contacting the server process. You can also purge backups by ID, or by backend name, and you can specify the number of backups to keep. For details, see dsbackup — Backup and restore backends.

To purge files as a server task, use the task options, such as --recurringTask. The user must have the backend-backup administrative privilege to schedule a purge task.

Cloud Storage

You can stream backup files to cloud storage, and restore them directly from cloud storage.

The implementation supports these providers:

  • Amazon AWS S3

  • Azure Cloud Storage

  • Google Cloud Storage

Follow these steps to store backup files in the cloud:

  1. Get a storage account and space from the cloud provider where the server can store backup files.

    This storage space is referred to below as cloud-bak.

  2. Get credentials from the cloud provider so the server has read/write access.

    If you are not yet familiar with cloud storage, see the documentation from your provider for help:

    ProviderHints

    Amazon AWS S3

    For details on setting up S3 and working with S3 buckets, see the Amazon Web Services documentation on Getting started with Amazon Simple Storage Service.

    Azure Cloud Storage

    DS authenticates to Azure with an Azure storage account. For details, see the Microsoft documentation on how to Create an Azure Storage account, or to Create a BlockBlobStorage account.

    Google Cloud Storage

    DS authenticates to Google Cloud with a service account. For details, see the Google documentation on Getting Started with Authentication.

    For details about creating and managing storage buckets, see the Google How-To documentation on Creating buckets, and Working with buckets.

  3. Set environment variables for the credentials:

    ProviderEnvironment Variable(s)

    Amazon AWS S3

    export AWS_ACCESS_KEY_ID=aws-access-key

    export AWS_SECRET_ACCESS_KEY=aws-secret-key

    Azure Cloud Storage

    export AZURE_ACCOUNT_NAME=azure-account-name

    export AZURE_ACCOUNT_KEY=azure-account-key

    Google Cloud Storage

    export GOOGLE_CREDENTIALS=/path/to/gcp-credentials.json (optional)

  4. Run dsbackup commands with all required provider-specific options:

    ProviderRequired Options

    Amazon AWS S3

     --storageProperty s3.keyId.env.var:AWS_ACCESS_KEY_ID \
     --storageProperty s3.secret.env.var:AWS_SECRET_ACCESS_KEY \
     --backupLocation s3://cloud-bak

    Azure Cloud Storage

     --storageProperty az.accountName.env.var:AZURE_ACCOUNT_NAME \
     --storageProperty az.accountKey.env.var:AZURE_ACCOUNT_KEY \
     --backupLocation az://cloud-bak

    Google Cloud Storage

     --storageProperty gs.credentials.path:/path/to/gcp-credentials.json \
     --backupLocation gs://cloud-bak

    or

     --storageProperty gs.credentials.env.var:GOOGLE_CREDENTIALS \
     --backupLocation gs://cloud-bak

    Cloud storage requires working space in the local system temporary directory. Some cloud storage providers require sending the content length with each file. To send the correct content length, the dsbackup command writes each prepared backup file to the system temporary directory before upload. It deletes each file after successful upload.

Disaster Recovery

A disaster might involve the crash of a single server or loss of all servers at once. Safeguard a copy of the following items to recover the service:

  • Your deployment description, documentation, plans, runbooks, scripts, and information specific to your deployment.

  • The system configuration and software, including Java installation.

  • The DS software you use with any of your own customizations, plugins, or extensions.

  • A recent backup of any external secrets required, such as an HSM or CA keys.

  • A recent backup of each server's configuration files.

    Recent means the backup matches the configuration used in production.

  • A recent verified backup of replicated data for each backend.

    Recent means newer than the replication purge delay. Verified means that the backup successfully passed the dsbackup list --verify test.

  • The deployment key and deployment password used to set up servers.

    It is possible to replace these, as described in "Replace Deployment Keys". However, you must then safeguard a copy of the old shared master key(s) as well.

Follow these high-level steps to recover from a disaster:

  1. If all the servers are healthy, and the disaster arose from a client software bug or from user error, see "Restore to a Known State" or "Recover From User Error" instead.

    Skip the following steps that focus on rebuilding your servers.

  2. Rebuild the underlying systems on which DS software runs, including the JVM settings.

  3. Install and set up the lost DS servers.

  4. Restore the DS servers' configuration from backup.

  5. Restore the DS servers' data from backup.

Read a different version of :