Notes:

1. RO = Read-only, RW = Read-Write
2. LDAP1 Client facing system is a Slave and is read only. Clients must issue Writes to the Master.

3. LDAP2 Client facing system is a Master and it is replicated to two slaves.

4. LDAP3 is a Multi-Master and clients may issues reads and/or Writes to either system. Each master in this configuration could, in turn, have one or more slave DITs.

7.2 Replication

Replication occurs at the level of the DIT and describes the process of copying updates from a DIT on one LDAP server to the same DIT on one or more other servers. Replication configurations may be either MASTER-SLAVE (the SLAVE copy is always read-only) or MULTI-MASTER. Replication is a configuration (operational) issue. Configurations for OpenLDAP and ApacheDS.

7.2.1 OpenLDAP Replication

Life was once very simple. OpenLDAP replication used slurpd and a temporary file. With version 2.3 of OpenLDAP a new method known as snycrepl (RFC 4533) was introduced while continuing support of slurpd style replication. OpenLDAP version 2.4 has discontinued support for slurpd style replication. The following sections therefore define slurpd style replication for versions up to 2.3 and the new syncrepl style from version 2.4+ (or 2.3+ for the brave).

OpenLDAP Slurpd Style Replication (up to 2.3)

Slurpd style replication is a 'push' replication (and is obsoleted from version 2.4). It is configured and controlled as shown in Figure 7.2-1:

7.2-1 Slurpd Style Replication

When slapd (1) running the Master DIT (7) receives a modify operation (9) it updates the DIT and a timestamped copy of the transaction is written to the log file (2) defined in the master's slapd.conf (5) file replogfile directive.

Slurpd (3) when initially loaded obtains its operational parameters from slapd.conf (5). At a periodic time defined by replicationinterval slurpd will read the log file (2) defined by the replogfile directive and write the updates (10) to one (or more) slave DITs (8) defined by the replica directive(s) in slapd.conf (5).

The slave DIT (8) is a read only copy for all clients except a client which binds using the DN defined by updatedn. The slave server (4) returns the LDAP URI defined by updateref to all modify operations from clients (except those initiated using the DN in updatedn). Both updatedn and updateref are defined in the slapd.conf (6) file. The DN defined in updatedn in (6) MUST be the same as that defined in the replica directive (binddn= parameter) in (5) for this slave instance.

Replication Errors

If slurpd (3) fails to update the slave instance it creates a REJECTION file whose name is the same as that defined in the replogfile directive with .rej appended as shown below:

# slapd.conf reploglogfile directive
replogfile /var/log/ldap/slave1.log

# REJECTION file will be named
# /var/log/ldap/slave1.log.rej

Each error message in the REJECTION log file is the same as that used in the normal transaction log but is preceded by a line starting with the keyword ERROR containing the error message. An example is shown below:

ERROR: No such attribute
replica: slave1.example.com:389
time: 809618633
dn: uid=rsmith,dc=example,dc=com
changetype: modify
replace: description
description: clown
-
replace: modifiersName
modifiersName: uid=rsmith,dc=example,dc=com
-
replace: modifyTimestamp
modifyTimestamp: 20000805073308Z

To fix the errors either the slave may edited (in the case above to add a description attribute to the entry) or the REJECTION log may be edited to correct the errors (in the above example the replace: description line could be changed to add: description). There is no need to remove lines beginning with ERROR since these are ignored. After appropriate remedial action the REJECTION file may be re-applied by running slurpd in a single-shot mode (after stopping any currentlu running slurpd) using the following command:

slurpd -o -r /var/log/ldap/slave1.log.rej

# where -r defines the path to the REJECTION file 
# and -o indicates single-shot mode

Slurpd will apply the transactions in the defined (-r) file and exit. The normal slurpd should now be restarted.

Slurpd Configuration Examples:

Master slapd.conf configuration:

# slapd master
# global section - check file every 5 minutes
replicationinterval 300

# database section
database bdb
...
# simple security to slave located at ldap-rep1.exmple.com 
# with a cleartext password
# directive only used by slurpd
replica uri=ldap://ldap-rep1.example.com bindmethod=simple
  binddn="uid=admin,ou=admin,dc=example,dc=com" credentials=guess
	
# saves changes to specfied file
# directive used by both slapd and slurpd
replogfile /var/log/ldap/slavedit.log

Slave slapd.conf configuration (on host ldap-rep1.example.com):

# slapd slave
# global section - check fie every 5 minutes


# database section
database bdb
...

# defines the dn that is used in the
# replica directive of the master
# directive only used by slurpd
updatedn "uid=admin,ou=admin,dc=example,dc=com"
	
# referral given if a client tries to update slave
updateref ldap://master-ldap.example.com

OpenLDAP syncrepl Style Replication (from 2.3)

OpenLDAP version 2.3 introduced support for a new LDAP Content Synchronization protocol and from version 2.4 this has become the only replication capability supported (slurpd style is now obsoleted). The LDAP Content Synchronization protocol is defined by RFC 4533 and generically known by the name of its controlling slapd.conf directive - syncrepl. syncrepl provides both classic master-slave replication and since version 2.4 allows for multi-master replication. The procotol uses the terms provider (rather than master) to define the source of the replication updates and the term consumer (rather than slave) to define a destination for the updates.

In syncrepl style replication the consumer always intiates the update process - unlike slurpd style where the master (provider) initiates the updates. The consumer may be configured to periodically pull the updates from the provider (refreshOnly) or request the provider to push updates (refreshAndPersist). In all cases, in order to unambiguously refer to an entry the server must maintain a universally unique number (entryUUID) for each entry in a DIT. The process of synchronization is show in Figure 7.2-2 (refreshOnly) and 7.2-3 (refreshAndPersist):

7.2.1.2.1 Replication refreshOnly (Consumer Pull)

7.2-2 snycrepl Provider-Consumer Replication - refreshOnly

A slapd server (1) that wants to replicate a DIT (8) (the consumer) is configured using a syncrepl directive in its slapd.conf file (6). The syncrepl directive defines the location (the name) of the slapd server (3) (the provider) containing the master copy of the DIT. The provider (3) is configured using the overlay syncprov directive in its slapd.conf file (5).

In refreshOnly type of replication the consumer (1) initiates a conection (2) with the provider (2) - synchronization of DITs takes places and the connection is broken. Periodically the consumer (1) re-connects (2) with the provider (3) and re-synchronizes. refreshOnly synchronization may be viewed as operating in burst mode and the replication cycle time is the time between re-connections.

More Detail: The consumer (1) opens a session (2) with the provider (3) and requests refreshOnly synchronization. Any number of consumers can contact the provider and request synchronization service - the provider is not configured with any information about its consumer(s) - as long as a consumer satisfies the security requirements of the provider its synchronization requests will be satisfied. The synchronization request is essentially an extended LDAP search operation which defines the replication scope - using normal LDAP search parameters (base DN, scope,search filter and attributes) - thus the whole, or part, of the providers DIT may be updated in a replica synchronization session depending on the search criteria.

The provider is not required to maintain per-consumer state information. Instead at the end of a synchronization session the provider sends a synchronization cookie (SyncCookie 11) - this cookie contains a Change Sequence Number (contextCSN) - essentially a timestamp indicating the last change sent to this consumer and which may be viewed as a change or synchronization checkpoint. When the consumer initiates a session the last cookie (10) it received from the provider is supplied to indicate to the provider the limits of this synchronization session. Depending on how the consumer was initialised the first time the consumer initiates communication it may not have a SyncCookie and thus the initial synchronization covers all records in the providers DIT (within the synchronization scope). A byproduct of this process allows the replica consumer to start with an empty DIT.

The provider (3) for the DIT will respond using one or both of two phases. The first is the present phase (13) and indicates those entries which are still in the DIT (or DIT fragment) and consists of:

1. For each entry that has changed (since the last synchronization) - the complete entry is sent including its DN and its UUID (entryUUID). The consumer can reliably update its entries from this data.

2. For each entry that has NOT changed (since the last synchronization) an empty entry with its UUID (entryUUID).

3. No messages are sent for entries which have been deleted. Theoretically the consumer may delete such non-referenced entries at the end of this phase.

In the delete phase (14):

1. The provider returns the DN and UUID (entryUUID) for each entry deleted since the last synchronization. The consumer can reliably delete these entries.

Whether both phases are required is determined by a number of additional techniques.

At the end of the synchronization phase(s) the provider sends a SyncCookie (the current contextCSN) and terminates the session. The consumer saves the SyncCookie and will initiate another synchroization session defined by the interval parameter of its syncrepl directive sending the saved SyncCookie to limit the scope of the subsequent synchronization session.

syncrepl refreshOnly Configuration Examples:

Master slapd.conf configuration (assumed host name master-ldap.example.com):

# slapd master
# global section
...

# database section
database bdb
...
# allows read access from consumer
# may need merging with other ACL's
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 

# NOTE: 
# the provider configuration contains no reference to any consumers

# define the provider to use the syncprov overlay
# (last directives in database section)
overlay syncprov
# allows contextCSN to saves to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

consumer slapd.conf configuration:

# slapd consumer
# global section


# database section
database bdb
...

# provider is ldap://master-ldap.example.com:389, sync interval 
# every 1 hour, whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://master-ldap.example.com
  type=refreshOnly
  interval=00:1:00:00
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

7.2.1.2.2 Replication refreshAndPersist (provider Push)

7.2-3 snycrepl Provider-Consumer Replication - refreshAndPersist

A slapd server (1) that wants to replicate a DIT (7) from a server (3) (the provider) is configured using a syncrepl directive in its slapd.conf file (6). The syncrepl directive defines the location (the name) of the slapd server (3) (the provider) containing the master copy of the DIT. The provider (3) is configured using the overlay syncprov directive in its slapd.conf file (5).

In refreshAndPersist type of replication the consumer (1) initiates a conection (2) with the provider (3) - synchronization (12) of DITs takes places immediately and at the end of this process the connection is maintained (it persists). Subsequent changes (4) to the provider (3) are immediately propagated to the consumer(1).

More Detail: The consumer (1) opens a session (2) with the provider (3) and requests refreshAndPersist synchronization. Any number of consumers can contact the provider and request synchronization service - the provider is not configured with any information about its consumer(s) - as long as any consumer satisfies the security requirements of the provider its synchronization requests will be satisfied. The synchronization request is essentially an extended LDAP search operation which defines the replication scope - using normal LDAP search parameters (base DN, scope,search filter and attributes) - thus the whole, or part, of the providers DIT may be updated in a replica synchronization session depending on the search criteria.

The provider (3) is not required to maintain per-consumer state information. Instead the provider periodically sends a synchronization cookie (SyncCookie 11) - this cookie contains a Change Sequence Number (contextCSN) - essentially a timestamp indicating the last change sent to this consumer and which may be viewed as a change or synchronization checkpoint. When a refreshAndPersist consumer (1) opens a session (2) with a provider (3) they must first synchronize the state of their DIT (or DIT fragement). Depending on how the consumer was initialised, when it initially opens a session (2) with the provider (3) it may not have a SyncCookie and therefore the scope of the changes is the entire DIT (or DIT fragment). A byproduct of this allows the consumer to start with an empty replication DIT. When a consumer (1) subsequently connects (2) to the provider (3) it will have a SyncCookie. In the case of a refreshAndPersist type of replication re-connection will occur after a failure of the provider, consumer or network each of which will terminate a connection which otherwise is maintained permanently.

During the synchronization process the provider (3) for the DIT will respond with one or both of two phases. The first is the present phase (13) and indicates those entries which are still in the DIT (or DIT fragment) and consists of:

1. For each entry that has changed (since the last synchronization) - the complete entry is sent including its DN and its UUID (entryUUID). The consumer can reliably update its entries from this data.

2. For each entry that has NOT changed (since the last synchronization) an empty entry with its UUID (entryUUID).

3. No messages are sent for entries which have been deleted. Theoretically the consumer may delete such non-referenced entries at the end of this phase.

In the delete phase (14):

1. The provider returns the DN and UUID (entryUUID) for each entry deleted since the last synchronization. The consumer can reliably delete these entries.

Whether both phases are required is determined by a number of additional techniques.

At the end of the synchronization phase(s) (12) the provider typically sends a SyncCookie (the current contextCSN) and MAINTAINS the session. Subsequent updates (4) - which may be changes, additions or deletions - to the provider's DIT (7) will be immediately sent (15) by the provider (3) to the consumer (3) where the replica DIT (8) can be updated. Changes or additions result in the complete entry (including all attributes) being transferred and the SyncCookie (11) is typically updated as well. The provider DIT (7) and consumer DIT (8) are maintained in synchronization with the replication cycle time approaching the transmission time between provider and consumer.

syncrepl refreshAndPersist Configuration Examples:

Master slapd.conf configuration (assumed host name master-ldap.example.com):

# slapd master
# global section
...

# database section
database bdb
...
# allows read access from consumer
# may need merging with other ACL's
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 
		 
# NOTE: 
# the provider configuration contains no reference to any consumers

# define as provider using the syncprov overlay
# (last directives in database section)
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

consumer slapd.conf configuration:

# slapd consumer
# global section


# database section
database bdb
...

# provider is ldap://master-ldap.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://master-ldap.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

7.2.1.2.3 OpenLDAP sysncrepl (N-Way) Multi-Master

OpenLDAP 2.4 introduced N-way Multi-Master support. In N-Way Multi-Master configurations any number of masters may be synchronized with one another. The functionality of replication has been previously described for refreshOnly and refreshAndPersist and is not repeated here. The following notes and configuration examples are unique to N-Way Multi-Mastering.

Note: When running N-Way Mutli-Mastering it is vital that the clocks on all the master (providers) are synchronized to the same time source, for example, they should all run NTP (Network Time Protocol).

In N-Way Multi-Mastering each provider of synchronization services is also a consumer of synchronization services as shown in Figure 7.2-4:

Figure 7.2-4: syncrepl N-Way Multi-Mastering

Figure 7.2-4 shows a 3-Way Multi-Master (1, 2, 3) Configuration. Each Master is configured - in its slapd.conf file (4, 5, 6) - as a provider (using the overlay syncprov directive) and as a consumer for all of the other masters (using the syncrepl directive). Each provider must be uniquely identified using a ServerID directive. Each provider is further, as noted above, synchronized to a common clock source. Thus the provider (1) of DIT (7) contains an overlay syncprov directive (the provider overlay) and two refreshAndPersist type syncrepl directives, one for each of the other providers (2, 3) as shown by the communication links (1-1, 1-2). Similarly each of the other providers has a similar configuration - a single provider capability and refreshAndPersist syncrepl directives for the other two masters.

In this configuration (and assuming that a refreshAndPersist type of synchronization is used - it is not clear why you would even want to use refreshOnly) then a modify (10) to any master will be propagated to all the other masters (providers).

Version 2.4 of N-Way Multi-Master replication does not support delta synchronization.

syncrepl refreshAndPersist Configuration Examples:

Assume three masters (ldap1.example.com, ldap2.example.com and ldap3.example.com) using syncrepl N-Way multi-master - then the three masters would have slapd.conf files as shown:

slapd.conf for ldap1.example.com:

# slapd master ldap1.example.com
# global section
...

# database section
database bdb
...
# allows read access from either consumer
# may need merging with other ACL's
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 
		 
# NOTE: 
# syncrepl directives for each of the other masters
# provider is ldap://ldap2.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap2.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

# provider is ldap://ldap3.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap3.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb
...
# syncprov specific indexing (add others as required)
index entryCSN eq
index entryUUID eq 
...

# define the provider to use the syncprov overlay
# (last directives in database section)
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

slapd.conf for ldap2.example.com:

# slapd master ldap2.example.com
# global section
...

# database section
database bdb
...
# allows read access from either consumer
# may need merging with other ACL's
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 
		 
# NOTE: 
# syncrepl directives for each of the other masters
# provider is ldap://ldap1.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap1.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

# provider is ldap://ldap3.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap3.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb
...
# syncprov specific indexing (add others as required)
index entryCSN eq
index entryUUID eq 
...
# define the provider to use the syncprov overlay
# (last directives in database section)
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

slapd.conf for ldap3.example.com:

# slapd master ldap3.example.com
# global section
...

# database section
database bdb
...
# allows read access from either consumer
# may need merging with other ACL's
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 
		 
# NOTE: 
# syncrepl directives for each of the other masters
# provider is ldap://ldap1.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap1.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

# provider is ldap://ldap2.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
syncrepl rid=000 
  provider=ldap://ldap2.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb

# syncprov specific indexing (add others as required)
index entryCSN eq
index entryUUID eq 
	
# define the provider to use the syncprov overlay
# (last directives in database section)
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

Notes:

1. Since the masters all replicate the same DIT the binddn is shown as having the same value throughout. This is perfectly permissable. However if any single server is subverted - all are subverted and a safer strategy may be to use a unique binddn entry in each server. This will also require change in the consumer access directives.

7.2.1.2.4 Session Logs, Access Logs and Delta-sync

It has all been simple up until this point. Now it gets a bit messy. All in the interests of reducing traffic flows between provider and consumer.

refreshOnly synchronization can have a considerable time lag before update propagation - depending on the interval parameter of the syncrepl directive. As the time interval between updates is reduced it effectively approaches refreshAndPersist but it incurs an initial synchronization on every re-connection. If the present phase is needed during synchronization then even if no changes have been made since the last synchronization, every unchanged entry will result in an empty entry (with its identifying entryUUID) which can take a considerable period of time.

In refreshAndPersist mode re-synchronization only occurs on initial connection (re-connection only occurs after a major error to the provider, consumer or network). However even in this mode updates to any attribute in an entry will cause the entire entry to transferred. For very big entries when a trivial change is made this can cause an unaceptable overhead.

Finally pathalogical LDAP implementations can create update problems. As a worst case assume an application is run daily that has the effect of changing a 1 octet value in every DIT entry. This would have the effect of transferring the entire DIT to all consumers - perhaps a tad inconvenient or perhaps even impossible within a 24 hour period.

OpenLDAP provides two methods to ameliorate these problems: the session log and the access log. The ojective of both methods is to minimise data transfer and in the case of Access Logs to provide what is called delta-synchronization.

Session Logs

The syncprov overlay has the ability to maintain a session log. The session log parameter takes the form:

syncprov-sessionlog ops
# where ops defines the number of
# operations that can be stored
# and is wrapped when full
# NOTE: version 2.3 showed a sid parameter
# which was removed in 2.4

# example of syncprov definition
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100

The session log is a memory based log and contains all operations (except add operations). Depending on the time period covered by the session log it may allow the provider to skip the optional present phase - thus significantly speeding up the synchroniziation process. If, for example, no changes have occurred in the session log since the last synchronization there is no need for a present phase. The session log may be used with any type (refreshOnly or refreshAndPersist) but is most useful with refreshOnly. If the session log is not big enough to cover the period since the last consumer synchronization request then a full re-synchronization squence (including present phase) is invoked.

Access Log (Delta Synchronization)

The access log provides a log of defined actions on a given DIT and makes then available in a related, but separate DIT. The access log feature is provided by the overlay accesslog directive.

In the normal case, replica synchronization operations perform the update using information in the DIT which is the subject of the search request contained within the Synchronization request. Alternatively the synchronization may be performed by targeting the syncrepl directive at the access log instead. Since the objects stored in the access log only contain the changes (including deletes, adds renames and modrn operations) the volume of data is significantly lower that than used when performing a full synchronization operation against the main DIT (or even a DIT fragment). Use of the access log is known as delta Replication or delta synchronization.

Use of this form of replication requires definition of an access log DIT in the provider and the the use of the logbase, logfilter and syncdata parameters of the syncrepl directive in the consumer as shown in the example below:

Delta Replication (Accesslog) Examples:

Provider configuration (assumed hostname of ldap1.example.com):

# slapd provider ldap3.example.com
# global section
...
# allow read access to main and access log DITs
# to consumer
access to *
     by dn.base="cn=admin,ou=people,dc=example,dc=com" read
     by * break 
...
# database section - replication DIT
# with suffix dc=example,dc=com
database bdb
suffix "dc=example,dc=com"
...
# syncprov specific indexing (add others as required)
index entryCSN eq
index entryUUID eq 
...
# define access log overlay and parameters
# prunes the accesslog every day:
# delete entries more than 2 days old
# log writes (covers add, delete, modify, modrdn)
# log only successful operations
# log has base suffix of cn=replog1
overlay accesslog
logdb cn=replog1
logops writes
logsuccess TRUE 
logpurge 2+00:00 1+00:00

# define the replica provider for this database
# (last directives in database section)
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10

# now define the DIT for the access log
# normal database definition
database bdb
...
suffix "cn=replog1"
# these are recommended to optimize accesslog
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart 
...
# the access log is also a provider
# inhibits the present phase of synchronization
# syncprov-reloadhint TRUE mandatory for delta sync
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE 

Consumer configuration:

# slapd master ldap3.example.com
# global section
...

# database section
database bdb
...

# NOTE: 
# syncrepl directive will use access log for 
# delta synchronization
# provider is ldap://ldap1.example.com:389,
# whole DIT (searchbase), all user attributes synchronized
# simple security with a SSHA encoded password
# binddn is used to authorize access in provider
# logbase references logdb in provider
# logfilter allows successful add, delete, modify, modrdn ops
# syncdata defines this to use accesslog format
syncrepl rid=000 
  provider=ldap://ldap1.example.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=example,dc=com"
  attrs=*
  bindmethod=simple
  binddn="cn=admin,ou=people,dc=example,dc=com"
  credentials={SSHA};gj;jjglsdknjghkjhgsdajggbjsdjb
  logbase="cn=accesslog"
  logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
  syncdata=accesslog 
...

7.2.2 ApacheDS Replication

One day real soon now ™

7.2.3 Synching DIT before Slurpd Replication

Before slurpd replication can occur the DITs (Master and Slave(s)) must be known to be in the same state. A manual sychronization process must be performed first as itemised below:

Note: In the case of OpenLDAP using syncrepl style replication a slave or multi-master can synchronize starting from an empty DIT. However the process defined below may also be used and depending on the volumes involved may offer a more efficient (quicker) starting point.

1. Stop the LDAP server that will contain the master DIT instance. This is essential to prevent further DIT updates.

2. Create an LDIF copy of the DIT to be replicated using the appropriate off-line tools for the LDAP server.

   For OpenLDAP use slapcat).

3. Configure the server running the master DIT instance.

   For OpenLDAP using slurp style replication - add the replica, replogfile and replicationinterval directives to the slapd.conf file. Do not restart the server at this time.

   Note: If running OpenLDAP using the run-time configuration feature (cn=config) the server must be active - detailed instructions to be supplied.

4. Move the LDIF file created in step 2 above to the the server(s) that will run the slave or multi-master instance.

5. Stop the LDAP server that will run the slate or multi-master instance.

6. Apply the LDIF file moved in step 4 to the server using the appropriate off-line tools for the LDAP server.

   For OpenLDAP use slapadd). Since the server has not be configured the -n (dbnum) option should be used.

7. Move the LDIF file created in step 2 above to the the server(s) that will run the slave or multi-master instance.

8. Configure the server that will run the slave or multi-master instance to act as either a slave or a multi-master.

   For OpenLDAP using slurpd style replication this will involve defining a database directive and all its associated directives (since the -n dbnum directive was used to add the DIT the order in which this is defined is very important - for replication specifically add the updatedn directive and the updateref directives.

   Note: If running OpenLDAP using the run-time configuration feature (cn=config) the server must be active - detailed instructions to be supplied.

9. If a master-slave configuration, start the server running the slave DIT instance. Confirm it is working. If a multi-master configuration start this copy of the master and confirm it is running.

10. Start the server running the master instance of the DIT or the second master in a multi-master configuration.

11. Perform a test transaction on the master (one of the masters in a multi-master configuration) and confirm it has been propogated to the slave (or second master). If not start looking at the logs. And panic. Always helps.

7.2.3.1 Synching DIT before syncrepl Replication

When initiating syncrepl replication there are two possible strategies:

1. Do nothing. After configuring the consumer there is no need to do anything further. The initial sychronization request will synchronize the replica from an empty state. Where the DIT is very large this may take an unacceptably long period of time.

2. Load an LDIF copy of the replica from the provider using slapadd before starting the replication. Depending on how this is done the initial synchronization may be minimal or non-existant. The following instructions itemize such a process when using a provider running OpenLDAP 2.2+ and assume that the provider has been configured for replication:

   1. Save an LDIF copy of the provider's DIT (using a Browser or even slapcat if using a BDB or HDB backend). There is no need to stop the provider since any inconsistencies during the saving process or between the state when the DIT was saved and loaded into the consumer will be resolved during initial synchronization.

   2. Move the LDIF file to the consumer location.

   3. Configure the consumer.

   4. Load the LDIF into the consumer using slapadd with the -w option to create the most up-to-date SyncCookie. Example:

      slapadd -l /path/to/provider/copy//ldif -w
      

   5. Start the consumer: