How Midships migrates Forgerock IDM from a Database to Forgerock DS without any disruption or interruption of service

By Ravivarma Baru, Senior Specialist at Midships (ravivarma.baru@midships.io)

This article is intended for anyone who is looking migrate PingIdentity (ForgeRock) Identity Manager from using a Database to Directory Service. Midships has been leading these migrations seamlessly since 2019 and we thought it was time to share :-)

---

Introduction:

ForgeRock’s Identity Management Solution (IDM) makes it easier for organizations to manage the identity of their members. IDM uses managed objects to represent different kinds of identities such as users, groups, devices, credentials, etc.

Attributes of these managed objects are usually synchronised with other systems including ForgeRock’s Directory Service(DS) which is used by Forgerock’s Access Manager(AM) as a identity store to enable authentication and authorisation of identities.

IDM requires a data repository backend such as a database to store the managed objects and metadata (used for syncing).

Problem Statement:

Prior to IDM v6, IDM used a database as it’s primary data repository.  Since v6, DS can also be used as IDM’s data repository and this both simplifies the solution by removing the operational overhead of managing a backend database and removes complexity around synchronizing which was previously required between the ForgeRock AM’s userstore and the IDM backend Database containing the managed objects.

Midships has significant experience helping our customers migrate their IDM from using a database to using DS seamlessly and with zero downtime.

In this blog, we will address how we do this.

Challenges:

The main challenges that we encountered at one of the banks at which this migration was performed:

1. How to design the DS servers, such that:

2. it can be consumed by both IDM as well as AM, without losing any data?

3. it doesn’t duplicate any data element, example IDM managed object and AM identity object can be shared?

4. How to deploy the newly designed DS servers, the IDM + AM servers (configuration updated to consume the new DS servers) without needing additional infrastructure or VMs?

5. How to reconcile the data from the IDM current backend DB as well as the current Forgerock DS(that was used by AM as idRepo) to the new DS servers?

6. How to use attributes that were defined in DB to hold certain data but in DS the corresponding attributes are operational and are not subjected to userchange? (More information in solution section)

7. How to keep the data in sync after performing initial reconciliation to all the way till the traffic is switched over to updated AM, IDM and DS instances?
   

Our Approach:

Category	Use-case / Issue	Solution
Design of DS	The new DS should hold identity data as shared objects such that both IDM and AM can refer a single identity entry without duplication of data.	The new data store (DS) is set up with two backends: userStore: This backend holds identity-related information such as users, credentials, and devices. idmRepo: IDM uses this backend to store IDM metadata (IDM Profile). Shared Objects: Explicit mappings are defined for IDM-managed objects. This ensures that both Access Management (AM) and IDM can reference the same organizational unit (OU) and share the identity entry. The schema is created or updated to include attributes required by both IDM and AM. There is no duplication of attributes or object classes. Additionally this also facilitates organizations to update schema of existing attributes or to update OU names to align with the realms used by AM, etc. We observed that organizations makes use of this migration to redesign their DS.
	The new DS must be highly performant to serve as both AM userStore and IDM backend idmRepo	DS is highly efficient for search operations. With shared objects used for IDM and AM, even with DS divided into two backends, JVM cache usage remains minimal. However, we recommend allocating extra heap space to the JVM after thorough performance testing for optimal resource use.
Deployment of DS	Organizations, particularly financial institutions, often opt to set up a new DS cluster separately to allow for a seamless rollback if needed. This setup is straightforward. However, when dealing with multiple data centres (DCs), one DC can be isolated for deploying changes. Here, we'll explore the process of isolating one DC and deploying the new DS setup. [Note: While we use 2 DCs as an example, this approach applies to scenarios with more than 2 DCs as well.]	For this requirement, we break the replication topology between the two DCs.   Though DS 7.x comes with an option of removing the bootstrap replication servers, removing the bootstrap replication servers of DC1 from DC2 and vice-versa, is insufficient, If DS servers are restarted within the purge delay period (default 3 days), they will re-join the replication topology. To fully isolate the replication topology, additional measures are taken along with removing the bootstrap replication servers of one DC from the other. The followed ways to prevent the network connectivity: The /etc/hosts file of the DC1 DS servers are updated, such that they don't resolve the actual DNS addresses of DC2 DS servers but resolve to a gibberish ip address that doesn’t connect. Optionally we disable the DNS caching & restart the DNS resolver service, along with updated /etc/hosts file, as DNS resolver service of the VM caches the ip addresses, to ensure that change in /etc/hosts picked up, we do the ping test as well to confirm. The above two options will help in most cases and still if connectivity or address resolution works, the replication port of the new DS servers are updated to completely block the connection between DC1 and DC2 replication servers. Subsequently, a single DS server is deployed in DC2 according to the updated design, after cutting the connectivity with active DS cluster of DC1.
Data Migration	Reconciliation of Data from DB	The IDM instance of DC2 is taken out of traffic and is deployed with backend pointing to the new DS deployed as in above step. The IDM instance also configured to have a JDBC connector to connect to the DB used by the current active system, to migrate the data. Mappings are used to reconcile data from DB to newly created DS. The managed object tables of DB are pointed to the shared OUs created in the new DS.
	Migrating system (operational) attributes, such as account creation time and password change time. Issue with system attributes is that they cannot be modified directly and are created internally by DS. Organizations wants to prefer the original values from DB to new DS, as they can be consumed by other teams, let’s say a call-centre assisting a customer.	To preserve the state of the data from DB, such as user account creation or password changed time etc, instead of creation of custom attributes, use of the equivalent attributes in DS createTimestmap, pwdChangedTime are preferred, to avoid duplicacy. But these are operation attributes and are not allowed for external modification. Since there is synchronization in old system between the DB and the DS in old system, which is used by AM as an userstore, it has the same state of data like the DB.   The old DS is used to get the values, i.e we exported the attributes(dn, createTimestamp, pwdChangedTime) data as an ldif from old system. The entire data set is exported from the new system after reconciliation. A simple and performant program is created which scans both the ldifs and updates the new data set the required operational attributes from old system. The updated ldif is imported to new DS, which will override the entire data including the operational attributes, thus preserving the account creation and password update timestamps in sync with old system. Subsequently, other DS’es are provisioned of DC2 in the topology. Then the updated AM is deployed as well in DC2 to consume DS’es of DC2.
	Delta Data Migration	From the start of reconciliation, to migrating operational attributes and deployment of other DSes in DC2, there are changes that happened on the active cluster of DS1. So, before start of reconciliation, we note down the time stamp. So, we can migrate this delta data to the new cluster before we made it live. Another ldif export from the active DS server of DC1 was taken and the newly created accounts, password updates, credentials change etc from the start of reconciliation till the current time is retrieved by means of a simple program that we developed. These records are reconciled as individual entries, using IDM's single record reconciliation API, also parameter to wait till the reconciliation is complete is added, to be sure. Additionally checks such as validating the records that are identified to be newly created or changes in passwords is verified with the logs from the monitoring system.
Miscallenous	queryId to queryFilter	IDM with DB as a backend repository supports queryId, a custom queryFilter to DB tables and that can be exposed directly via rest to the IDM consumers, this is not possible with DS as a backend for IDM. The proxy servers in-front of IDM is updated to re-write the query parameters from queryId to respective queryFilter, to prevent any change to consumers of IDM.
	IDM Schema	To mitigate the issues encountered during reconciliation and to standardize overall schema of the managed objects, IDM schema is updated.
	Making the changes live	Once the Data is migrated, then the traffic is switched to DC2, the updated DC, where new DS’es, AM and IDM are deployed. Post successful business testing, All DC1components are deployed enabling back the connectivity with DC2.

Conclusion:

Organisations and Enterprises often deem that migrating the IDM backend repo from JDBC to ForgeRock DS is difficult. Particularly with financial institutions, there is always an inertia of thought to stay where they are at, especially when it comes to customer or business data and it is impossible to migrate backend repo without downtime, without losing data or without spending additional cost on new infrastructure. But as discussed above having a holistic approach and defining solid automation will make the transition easy. One caveat though in migrating to the DS is it doesn’t support workflows, unlike a DB backend.

In summary, the migration of the IDM backend repository from DB to DS can be achieved with zero downtime by designing the DS, planning ways to synchronize data, identifying the functional issues in lower environments, fixing them and automating the processes that can be automated.

If you would like to learn more, please don't hesitate to contact me.

Ravi