Streamlining MongoDB Migration to the Cloud

MongoDB Migration from On-Prem to Cloud

• The database size was in terabytes, with an oplog retention limited to three days due to restricted server storage capacity. This made data migration difficult, as the transfer time over the MPLS line exceeded the oplog retention period.
• Even after addressing storage limitations, cloud performance remained suboptimal.
• The migration had to be completed within a tight deadline to prevent the server storage from being fully utilized.
• The customer was using MongoDB version 3.2, which had reached end-of-life, leaving no support available for the outdated libraries. This posed an additional challenge in the migration process.

IOPSHub identified the issues with the oplog size and data transfer, recommending the use of an external tool called CloudEndure. This tool enabled block-level data migration, completing the transfer within a day.

While the migration was successful, the application faced performance issues, with transaction times doubling. IOPSHub thoroughly optimized the older MongoDB version 3.2 by adjusting settings such as PID limits, file system readahead, transparent huge pages, swap space, keepalive, and other performance-enhancing configurations.

The team then tackled the challenges of upgrading MongoDB, ensuring application compatibility by sequentially upgrading from version 3.2 to 3.4, 3.6, 4.0, 4.2, 4.4, and finally to 5.0, as a direct upgrade to the latest version was not possible.

Ultimately, IOPSHub achieved optimal performance on the cloud, completing a challenging migration while continuously improving cost control, performance, and security.