Optimizing cPanel Managed Services for High-Availability Clusters?

Hey everyone,

We're currently scaling our 'Website Maintenance & cPanel Management Services' to cater to high-traffic clients who demand true high-availability (HA) setups. Our goal is to ensure maximum infrastructure reliability and uptime, but we're hitting a few roadblocks integrating cPanel seamlessly into active-passive or active-active HA clusters.

The core technical challenge lies with cPanel's native synchronization mechanisms and resource management. Specifically, we're encountering persistent issues with database replication and shared storage consistency across nodes. For instance, when attempting to failover or switch active nodes, we often see replication lags or outright failures, particularly with MySQL/MariaDB. Here's a typical snippet from our error logs after a simulated failover:

[2023-10-27 14:35:01] cpanel::cpanel::datastore::mysql::replication_status: Replication stopped. Slave_IO_Running: No, Slave_SQL_Running: No
[2023-10-27 14:35:01] cpanel::cpanel::datastore::mysql::replication_status: Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master'
[2023-10-27 14:35:01] cpanel::cpanel::datastore::mysql::replication_status: Last_SQL_Error: Error 'Table 'db_name.table_name' doesn't exist' on query. Default database: 'db_name'. Query: 'INSERT INTO `table_name` ...'
[2023-10-27 14:35:02] cpanel::cpanel::server::services: Service 'mysql' is not running on this node. Attempting restart...
[2023-10-27 14:35:05] cpanel::cpanel::server::services: Service 'mysql' restart failed. PID file not found.

We're looking for insights on best practices, custom configurations, or any third-party tools that effectively enable cPanel to operate robustly within a multi-node HA environment without significant manual intervention or data inconsistencies. Has anyone successfully implemented cPanel in a production-grade HA cluster and can share their setup architecture or common pitfalls to avoid? Anyone faced this before?