MariaDB Cluster Node Memory Overload Issue and OOM Kill
Hello, We are currently facing a critical issue with one of our MariaDB cluster nodes, replicated with Galera (MariaDB Ver 15.1 Distrib 10.4.30-MariaDB). The RAM usage by MariaDB on this Galera node is experiencing a drastic increase, eventually overloading the server. At this point, the OOM Killer process intervenes, terminating MariaDB, and consequently, rendering the site inaccessible.
Remarkably, MariaDB is demanding approximately 90GB of RAM, surpassing our configured limits. Despite our attempts to address the situation, including increasing the SWAP space to 150GB, with a total RAM capacity of 180GB, MariaDB continues to escalate its memory usage, exceeding set quotas.
We have also tried upgrading MariaDB to the latest version 10.4.32, but this had no effect. Efforts to limit MariaDB's memory parameters, such as the Galera cache mem_size and the number of invoked instances, have proven unsuccessful. Notably, there are no error messages in the logs, and we are unable to run checkmysql on the databases as MariaDB never completes its startup.
It's important to note that the other two nodes in the cluster are functioning normally, and the configuration parameters are identical.
Any guidance, insights, or recommendations you can provide will be highly appreciated.
Thank you for your prompt attention to this matter.