Comments - Process: 1037 ExecStart=/usr/sbin/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)

1 year, 9 months ago Nils Scheunemann

Ok here is the result from the sytemctl status mariadb.service command:

pi@OIT-ZabbixPi:~ $ systemctl status mariadb.service ● mariadb.service - MariaDB 10.5.15 database server Loaded: loaded (/etc/systemd/system/mariadb.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2022-07-12 07:17:14 CEST; 3h 6min ago Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Process: 491 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS) Process: 516 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 522 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || e> Process: 610 ExecStart=/usr/sbin/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE) Main PID: 610 (code=exited, status=1/FAILURE) Status: "MariaDB server is down" CPU: 2.307s

Jul 12 07:17:13 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:13 0 [Note] InnoDB: Starting shutdown... Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Plugin 'InnoDB' init function returned error. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [Note] Plugin 'FEEDBACK' is disabled. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Unknown/unsupported storage engine: InnoDB Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Aborting Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Failed with result 'exit-code'. Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: Failed to start MariaDB 10.5.15 database server. Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Consumed 2.307s CPU time.

and the result from the journalctl -u mariadb.service -n 40 command:

pi@OIT-ZabbixPi:~ $ journalctl -u mariadb.service -n 40 -- Journal begins at Fri 2022-01-28 02:22:52 CET, ends at Tue 2022-07-12 10:17:01 CEST. -- Jul 11 13:33:35 OIT-ZabbixPi systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Jul 11 13:33:35 OIT-ZabbixPi systemd[1]: mariadb.service: Failed with result 'exit-code'. Jul 11 13:33:35 OIT-ZabbixPi systemd[1]: Failed to start MariaDB 10.5.15 database server. Jul 11 13:33:36 OIT-ZabbixPi systemd[1]: mariadb.service: Consumed 1.327s CPU time. -- Boot 2b8ca3a1b40746bc9005f8c54f04f88e -- Jul 12 07:17:05 OIT-ZabbixPi systemd[1]: Starting MariaDB 10.5.15 database server... Jul 12 07:17:06 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:06 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1) starting as process 610 ... Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Uses event mutexes Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Number of pools: 1 Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Using generic crc32 instructions Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Using Linux native AIO Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Completed initialization of buffer pool Jul 12 07:17:07 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:07 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=87766966505,87766966505 Jul 12 07:17:08 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:08 0 [Note] InnoDB: Starting final batch to recover 4124 pages from redo log. Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=0], should be [page id: space=75, page number=262144] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=65], should be [page id: space=75, page number=262209] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Not applying INSERT_REUSE_DYNAMIC due to corruption on [page id: space=75, page number=262209] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Set innodb_force_recovery=1 to ignore corruption. Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=128], should be [page id: space=75, page number=262272] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Not applying INSERT_HEAP_DYNAMIC due to corruption on [page id: space=75, page number=262272] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Set innodb_force_recovery=1 to ignore corruption. Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=192], should be [page id: space=75, page number=262336] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Not applying INSERT_HEAP_DYNAMIC due to corruption on [page id: space=75, page number=262336] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Set innodb_force_recovery=1 to ignore corruption. Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=320], should be [page id: space=75, page number=262464] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Not applying INSERT_HEAP_DYNAMIC due to corruption on [page id: space=75, page number=262464] Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Set innodb_force_recovery=1 to ignore corruption. Jul 12 07:17:11 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:11 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=75, page number=384], should be [page id: space=75, page number=262528] Jul 12 07:17:13 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:13 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption Jul 12 07:17:13 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:13 0 [Note] InnoDB: Starting shutdown... Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Plugin 'InnoDB' init function returned error. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [Note] Plugin 'FEEDBACK' is disabled. Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Unknown/unsupported storage engine: InnoDB Jul 12 07:17:14 OIT-ZabbixPi mariadbd[610]: 2022-07-12 7:17:14 0 [ERROR] Aborting Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Failed with result 'exit-code'. Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: Failed to start MariaDB 10.5.15 database server. Jul 12 07:17:14 OIT-ZabbixPi systemd[1]: mariadb.service: Consumed 2.307s CPU time. pi@OIT-ZabbixPi:~ $

thanks for your help.

 
1 year, 9 months ago Daniel Black

Hi,

This looks like a bug and not one we've encountered before.

Can you please report a new bug on https://jira.mariadb.org? If you are willing to share the database contents you can do a private upload to https://mariadb.com/kb/en/meta/mariadb-ftp-server/.

Its failing during crash recovery however what would be useful is the original cause of the crash. This will be further back in the logs and including it in the bug report would be useful. This will be the MariaDB start up that doesn't include "Starting crash recovery"which will continue to a shutdown that doesn't say '[ERROR] Plugin 'InnoDB' init function returned error".

For your recovery I can't offer a suggestion beyond restore from backup.

 
Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.