MariaDB fails to start after snapshot restore on Vultr

     

I had to restore from snapshot, because JetPack messed up my WordPress installation. Unfortunately, after the restore my website looked like this:

Error establishing a database connection

Gee, that’s nice. What’s up with the DB engine?

[root@noobient ~]# systemctl status mariadb.service
● mariadb.service - MariaDB 10.3.9 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─migrated-from-my.cnf-settings.conf
   Active: failed (Result: exit-code) since Sun 2018-08-19 23:15:13 CEST; 45s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 910 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 880 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 871 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 910 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"

Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [ERROR] Plugin 'InnoDB' init function returned error.
Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [ERROR] InnoDB: os_file_get_status() failed on './ibdata1'. Can't determine file permissions
Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [Note] Plugin 'FEEDBACK' is disabled.
Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [ERROR] Unknown/unsupported storage engine: InnoDB
Aug 19 23:15:13 noobient.com mysqld[910]: 2018-08-19 23:15:13 0 [ERROR] Aborting
Aug 19 23:15:13 noobient.com systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Aug 19 23:15:13 noobient.com systemd[1]: Failed to start MariaDB 10.3.9 database server.
Aug 19 23:15:13 noobient.com systemd[1]: Unit mariadb.service entered failed state.
Aug 19 23:15:13 noobient.com systemd[1]: mariadb.service failed.

Wonderful. What’s up with SELinux?

type=AVC msg=audit(1534713376.564:123): avc: denied { getattr } for pid=1271 comm=”mysqld” path=”/var/lib/mysql/ibdata1″ dev=”vda5″ ino=62194 scontext=system_u:system_r:mysqld_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file

What? How on Earth did it work before then?

[root@noobient ~]# restorecon -rv /var/lib/mysql/
restorecon reset /var/lib/mysql/mysql/innodb_table_stats.ibd context system_u:object_r:unlabeled_t:s0->system_u:object_r:mysqld_db_t:s0
restorecon reset /var/lib/mysql/mysql/innodb_index_stats.ibd context system_u:object_r:unlabeled_t:s0->system_u:object_r:mysqld_db_t:s0
restorecon reset /var/lib/mysql/ibdata1 context system_u:object_r:unlabeled_t:s0->system_u:object_r:mysqld_db_t:s0
restorecon reset /var/lib/mysql/ib_logfile0 context system_u:object_r:unlabeled_t:s0->system_u:object_r:mysqld_db_t:s0
restorecon reset /var/lib/mysql/wordpress/wp_options.ibd context system_u:object_r:unlabeled_t:s0->system_u:object_r:mysqld_db_t:s0

Wha-whaat? How does the fcontext reset after a snapshot restore? That’s beyond me. Nevertheless, it works now. I even ran a restorecon -rv / then, it reset a few dozen other files, too. Weird.