#1 2019-05-07 13:08:15

schwim

Member

From: Coastal VA, Murica

Registered: 2015-09-29

Posts: 342

Website

[SOLVED] Can not start crashed MySQL process

Hi there guys,

I've got a server that went all screwy and now I'm having problems getting MySQL to start back up.  It looks like it crashed during db backup since that backup was partial and unzipped. When trying to restart the service, I get:

root@server1:~# /etc/init.d/mysql start
[....] Starting mysql (via systemctl): mysql.serviceJob for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
failed!

And here's my results of systemctl status and journalctl: https://hastebin.com/epuyoyefip.pl

That's a LOT of information and I'm having a terrible time trying to google through it.  I would be eternally grateful if someone could guide me through getting this back online as it's caused the server to be down for almost a week now.

Thanks for your time!

Last edited by schwim (2019-05-07 14:04:08)

---

Schw.im! A social site with an identity crisis.

linux_user

50/50 it's a spam account

Registered: 2016-11-04

Posts: 484

Re: [SOLVED] Can not start crashed MySQL process

Posted by mistake, admin delete it.

Last edited by linux_user (2019-05-07 13:33:56)

---

"Blind faith to authority is the greatest enemy of truth."

iMBeCil

WAAAT?

From: Edrychwch o'ch cwmpas

Registered: 2015-09-29

Posts: 767

Re: [SOLVED] Can not start crashed MySQL process

Your systemctl status and journalctl are not really informative (not your mistake, of course).

I would suspect disk space problems ... Please post output of:

$ df -h

Furthermore, additional information can be found in mysql (i.e. mariadb) log files. On debian, they are usually in /var/log/mysql/ directory. Please examine, and possibly post, output of (as root):

cat /var/log/mysql/error.log
cat /var/log/mysql/mysql.log

Last edited by iMBeCil (2019-05-07 13:37:19)

---

Postpone all your duties; if you die, you won't have to do them ..

schwim

Member

From: Coastal VA, Murica

Registered: 2015-09-29

Posts: 342

Website

Re: [SOLVED] Can not start crashed MySQL process

I most definitely had disk space issues but deleted a bunch in an effort to resolve this issue.  This is where I stand now with disk space:

root@server1:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            992M     0  992M   0% /dev
tmpfs           201M   23M  179M  12% /run
/dev/sda         49G   33G   14G  71% /
tmpfs          1003M     0 1003M   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs          1003M     0 1003M   0% /sys/fs/cgroup
tmpfs           201M     0  201M   0% /run/user/0
root@server1:~#

Space shows as sufficient now but issue persists.

---

Schw.im! A social site with an identity crisis.

iMBeCil

WAAAT?

From: Edrychwch o'ch cwmpas

Registered: 2015-09-29

Posts: 767

Re: [SOLVED] Can not start crashed MySQL process

^Yeah, the disk space seems OK.

Now, try to examine mysql log files, as I explained above.

EDIT: also, be sure that mysqld isn't already running by issuing 'ps ax | grep mysqld'.

Last edited by iMBeCil (2019-05-07 13:47:30)

---

Postpone all your duties; if you die, you won't have to do them ..

schwim

Member

From: Coastal VA, Murica

Registered: 2015-09-29

Posts: 342

Website

Re: [SOLVED] Can not start crashed MySQL process

Sorry, I missed that part.  Here's the logs right after a start attempt:

root@server1:~# ps ax | grep mysqld
1757 pts/0    S+     0:00 grep mysqld
root@server1:~#

root@server1:~# cat /var/log/mysql/error.log
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: The InnoDB memory heap is disabled
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Using Linux native AIO
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Using SSE crc32 instructions
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Completed initialization of buffer pool
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Highest supported file format is Barracuda.
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: 128 rollback segment(s) are active.
2019-05-07 13:57:02 139735515939264 [Note] InnoDB: Waiting for purge to start
2019-05-07 13:57:02 139735515939264 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.41-84.1 started; log sequence number 5422761850
2019-05-07 13:57:03 139735515939264 [Note] Plugin 'FEEDBACK' is disabled.
2019-05-07 13:57:03 139735515939264 [Note] Recovering after a crash using tc.log
2019-05-07 13:57:03 139735515939264 [ERROR] Can't init tc log
2019-05-07 13:57:03 139735515939264 [ERROR] Aborting

root@server1:~#

root@server1:~# cat /var/log/mysql/mysql.log
cat: /var/log/mysql/mysql.log: No such file or directory
root@server1:~#

Last edited by schwim (2019-05-07 14:00:24)

---

Schw.im! A social site with an identity crisis.

schwim

Member

From: Coastal VA, Murica

Registered: 2015-09-29

Posts: 342

Website

Re: [SOLVED] Can not start crashed MySQL process

Ah, it was that tc.log file , it was unwritable.  I deleted it and mysql started right up!

Thank you for once again getting me out of my pickle!

Last edited by schwim (2019-05-07 14:06:18)

---

Schw.im! A social site with an identity crisis.

iMBeCil

WAAAT?

From: Edrychwch o'ch cwmpas

Registered: 2015-09-29

Posts: 767

Re: [SOLVED] Can not start crashed MySQL process

I have no idea what is tc.log, but I'm glad you solved your problem  You're welcome.

Last edited by iMBeCil (2019-05-07 14:24:46)

---

Postpone all your duties; if you die, you won't have to do them ..

schwim

Member

From: Coastal VA, Murica

Registered: 2015-09-29

Posts: 342

Website

Re: [SOLVED] Can not start crashed MySQL process

I don't either but I would have never found it without your instruction of which logs to look at so you're the MVP

---

Schw.im! A social site with an identity crisis.