Update to RC10 breaks mysql-server install

[neodawg]

Updating from RC8 to RC10, getting this issue. CentOS 6.8

 * Configuring services

 * Setting up fog user.........................................Already setup
 * Setting up fog password.....................................OK
 * Stopping FOGMulticastManager Service........................OK
 * Stopping FOGImageReplicator Service.........................OK
 * Stopping FOGSnapinReplicator Service........................OK
 * Stopping FOGScheduler Service...............................OK
 * Stopping FOGPingHosts Service...............................OK
 * Setting up and starting MySQL...............................Failed!
[root@fog bin]# service mysqld restart
/etc/init.d/mysqld: line 103: syntax error near unexpected token `fi'
/etc/init.d/mysqld: line 103: ` fi'

yum reinstall mysql-server fixes the issue.

uname -r
2.6.32-642.4.2.el6.x86_64

[Tom Elliott]

@neodawg this isn’t something fog is doing. We make 0 edits to the MySQL init scripts. I susprct, however, maybe just a bad time updating mat have caused this temporarily? If you rerun the installer does it break it again?

[neodawg]

Yep, did yum update on the server too, was working fine on RC8, ran the installer for RC10, broke it, fixed it and redid the svn up command on the fog install folder and tried installing again. Got to the step above and died and listed the restart to show it was broken.

Here is the mysql.log file that might tell you more, i suspect it has something to do with the new performance tweaks you have made?

160909  4:01:05  InnoDB: Starting shutdown...
160909  4:01:07  InnoDB: Shutdown completed; log sequence number 3694098
160909  4:01:07 [Note] /usr/libexec/mysqld: Shutdown complete

160909 04:01:07 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160909 04:01:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160909  4:01:07 [Note] /usr/libexec/mysqld (mysqld 5.5.52) starting as process 15752 ...
160909  4:01:07 [Note] Plugin 'FEDERATED' is disabled.
160909  4:01:07 InnoDB: The InnoDB memory heap is disabled
160909  4:01:07 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160909  4:01:07 InnoDB: Compressed tables use zlib 1.2.3
160909  4:01:07 InnoDB: Using Linux native AIO
160909  4:01:07 InnoDB: Initializing buffer pool, size = 128.0M
160909  4:01:07 InnoDB: Completed initialization of buffer pool
160909  4:01:07 InnoDB: highest supported file format is Barracuda.
160909  4:01:07  InnoDB: Waiting for the background threads to start
160909  4:01:08 InnoDB: 5.5.52 started; log sequence number 3694098
160909  4:01:08 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160909  4:01:08 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160909  4:01:08 [Note] Server socket created on IP: '0.0.0.0'.
160909  4:01:08 [ERROR] Missing system table mysql.proxies_priv; please run mysql_upgrade to create it
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_current' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_history' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_history_long' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'setup_consumers' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'setup_instruments' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'setup_timers' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'performance_timers' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'threads' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_thread_by_event_name' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_instance' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'events_waits_summary_global_by_event_name' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'file_summary_by_event_name' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'file_summary_by_instance' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'mutex_instances' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'rwlock_instances' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'cond_instances' has the wrong structure
160909  4:01:08 [ERROR] Native table 'performance_schema'.'file_instances' has the wrong structure
160909  4:01:08 [Note] Event Scheduler: Loaded 0 events
160909  4:01:08 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi

In digging into the log some more, it looks like this error occured prior to me messing with RC10, part of my inital fixing was run the update like it says. Now I no longer see those performance_schema errors.

I can try test installing RC10 again if you like.

[Tom Elliott]

@neodawg what’s in the installer error log for this? It should be under trunk/bin/error_logs/ and should show the version installed in the name of the file.

See the messages you’re showing appear to be from the init of MySQL, as the restart failed after you were out of the installer. This failure is before any performance updates are even called. So if you try installing mysql-server with: yum -y --enablerepo=remi install mysql-server and restart the service does it give the same error? I’m suspecting the remi install of the MySQL startup script is what’s doing this. Maybe it will be fixed tomorrow as well? This is definitely not something the installer has done from bad code on my side. If anything it’s only because I use the remi repo.

[Wayne Workman]

@Tom-Elliott remi has been having some problems lately. I’ve actually been kicking around the idea of creating a fog repo for x86_64 and Arm, for debian and CentOS. Still looking into it and learning, but the idea is to provide exactly what fog needs, and any needed dependencies for those things. One repo for fog, based on os and cpu, from one place.

[Tom Elliott]

@Wayne-Workman The idea of our own repo is nice but damn near impossible to maintain. We would need a repo not only per arch but also for the different distributions fog installs on. This means a separate repo for debian/Ubuntu, red hat, fedora, and centos at a minimum and one for all those for at least 32 bit and 64 bit. If you’re going to do all that the maintaining of the installer script would be moot, but it puts a whole new level of complexity to the whole thing.

[Wayne Workman]

@Tom-Elliott yeah, a whole lot of work. I would automate it all. I ordered two nice core i5 towers that I plan to put to work, this could be one of the tasks for one of them. And as far as the installer goes, I’d vote not to change it. The idea is to use said repo to install needed stuff before hand so that at least stuff is installed.

[Tom Elliott]

Can you try installing again @neodawg

[ludeeus]

I have the same problem as @neodawg
Currently setting up an new FOG-Server, and it fails everytime on Mysql…

RC10 from trunk
CentOS 6.8
rev 5955

[Tom Elliott]

@ludeeus like I said, this has nothing to do with rc10, it’s a repo issue. I can’t get to my dev systems right now but can you edit the file in trunk/lib/red hat/config.sh

Remove the --enablerepo=remi,remi-php56 from the package lines throughout the file?

[Tom Elliott]

@Tom-Elliott correction change line 20of the file to just use epel

[Wayne Workman]

Just ran into the same issue in the OP. Plus a ton of other packages didn’t install as well.

@Tom-Elliott Remi RPM isn’t getting installed for whatever reason during the installer run, or the installer is not using it to install packages for whatever reason. It could be due to version changes. I.E. Remi rpm PHP is now on version 7.0.10

The solution I’ve done is just to manually install the remi rpm and then simply enable it perminantly. Steps for CentOS/Fedora/RHEL can be found with the configuration wizard here:
http://rpms.famillecollet.com/

• Choose configuration wizard,
• Select your OS
• Pick PHP 7.0.x
• pick Single Version
• follow steps displayed to install remi rpm and to enable it. CentOS/RHEL will also display steps for installing the epel rpm, which you should do if you’re on one of those.

After that, install all the packages FOG needs (on Fedora) with:

dnf install bc curl gcc gcc-c++ gzip httpd lftp m4 make mariadb mariadb-server mod_ssl net-tools nfs-utils php php-bcmath php-cli php-common php-fpm php-gd php-mbstring php-mcrypt php-mysqlnd php-process tar tftp-server vsftpd wget xinetd

The packages needed is listed in the installer. If you’re on something besides Fedora, the installer may list other things, just copy those and change the above command with them.

or yum instead of dnf if you’re on CentOS or RHEL. Then run the fog installer and it should work.

[neodawg]

This can be marked as solved.

Commenting out the lines on the file fixed it for me.

Reference:
https://forums.fogproject.org/topic/8576/upgrade-1-2-1-3-rc10-error-mysql/2

[Tom Elliott]

https://forums.fogproject.org/topic/8758/fog-installation-error-cannot-restart-mysql/9#

I’ve found and fixed the cause of the commenting randomly appearing.