Could not image, Input/output error http://ipxe.org/Id0c6539

[Bob Henderson]

Just pulled from GIT this morning, on version 8413

Host will register fine, but when I either attempt an AutoReg task, or a quick image, I get :

Could not image, input/output error (http://ipxe.org/id0c6539) and then it reboots.

When I browse to http://fog-ip/fog/service/ipxe/boot.php?mac=client0mac I get:

#!ipxe
set fog-ip 10.1.0.24
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}

And that’s all.

Help? Let me know if you need other logs, etc.

[george1421]

Please also post the tail of the apache error log. That should give the devs a better clue what really happened.

[Bob Henderson]

Here is the apache log with everything from Today:

[Tue Jul 05 08:59:02.976170 2016] [:error] [pid 21672] [client 10.1.0.111:34431] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24
[Tue Jul 05 08:56:58.739643 2016] [:error] [pid 21671] [client 10.1.0.199:52938] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24
[Tue Jul 05 08:53:13.733738 2016] [:error] [pid 21671] [client 10.1.0.111:33092] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24
[Tue Jul 05 08:51:50.436565 2016] [core:notice] [pid 21668] AH00094: Command line: ‘/usr/sbin/apache2’
[Tue Jul 05 08:51:50.436466 2016] [mpm_prefork:notice] [pid 21668] AH00163: Apache/2.4.7 (Ubuntu) OpenSSL/1.0.1f configured – resuming normal operations
[Tue Jul 05 08:51:35.704898 2016] [mpm_prefork:notice] [pid 18204] AH00169: caught SIGTERM, shutting down
[Tue Jul 05 08:51:29.633638 2016] [:error] [pid 20670] [client 10.1.0.199:52574] PHP Warning: PDO::__construct(): MySQL server has gone away in /var/www/fog/lib/db/pdodb.class.php on line 32, referer: http://10.1.0.24/fog/management/index.php?node=task&sub=active
[Tue Jul 05 08:51:29.612471 2016] [:error] [pid 18211] [client 10.1.0.199:52573] PHP Warning: PDO::__construct(): MySQL server has gone away in /var/www/fog/lib/db/pdodb.class.php on line 32, referer: http://10.1.0.24/fog/management/index.php?node=task&sub=active
[Tue Jul 05 08:49:26.891831 2016] [:error] [pid 18334] [client 10.1.0.111:7252] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24
[Tue Jul 05 08:48:34.090955 2016] [:error] [pid 18276] [client 10.1.0.111:62502] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24
[Tue Jul 05 07:59:31.363747 2016] [:error] [pid 18209] [client 10.1.0.199:63368] PHP Strict Standards: Only variables should be passed by reference in /var/www/fog/lib/pages/schemaupdaterpage.class.php on line 46, referer: http://10.1.0.24/fog/management/index.php?node=schema
[Tue Jul 05 07:27:04.514030 2016] [core:notice] [pid 18204] AH00094: Command line: ‘/usr/sbin/apache2’
[Tue Jul 05 07:27:04.513958 2016] [mpm_prefork:notice] [pid 18204] AH00163: Apache/2.4.7 (Ubuntu) OpenSSL/1.0.1f configured – resuming normal operations
[Tue Jul 05 07:26:44.892635 2016] [mpm_prefork:notice] [pid 4381] AH00169: caught SIGTERM, shutting down

[george1421]

@Bob-Henderson said in Could not image, Input/output error http://ipxe.org/Id0c6539:

[Tue Jul 05 08:59:02.976170 2016] [:error] [pid 21672] [client 10.1.0.111:34431] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24

Just to be sure this was from the apache error log? If so I would say this is AN issue, but shouldn’t be used in the area of the code where you are seeing a problem.

[Tue Jul 05 08:59:02.976170 2016] [:error] [pid 21672] [client 10.1.0.111:34431] PHP Fatal error: Call to a member function isValid() on null in /var/www/fog/lib/plugins/location/hooks/changeitems.hook.php on line 24

[Bob Henderson]

@george1421 Just double checked, yep, that’s the Apache error log (error.log) pulled from the Log viewer under Fog configuration.

it also worries me where it stays the MySql server has gone away …

[george1421]

@Bob-Henderson sql server going away might cause your issue too. If you key in sudo ps aux|grep sql does it show the mysql server running?

[Bob Henderson]

@george1421

yep, and I can connect to it via the root login as well.

[george1421]

@Bob-Henderson I’m going to have to defer to the @Developers then so they can take a look at this error.

[Quazz]

Can you post default.ipxe here?

Just going to start over here.

Looks like a problem with the location plugin!

[Bob Henderson]

@Quazz
Content is:

#!ipxe
cpuid --ext 29 && set arch x86_64 || set arch i386
params
param mac0 ${net0/mac}
param arch ${arch}
param platform ${platform}
param product ${product}
param manufacturer ${product}
param ipxever ${version}
param filename ${filename}
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
:bootme
chain http://10.1.0.24/fog/service/ipxe/boot.php##params

[Bob Henderson]

@Quazz

Appears you’re right. I disabled the location plugin (we only have one location at the time anyways, so why bother) and it booted right up.

Now, I’m hoping to add a second location within the week here as a pure storage node, now that we finally got our high speed WAN in place. Hopefully this can be resolved by then, so I don’t have to worry about stuff pulling across the WAN!

[Wayne Workman]

@Bob-Henderson When you access the boot.php page, can you then immediately look at your apache error logs and see if there is anything relevant in there? They will have timestamps.
Web interface -> FOG Configuration -> Log Viewer -> Apache error logs

[Tom Elliott]

I’m fairly sure this is not related to the location plugin, at least not directly. I believe this is because the location that was trying to connect could not communicate with the database. I could be wrong, it happens a lot.

[Wayne Workman]

@Tom-Elliott Ah, right. That’s happening more often now.

@Bob-Henderson try to remotely connect to the fog server’s MySQL db using the fogstorage credentials and see what happens:
https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_MySQL#Testing_MySQL

[Bob Henderson]

@Wayne-Workman I haven’t touched it in a while, but where do I grab the fogstorage credentials?

I can connect locally as the root user and such

[Wayne Workman]

@Bob-Henderson said in Could not image, Input/output error http://ipxe.org/Id0c6539:

where do I grab the fogstorage credentials?

Web interface -> FOG Configuration -> FOG Settings -> FOG Storage Nodes -> FOG_STORAGENODE_MYSQLUSER and FOG_STORAGENODE_MYSQLPASS

[Bob Henderson]

When I attempt that, I get ERROR 2003 (HY000): Can’t connect to MySQL server on ‘10.1.0.24’ (111) . Tried from 2 other linux boxes.

[Tom Elliott]

@Bob-Henderson on your main server cd to /etc and run grep -rl bind-address .

You will need to figure out the right file but edit them and comment the bind-address lines so they read as #bind-address = 127.0.0.1 then save the files. Once done restart MySQL services and try again.

[Bob Henderson]

@Tom-Elliott

After getting rid of the localhost bind, I can login fine. No issues at all.

[Tom Elliott]

I’ll bet your location will work fine now too.