I randomly have the issue that after restoring, the FOG boot menu does not proceed at all.
Next to the entry “Boot from hard disk”, there is normally a counter from 3 to 1, and then it boots.
But in this case, there is no counter, nothing happens.
If I press Return, the boots process continues normally.
If reboot the machine once more, the “Boot from hard disk” also works again as expected.
What could be causing this?
I am running the latest FOG version.
@Tom-Elliott I really appreciate that you are putting effort into providing more frequent releases, which makes it easier for everyone to deploy new security fixes in time. Keep up the good work!
@Sebastian-Roth Thanks a lot for these troubleshooting tips!
I haven’t been using systemd-analyze, quite interesting.
It looks to me a bit like it’s something with the network. The SVG shows this in red:
Also, smbd was marked red by critical-chain. I have disabled this service now, it is not needed to start on boot really.
graphical.target @2min 2.330s
└─multi-user.target @2min 2.330s
└─smbd.service @2min 2.126s +203ms
└─network-online.target @2min 2.060s
└─network.target @2.156s
The output for all the PHP commands is identical on my machine.
I’ll wait for the next boot to see if disabling smbd maybe helped.
@Sebastian-Roth That’s nice - but as far as I understand, it still doesn’t protect against the new image needing 10GB, but there are only 9GB left.
Right?
I now also checked the other services, and the funny thing is that they all have the same long list of errors, but start successfully in the end (e.g. FOGImageReplicator). Also FOGSnapinHash itself.
But only FOGSnapinHash cycle is shown on boot.
I guess it could all be fixed by making /usr/bin/env php work, but it’s not clear to me how to do that.
@Sebastian-Roth I collected the journalctl log as you suggested and found this:
First there are many of this entry:
Nov 17 11:29:39 host systemd[1]: Started FOGSnapinHash.
Nov 17 11:29:39 host env[26388]: /usr/bin/env: ‘php’: No such file or directory
Nov 17 11:29:39 host systemd[1]: FOGSnapinHash.service: Main process exited, code=exited, status=127/n/a
Nov 17 11:29:39 host systemd[1]: FOGSnapinHash.service: Failed with result 'exit-code'.
Nov 17 11:29:40 host systemd[1]: FOGSnapinHash.service: Scheduled restart job, restart counter is at 762.
Nov 17 11:29:40 host systemd[1]: Stopped FOGSnapinHash.
The problem here is that /lib/systemd/system/FOGSnapinHash.service says:
ExecStart=/usr/bin/env php /opt/fog/service/FOGSnapinHash/FOGSnapinHash
but it doesn’t find php this way, probably because at this point a full shell isn’t available yet (?).
I remember reading about this issue in FOG project years ago, I don’t know if it was fixed?
There are more errors in the log afterwards, they all look like this:
Nov 17 11:29:42 host systemd[1]: Started FOGSnapinHash.
Nov 17 11:29:42 host env[27676]: PHP Fatal error: Uncaught Exception: Missing one or more extensions. in /var/www/fog/commons/init.php:439
Nov 17 11:29:42 host env[27676]: Stack trace:
Nov 17 11:29:42 host env[27676]: #0 /var/www/fog/commons/init.php(306): Initiator::_extCheck()
Nov 17 11:29:42 host env[27676]: #1 /var/www/fog/commons/base.inc.php(46): Initiator::startInit()
Nov 17 11:29:42 host env[27676]: #2 /opt/fog/service/lib/service_lib.php(22): require('...')
Nov 17 11:29:42 host env[27676]: #3 /opt/fog/service/FOGSnapinHash/FOGSnapinHash(25): require('...')
Nov 17 11:29:42 host env[27676]: #4 {main}
Nov 17 11:29:42 host env[27676]: thrown in /var/www/fog/commons/init.php on line 439
Nov 17 11:29:42 host systemd[1]: FOGSnapinHash.service: Main process exited, code=exited, status=255/EXCEPTION
Nov 17 11:29:42 host systemd[1]: FOGSnapinHash.service: Failed with result 'exit-code'.
Nov 17 11:29:43 host systemd[1]: FOGSnapinHash.service: Scheduled restart job, restart counter is at 764.
Nov 17 11:29:43 host systemd[1]: Stopped FOGSnapinHash.
I check the code in init.php, but the extension mentioned there are loaded in PHP (mysqli, gettext).
I assume it is related to the other error.
@sebastian-roth Unfortunately, this is all I could gather when the issue happened. It happens on every boot, but only on this server.
If you can let me know which logs I could collect, I’ll do that.
I have several Ubuntu 22.04 servers running the current dev-branch of FOG.
On one of these, the boot process loops a while and shows that it is starting and stopping FOGSnapinHash.
See the attached picture:
At some point, it stops doing that and finishes the boot.
FOG works normally then.
What could this be due to? How can I debug it?
@sebastian-roth Thanks for your work, we are in the process of switching to the dev-branch on Ubuntu 22 now and will report any additional bugs we find.
@zfeng +1 We are now moving our servers to Ubuntu 22.04 and rather want to install a somewhat stable release than a working-branch version that may change from day to day.
PR created in dev-branch
@wayne-workman I don’t have enough time to work on creatingmore tests, but I had a look at the PHP issue myself.
It is because of a breaking change in PHP 8:
https://php.watch/versions/8.0/named-parameters#named-params-call_user_func_array
It comes into play when call_user_func_array is used.
I’ll try to fix this myself as suggested in the link and create a PR on Github.
@wayne-workman I have looked at your daily tests, it looks like they only confirm that the installer runs successfully.
Do you have automated tests for the API? Please let me know when you have had a look at this.
@wayne-workman Actually the migration seemed to have worked very well overall, given that we used the dev branch.
The web UI works fine, it’s only the API that is returning errors.
So to me it looks more like a PHP problem not related to our instance specifically.
We will likely wait for your next release that officially supports Ubuntu 22, but we would very much appreciate if you could take a look at the API issue.
@wayne-workman Thanks for the great work on new distro releases.
Have you had a chance to look into this API issue yet?
https://forums.fogproject.org/topic/16370/api-doesn-t-work-in-1-5-9-dev-error-in-route-class-php
We have upgraded a Ubuntu 18 server that was running 1.5.7 to Ubuntu 22.04 and FOG 1.5.9 dev branch (required because of PHP version check).
The UI works, but any call to the API fails, the Apache error log shows this:
Got error 'PHP message: PHP Fatal error: Uncaught Error: Unknown named parameter $method in /var/www/fog/lib/router/route.class.php:351\nStack trace:\n#0 /var/www/fog/lib/router/route.class.php(231): Route::runMatches()\n#1 /var/www/fog/api/index.php(23): Route->__construct()\n#2 {main}\n thrown in /var/www/fog/lib/router/route.class.php on line 351'
Is this easy to fix?
Otherwise, do you already have a timeline for a new FOG release that works on Ubuntu 22?
@wayne-workman This call has to be made using the DELETE method.
I was looking for the API documentation, but the link https://news.fogproject.org/simplified-api-documentation/ leads to a page saying “Bad gateway”.
Is this a permanent problem?
Could you move the API docs to the Wiki please?
@Tom-Elliott Yes, this might work, but I didn’t want to spend too much time experimenting. If somebody wants to follow up on this, that’s fine, but I won’t. 
@Sebastian-Roth Don’t worry, I didn’t spend too much time on it. I have a use case where it would have been helpful to have this feature.
I have used secure erase for extra SSDs on Linux, but not the root disk. Most likely the primary disk that holds the OS is locked by the BIOS.
I am happy if you merge the PR, would be interesting to see if it works for some setups.