Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors

[JJ Fullmer]

I just upgraded to fog 1.5.9 (technically 1.5.9.2 from the dev-branch)
I went to image a new surface go and it got stuck on bzImage. I tried updating the bzImage to the latest (that was probably the one included 4.19.143) and there was no change.

I reverted to 4.18.11 (it was the closest to 4.18.6 I saw in the official kernels, which is what I looked for after reading this post https://forums.fogproject.org/topic/12465/microsoft-surface-go-usb-c-to-ethernet-adapter-compatibility/5 probably didn’t need to go that far back )

So maybe we took out the microsoft driver in the latest update? These were imaging fine in 1.5.8

Wait, now I’m getting an error after the image finished…

[Sebastian Roth]

@JJ-Fullmer said in Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors:

I will update my install shortly and re-image one of the machines. Will this also change my kernel? Will I need to remake that change for the surface ethernet adapter?

You will need to re-download the 5.6.18 kernel.

[JJ Fullmer]

@Tom-Elliott no worries, a mistake anyone could make.

@Sebastian-Roth I will update my install shortly and re-image one of the machines. Will this also change my kernel? Will I need to remake that change for the surface ethernet adapter?

[Sebastian Roth]

@george1421 said in Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors:

So for fog admins that have this issue would the instructions be to do a git pull on the local fog repo and then rerun the installer?

Yes but as I said, this is only in the latest dev-branch but not if you use the last official release 1.5.9…

[george1421]

@Sebastian-Roth said in Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors:

I will push a fix soon!

So for fog admins that have this issue would the instructions be to do a git pull on the local fog repo and then rerun the installer?

[Tom Elliott]

@Sebastian-Roth I’ve added the change just to keep it up to snuff. Sorry I missed that part. @JJ-Fullmer and @Sebastian-Roth

[Sebastian Roth]

@JJ-Fullmer Please try the following: Add back the three lines you commented and add some ->format(...) code to lines 296 and 297 to make them look like this:

...
        $ImageStartTime = self::niceDate($this->Task->get('checkInTime'))->format('Y-m-d H:i:s');
        $ImageEndTime = self::niceDate()->format('Y-m-d H:i:s');
...

I will push a fix soon!

[Sebastian Roth]

@JJ-Fullmer Thanks heaps. Somehow must have missed that one in the last days. I am onto this and will get back to you soon.

@george1421 It’s no in 1.5.9 but was added just after he release.

[george1421]

Mod Note: Moving this over to the Bug Reports forum.

I know that was a late add-in feature just before 1.5.9 was released. Its strange how/why it is impacting imaging through.

@Developers thoughts?

ref: https://forums.fogproject.org/topic/14749/feature-modification-request-image-task-email-addition

[JJ Fullmer]

@Tom-Elliott @Sebastian-Roth @george1421

I gave something else a try. I looked at the commits included in the dev_branch since 1.5.9 and found that Tom had added the machine details array in the taskqueue.class.php file.

First thing I tried was adding a primary user to the host, in case that being empty was somehow a problem. No dice.

Looking at it, it’s just a prettier more informative email array. But the parameters in that object must be used somewhere else, because I reverted the file to its last commit (made a .old version of it on my server, made a new version of it with the previous version’s contents) just to see if that change maybe broke something.

My debug task was waiting for me to hit enter at another updating database attempt, so I hit enter, and viola!

So now we gotta figure out why in the world that change in the email sent would break the database update to say the task is completed.

edit:
I debugged it a bit further. Restored the newer version of the file and started commenting things out one at a time to confirm the problem.

The code here is where the problem lies (line 293-315 of taskqueue.class.php found at https://github.com/FOGProject/fogproject/blob/99bb17b168e929796a8d76526f73bcfdac11bb4b/packages/web/lib/reg-task/taskqueue.class.php)

$Inventory = self::$Host->get('inventory');
        $mac = self::$Host->get('mac')->__toString();
        $ImageName = $this->Task->getImage()->get('name');
        $ImageStartTime = self::niceDate($this->Task->get('checkInTime'));
        $ImageEndTime = self::niceDate();
        $duration = self::diff($ImageStartTime, $ImageEndTime);
        $Snapins = implode(',', (array)$SnapinNames);
        $email = array(
            sprintf("%s:-\n", _('Machine Details')) => '',
            sprintf("\n%s: ", _('Host Name')) => self::$Host->get('name'),
            sprintf("\n%s: ", _('Computer Model')) => $Inventory->get('sysproduct'),
            sprintf("\n%s: ", _('Serial Number')) => $Inventory->get('sysserial'),
            sprintf("\n%s: ", _('MAC Address')) => $mac,
            "\n" => '',
            sprintf("\n%s: ", _('Image Used')) => $ImageName,
            sprintf("\n%s: ", _('Snapin Used')) => $Snapins,
            "\n" => '',
            sprintf("\n%s: ", _('Imaged By')) => $engineer,
            sprintf("\n%s: ", _('Imaged For')) => $primaryUser,
            sprintf("\n%s: ", _('Imaging Started')) => $ImageStartTime,
            sprintf("\n%s: ", _('Imaging Completed')) => $ImageEndTime,
            sprintf("\n%s: ", _('Imaging Duration')) => $duration
        );

If I comment out the last 3 pieces of the array and try again, the database updates and all is well again

$Inventory = self::$Host->get('inventory');
        $mac = self::$Host->get('mac')->__toString();
        $ImageName = $this->Task->getImage()->get('name');
        $ImageStartTime = self::niceDate($this->Task->get('checkInTime'));
        $ImageEndTime = self::niceDate();
        $duration = self::diff($ImageStartTime, $ImageEndTime);
        $Snapins = implode(',', (array)$SnapinNames);
        $email = array(
            sprintf("%s:-\n", _('Machine Details')) => '',
            sprintf("\n%s: ", _('Host Name')) => self::$Host->get('name'),
            sprintf("\n%s: ", _('Computer Model')) => $Inventory->get('sysproduct'),
            sprintf("\n%s: ", _('Serial Number')) => $Inventory->get('sysserial'),
            sprintf("\n%s: ", _('MAC Address')) => $mac,
            "\n" => '',
            sprintf("\n%s: ", _('Image Used')) => $ImageName,
            sprintf("\n%s: ", _('Snapin Used')) => $Snapins,
            "\n" => '',
            sprintf("\n%s: ", _('Imaged By')) => $engineer,
            sprintf("\n%s: ", _('Imaged For')) => $primaryUser
            //sprintf("\n%s: ", _('Imaging Started')) => $ImageStartTime,
            //sprintf("\n%s: ", _('Imaging Completed')) => $ImageEndTime,
            //sprintf("\n%s: ", _('Imaging Duration')) => $duration
        );

[JJ Fullmer]

@Sebastian-Roth @george1421
Ran the debug task.
It gets through everything normal all the way up to Task Complete
then it says
Updating Database.........................Failed
Then it repeats the Error Returned: Machine Details:- repeatedly. (It’s hard to count them with shift+ page up/page down but I believe it’s about 10 retries for updating the database if that helps to find where this is happening in the code)

[JJ Fullmer]

@Sebastian-Roth Another thing I just noticed is that the debug mode loaded from the main server (i.e. the task in the gui showed ‘working with fogServer’) But the other deploy task was working with my storage node

edit: This is probably nothing, the error happened in both cases.

[JJ Fullmer]

@Sebastian-Roth The same errors are happening on the other machine =(
What’s odd is that it’s claiming to be a database issue, but I also am adding the hosts through the pxe menu and they’re getting added to the database just fine.

[JJ Fullmer]

@Sebastian-Roth I waited until just now and it got into the debug console. I’ve never seen it take 7 minutes to load the init. (full discolsure, I wasn’t watching it the whole time and it may have rebooted itself and worked the second time around)
It also started working as soon as I started deploying on another machine. Gonna wait to see if the other machine gets through without error before starting any debugging. Are there any debug commands that would help in troubleshooting this?

[Sebastian Roth]

@JJ-Fullmer Ok, I should have looked at the code myself before saying that this is not part of the FOG code. I was wrong! This part of the code is trying to send out an email on finishing a task. Did you play with those settings recently?

Started the debug task and it’s stuck at loading the init.

How long did yo wait? Does it boot beyond that point when running a non-debug task?

[JJ Fullmer]

@Sebastian-Roth and @george1421
Started the debug task and it’s stuck at loading the init.

[JJ Fullmer]

@Sebastian-Roth said in Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors:

find /var/www/html/fog -type f -exec grep “Machine Details” {} /dev/null ;

 find /var/www/html/fog -type f -exec grep "Machine Details" {} /dev/null \;
/var/www/html/fog/management/languages/zh_CN.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/pt_BR.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/it_IT.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/es_ES.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/fr_FR.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/de_DE.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/en_US.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/messages.pot:msgid "Machine Details"
/var/www/html/fog/lib/reg-task/taskqueue.class.php:            sprintf("%s:-\n", _('Machine Details')) => '',

[Sebastian Roth]

@JJ-Fullmer Please run the following command as root on your FOG server: find /var/www/html/fog -type f -exec grep "Machine Details" {} /dev/null \;

[JJ Fullmer]

@Sebastian-Roth I don’t have any custom scripts at the fog level.
I also don’t think this is an issue with the surface device. I need to image another machine today and see if it’s happening everywhere or just on this one machine.
I’m gonna run a debug deploy on the surface this morning and a fresh image on another new machine.

[Sebastian Roth]

@JJ-Fullmer I am wondering about the Error returned: Machine Details:- and I can’t imagine this to be a specific issue with this Surface device. This is not part of the FOG code as far as I know. Custom script?