Feature request: a way to handle USB Ethernet adapters shared between many hosts
-
@Tom-Elliott Thanks so much!
-
@Tom-Elliott Thanks a lot, Tom. I tested it today on 1.6.0.14, with the two files from PR #2100 applied by hand (
SmbiosIdentity.phpandIpxeBootMenu.php) andFOG_HOST_IDENTIFY_SMBIOSset toenforce.I used two Proxmox VMs, both already registered, each with its own UUID. I gave one of them the MAC of the other and booted it by PXE. The menu said
Host is registered as bios-test!, which is the right one, and the log shows what happened:FOG host identity (enforce): MAC resolved "uefi-test" (id 658); SMBIOS resolved "bios-test" (id 659) on sysuuid=795efc04-...; SMBIOS winsBoth inventories were left alone and the MAC stayed on the first host. Without the patch the same thing overwrote the UUID of the host that owned the MAC, as you described.
I also hit the gap you mentioned, with a machine FOG did not know yet. Before registering the second VM I gave it the MAC of the first, and FOG took it for the first host. The agent did the same: the enrollment request of the second VM got attached to the first host (reason
known-host-no-agent). I don’t know if that last part is worth a look on your side.I have only tested VMs so far. The real case is a few laptops that share one USB adapter,
-
@lperoma Thank you for testing this so thoroughly. The
SMBIOS winsline and the untouched inventories are exactly the expected result.The agent part is worth a look, yes. Agent enrollment falls back to the MAC when the firmware matches no host. So an unregistered machine with a known MAC lands on that MAC’s host. Within the deploy window after an image (
FOG_AGENT_ENROLL_DEPLOY_WINDOW, 24 hours by default), that request can be approved without an admin click. I am fixing that so a machine whose serial contradicts the host is never bound to it by MAC.Until the fix is out, set
FOG_AGENT_ENROLL_DEPLOY_WINDOWto 0 if you image through shared adapters. Every enrollment then waits for an approval or a token.Your last post looks cut off after “a few laptops that share one USB adapter,”. Did you mean to add more?
-
@lperoma The agent fix is done (PR #2104).
What changes: suppose the firmware of an enrolling machine matches no host, and only its MAC finds one. If the machine’s serial contradicts the serial stored for that host, FOG no longer binds it as that host’s first agent. The request is recorded as an identity conflict. The deploy window never approves a conflict, so it waits for an admin. An admin can still approve it on purpose, for example after a real motherboard swap. The directory identity check treats the same case as a conflict too.
What does not change:
- An enroll token still approves the request. A token is an admin’s explicit vouch.
- A host with no inventory row has no stored serial to compare, so its MAC still binds as before. Run a full inventory on hosts that share an adapter.
- A request that was already pending before you update keeps its old reason. Deny any such request that you do not recognize.
To get it: it reaches the patches channel when the PR merges. After you update, you can set
FOG_AGENT_ENROLL_DEPLOY_WINDOWback to its default (24) if you set it to 0. -
@Tom-Elliott Thanks a lot Tom, that was fast.
I’m still on 1.6.0.14, and I couldn’t apply #2104 by hand like I did with #2100: it patches
MachineIdentity.php, which doesn’t exist in my version, and the change inEnrollment.phpcalls it. So I’ll wait for the update instead. Do you know which stable release is expected to include it?I’ll also run a full inventory on the hosts that share an adapter, as you suggest, so they have a serial stored. And I’ll report back once I’ve tried it with real laptops.
-
@lperoma The next stable will be auto promoted from dev-branch -> stable on December 11th at some point in the day.
11th of every month is when dev-branch fixes get pulled into stable.
As to the “Version” it will be, it’s however many commits since.
Right now .14 means there were 14 commits after master containing 1.6.0, 158 is the current stable branch meaning 158 total commits since it’s original release.
Some months will be less, some will be much more.
If you wanted you can install dev-branch but I understand the feeling of “well it’s not stable”.
-
Thanks for #2104. The case that is still open for us is PXE registration and the PXE menu.
A new laptop (not in FOG) boots through the shared USB adapter whose MAC is still on another host Y. Its firmware matches no host, so FOG falls back to the MAC and answers “This machine already registered as Y”.
Could the identification order be simply this? Always check UUID and serial first. Use the MAC only when the machine reports no usable firmware data (empty or placeholder values). If it does report firmware data and it matches no host, it is a new machine: offer registration, without moving any MAC and without giving the new host the adapter’s MAC.
It could be a new value of
FOG_HOST_IDENTIFY_SMBIOSsoenforcestays as it is.We understand that iPXE already sends the UUID and serial numbers with every PXE request (
sysuuid,sysserial,mbserial), also when booting through a USB adapter, since the firmware is read from the laptop and not from the NIC. Please correct us if that is wrong for registration.In our case the laptops that boot only through a USB adapter are recent and always report UUID and serial.
Happy to test a patch.
-
@lperoma 1.6.0.14 is an early 1.6.0 build. The current stable release is 1.6.0.158 (November 11), and it has
MachineIdentity.php. #2100 and #2104 are on the patches channel now. They reach stable with the next monthly release on the 11th.On the registration question: yes, iPXE sends
sysuuid,sysserialandmbserialwith every boot, through any NIC. They come from the laptop’s firmware, not from the adapter.PR #2111 adds what you asked for, as a new value for
FOG_HOST_IDENTIFY_SMBIOS:register. It works likeenforce, plus one rule. Suppose a laptop boots on an adapter whose MAC belongs to host Y, and the laptop’s serial is different from Y’s stored serial. Then:- The boot shows the “not registered” menu, so the laptop never runs Y’s task.
- Registration creates a new host for the laptop. Y keeps its MAC.
- The new host gets a generated MAC that starts with
02:. During a task, FOS identifies itself to the server by the host’s MAC, so every host needs one of its own. The network never sees the generated MAC. - After that, the laptop is found by its firmware through any adapter.
Two things to know:
- Y needs a stored serial, so run a full inventory on hosts that share an adapter. With no stored serial, the MAC still decides.
- In
registermode, a motherboard swap that changes the system serial looks like a new machine.
It is on the patches channel now. It reaches stable with the next monthly release on December 11.
-
Quick update: we’ve tested PR #2111 and it works perfectly for us.
Setup
- We also updated from 1.6.0.14 to 1.6.0.158 (stable).
- On top of it we applied #2100, #2104 and #2111 by hand (we skipped the schema 462 step, since it only changes a setting description).
FOG_HOST_IDENTIFY_SMBIOSis set toregister.
Test (two Proxmox VMs)
We moved the MAC of an already registered host onto a different machine. The machine with the different serial was not taken for the registered host: it booted as unregistered and registered as a new host, and the original host kept its MAC. Exactly the shared USB adapter case from this thread.
Real laptops with the shared adapter are the next step, and I’ll report back if anything behaves differently. We’ll switch to the stable release on December 11.
Thanks, Tom, this solves our problem.
-
Hi Tom, thanks again for the shared-adapter work.
We hit a case with the new agent that your replies in this thread don’t spell out, and while reproducing it we found a few more things, so here are the details.
Setup: FOG 1.6.0.158 stable with #2100, #2104 and #2111 applied by hand,
FOG_HOST_IDENTIFY_SMBIOS=register, agent 0.1.24, Windows 10 IoT LTSC images, HP laptops that boot from the shared USB-C Ethernet adapter.What worked yesterday and what doesn’t today: yesterday, with #2104 and #2111 already applied, we tested two VMs with a duplicated MAC and FOG handled it fine at PXE (
register, firmware first). Today, when we reinstall hosts whose firmware is already registered, it is the agent’s enrollment that fails when a MAC belongs to another host.The laptop that could never enroll
The laptop’s host has a full inventory (system serial, UUID, board serial), and no other host has any of those values. Boot worked: PXE identified it correctly through the shared adapter, since in
registera unique firmware match wins over the MAC. But the agent tried to enroll every 5 minutes and always gotHTTP 409 with status "error", 16 times in a row. Each attempt left this infaults-web.log:Database save failed: Class: AgentEnrollment ... SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`fog`.`agentEnrollment`, CONSTRAINT `fk_agentEnrollment_aeHostID` FOREIGN KEY (`aeHostID`) REFERENCES `hosts` (`hostID`) ON DELETE CASCADE)The agent sends two MACs: the shared adapter’s, still stored on another laptop’s host, and one of its own that no host had. Reading the code,
MachineIdentity::choose()returnsconflictwithhost_id0 when the firmware points to one host and a MAC points to a different one.Enrollment::enroll()then sets the reason toconflictand saves the row withhostID0, which doesn’t exist inhosts, so the foreign key fails and nothing is stored. Nothing shows up in All Pending Agents, so the admin never sees it. The agent enrollment code only checks whether the setting isoff, soregisterandenforcemake no difference there.We worked around it by deleting the adapter’s MAC from the other host. The very next attempt enrolled fine (
known-host-no-agent).Reproducing it with a VM
We gave a VM the MAC of another VM’s host (approved,
hmPending=0). Its firmware UUID names its own host. This time the conflict was stored: All Pending Agents shows it asidentity-conflict(the enroll got a 202) and the agent logs:directory rejoin: this machine's MAC addresses or SMBIOS identity match more than one FOG host; ... this state was made on another machine (a cloned or reimaged disk); generated a new key and enrolling as this machine pending approval on the server (identity-conflict); waitingThat row has no
_directoryIdentity. After the agent restarted, every enroll gotHTTP 409 with status "error"again, but with no database error infaults-web.log. We approved the pending row, restarted the VM, and the agent still gets 409 while the row staysissuedwith its certificate uncollected. As far as we can tell fromMachineIdentity::enrollmentGate(), it returnsupgrade_blocked: the stored row doesn’t have_directoryIdentityand the new request declares it, and that check runs before the row’s state is looked at. We haven’t confirmed that part on the agent side.So the same mismatch seems to end in two different 409 loops. If the agent declares the directory-identity feature from its first contact (the laptop, whose rows all have
{"required": true}), the conflict goes throughMachineIdentity::resolve()and ends with host 0 and the foreign key error. If it doesn’t declare it at the first contact (the VM), the conflict is stored, but the agent is locked out as soon as it starts declaring it, even after the admin approves.What we expected
A shared MAC shouldn’t be a conflict at all when the firmware identifies a different machine, which is what
enforce/registeralready do at boot. So:- Could agent enrollment follow the same rule as boot, so that a unique firmware match wins over the MAC?
- If a conflict is meant to apply there anyway, the row should be saved with the host the firmware matched (or none) instead of host 0, so it doesn’t fail on the foreign key.
- All Pending Agents only shows the Windows name, the reason, the IP and the UUID. It would help to show which host the conflict is with, so the admin can tell it’s another machine before approving.
- Approving a conflict should let the agent collect its certificate, even if the stored row was created before the agent started declaring the feature.
Happy to send full log lines or test a patch. We’re posting a second, unrelated problem (the “Clear directory operation review” button leaving a host unable to join) separately.
-
@lperoma Thanks for the detail. There are two server fixes and one agent fix.
Please update instead of copying files this time. This fix changes the database (schema step 465), and the code depends on it. Copied files without that step make enrollment saves fail in a new way. Your server is also now several changes past 1.6.0.158. Run the updater on the patches channel. It installs the files and runs the schema update together. Or wait for the stable release on December 11.
Server (FOG PRs #2116 and #2118):
- Agent enrollment now follows the boot rule for a shared adapter. The firmware names one host, and the adapter’s MAC is on another host. If that other host’s stored serial differs from the reported serial, the firmware’s host wins. Your laptop then enrolls as its own host, with the normal “known-host-no-agent” reason.
- The firmware host must match the serial too. A copied UUID alone never wins over a MAC.
- Any other disagreement stays an identity conflict. The row is now saved with no host, instead of host 0, so the foreign key no longer refuses it. The agent gets “pending”, not 409, and the row shows in All Pending Agents.
- For a conflict, All Pending Agents now shows which other host has the MAC. After you remove the cause (for example the stray MAC), the next agent poll binds the request to the right host. Then you approve it. A conflict is never approved automatically, not by a token and not by a recent deploy (#2118 closes this for older agents too).
Agent (fog-agent PR FOGProject/fog-agent#52): a key whose first request went in without the directory feature cannot add the feature later. That is deliberate on the server. The agent now makes a new key once when the server answers
enrollment_upgrade_requires_new_key, and enrolls again. This is in agent v0.1.28. Agents update to it on their own.