• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    A reimaged host stays stuck on an old directory join plan ("marker absent; automatic retry refused")

    Scheduled Pinned Locked Moved Unsolved FOG Problems
    2 Posts 2 Posters 11 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lperoma
      last edited by

      Hi Tom and everyone,

      First, thanks for the quick work on the shared adapter and enrollment issues. I tested #2116/#2118 today and enrollment behaves now. While doing that I hit something else with the directory join that I think is worth a thread of its own.

      Setup: FOG 1.6.0.158 stable moved today to the patches channel (schema 465, so it includes #2111, #2112, #2113, #2116 and #2118). Two Proxmox VMs (test-uefi and test-bios) deployed from the same image, whose source machine already has agent 0.1.28. Both hosts are set to join AD and both had been deployed once before today.

      After the second Deploy, test-bios joined the domain without any trouble. test-uefi never did. It kept the Windows name “instalacion”, stayed in the workgroup, and installed its software normally (Chrome, VLC, 7-Zip and Avast, before the activation snapin, which is the order we wanted). The agent log shows this every minute, and nothing else about the directory:

      2026-11-20T19:24:29+01:00 this state was made on another machine (a cloned or reimaged disk); generated a new key and enrolling as this machine
      2026-11-20T19:24:29+01:00 enrolled as host 666, certificate valid until 2027-11-20 18:24:29
      2026-11-20T19:26:01+01:00 facts: sending inventory=true software=6 directory=true printers=4 network=1
      2026-11-20T19:26:04+01:00 directory rejoin: directory rejoin marker absent; automatic retry refused
      2026-11-20T19:28:55+01:00 directory rejoin: directory rejoin marker absent; automatic retry refused
      

      So the very first directory poll after the reimage was already refused. On the server, hostDirectory.hdRejoinState for that host had this (trimmed):

      "phase":"awaiting_reboot", "attempt_id":"add87a0f...", "trigger_task_id":16923,
      "last_deploy_task_id":16923, "retry_authorized":false, "placement_hold":true
      

      The Deploy that had just reimaged the machine was task 16927, newer than 16923, but it did not replace that plan. The computer account was fine in AD (it existed and belonged to the account FOG uses to join), so it was not an account problem.

      Reading the code, this looks like the intended protection: PROTECTED_PHASES is started, awaiting_reboot, unknown and failed, and only a plan in ready is dropped when a newer Deploy arrives (dropSupersededPlan, reason directory_task_superseded). That makes sense for the old disk, where the join may really have happened. But after a Deploy the old disk is gone. The plan waits for a local marker and a reboot that can no longer exist, the new install has no marker, so the agent refuses, and the server keeps waiting for the agent. Neither side ever gives up.

      What happened to this host before is simple: it had a join plan pending that never actually ran, I reinstalled it, and it got stuck on that old plan. I can’t show you the log of the earlier install, because the reimage overwrote it, so the odd part is that the stored state says awaiting_reboot for something that, as far as I know, was never executed. test-bios did not get stuck, and its previous plan had ended in “verified”.

      The way out is the “Clear directory operation review” checkbox, and here is where I got a second surprise. clearReview() writes a new state with last_deploy_task_id set to the latest Deploy and drops workgroup and observation. As far as I can tell, with the Deploy counted as already used and no workgroup flag, nothing gets planned afterwards. On a real PC earlier today, ticking the box left the host with no plan at all. I added "workgroup":true by hand and FOG did plan a join, but that agent never ran it and I joined the machine manually.

      For test-uefi I wrote the cleared state by hand instead: reason: operator_cleared, workgroup: true, and last_deploy_task_id set to the previous Deploy (16923), so that 16927 was still pending. On the next poll FOG created a new plan (ready, trigger 16927), the agent ran it (awaiting_reboot, no error), and after a reboot it went to verified with the host joined. So the join itself works, and only the stuck state was the problem.

      What I would suggest, if you agree:

      1. A new Deploy for a host should clear everything that was pending for the previous installation: queued tasks, snapin tasks and directory join plans, not only plans in ready. This includes plans in started, awaiting_reboot, unknown and failed, because they refer to a disk that has been replaced. Whatever the Deploy itself creates (for example the snapins attached to it) would of course stay. I only checked the join plans; I have not tested what happens to old queued tasks and snapins when a Deploy is created, so please take that part as a suggestion.
      2. The review checkbox should keep workgroup and observation, and should not mark the latest Deploy as used, so the next poll can plan the join again.
      3. When the agent says “marker absent; automatic retry refused”, the host’s Active Directory panel could say that this needs the checkbox. Right now the only hint is in the agent log on the machine.

      Open question on my side: I don’t know why that first plan was in awaiting_reboot if it never ran. If there is something I should capture the next time (agent log before reimaging, a particular field of the state), tell me and I will.

      Thanks!

      Tom ElliottT 1 Reply Last reply Reply Quote 0
      • Tom ElliottT
        Tom Elliott @lperoma
        last edited by

        @lperoma Thanks for the detailed report. Your reading of the code is right. I opened https://github.com/FOGProject/fogproject/pull/2122 to fix both problems.

        On your open question: the first attempt did run. The server stores awaiting_reboot only when the agent reports that Windows accepted the join. Your state also has last_deploy_task_id equal to trigger_task_id (16923), and that is set only when the agent claims the plan. So the join ran after Deploy 16923, and Windows waited for a restart. The disk was reimaged by 16927 before that restart, so the agent never reported “verified”. That also explains why the computer account in AD looked healthy.

        What changes in #2122:

        • A newer Deploy now supersedes an old attempt in started, awaiting_reboot, unknown or failed. Three things must be true: the agent has no marker for the old attempt, a newer Deploy has completed, and the machine booted after that Deploy ended. FOG then drops the old attempt, writes an audit entry, and plans a new join from the new Deploy. If the agent still has the marker, the old attempt stays.
        • The “Clear directory operation review” checkbox no longer marks the newest Deploy as used. It uses up only the cleared attempt’s own Deploy. It also keeps the workgroup flag and the observation. If a newer Deploy exists, the next check-in can plan the join again.
        • With the first change, a reimaged host no longer needs the checkbox, so I did not add a hint to the Active Directory panel.

        On queued tasks and snapins from the previous install: this PR changes only the directory join plans.

        The fix reaches the patches channel after the PR merges. Your hand-written state on test-uefi does no harm.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post

        33

        Online

        12.8k

        Users

        17.7k

        Topics

        157.5k

        Posts
        Copyright © 2012-2026 FOG Project