Categories

  • 13k Topics
    115k Posts
    Tom ElliottT

    @FlareImp Thanks, the log shows the cause. 1.6.0.14 has a replication bug: the check for a transfer that is still running never matched. So each cycle compared the files while lftp was still copying them. It saw the half-copied file on the node, logged “File size mismatch”, deleted it, and started the copy again. A large file over your rate-limited link never finished. That is why rsync worked and the replicator did not.

    1.6.0.158 fixes this. Please update the master and every storage node to 1.6.0.158, then turn the replicator back on.

    The “File does not exist win11” lines are the subfolder name, not a missing file. 1.5 logs the same lines. They make lftp run once per cycle, but it copies nothing when the files already match.

  • Get the latest news on what's happening.
    207 Topics
    848 Posts
    Tom ElliottT

    FOG 1.6.0 is released. It is on the stable branch and reports version 1.6.0.

    This is the first release of the 1.6 line, and it replaces 1.5. The upgrade from 1.5 to 1.6 is one-way: it changes the database schema, and there is no down-migration. Back up before you upgrade.

    Thank you to everyone who tested the 21 release candidates. Thank you also to everyone who reported a problem, on the forums or on GitHub, and to everyone who sent a pull request. Each report and each fix helped make 1.6.0 what it is.

    Highlights

    Role-based access control is part of core. Roles and their permissions are built in. You no longer install a plugin before permissions exist. Sites are part of core. Multi-site scoping is built in. The upgrade moves the site plugin’s data into the core sites table. Groups grant snapins, printers and modules. A host that joins a group later gets the grant, and a host that leaves loses it. Mass edit on the host list. It replaces the group page’s one-time push. Each field has its own No change, Set on all, or Clear on all choice. LDAP and OIDC sign-in are built-in identity providers. Directory groups map to FOG roles, so an external user no longer gets full admin by default. API Bearer tokens. Create one on a user’s API tab and send it as Authorization: Bearer <token>. Each token has its own label and on/off switch, and you can rotate it without touching your other integrations. FOG stores only a hash and shows the token once. The old fog-api-token plus fog-user-token pair still works, but a Bearer token is simpler to use and to maintain. Use it for new integrations. An audit trail. FOG records who changed what, and shows it as an activity view. A retention policy. A retention daemon removes log and audit records older than the number of days you set. The default keeps everything. The database enforces its relationships. Real foreign keys now cover every core relationship that can carry one. The upgrade cleans up orphan rows first and logs each cleanup. Plugins are installable artifacts. A plugin has its own install, update and removal, and it survives a FOG upgrade. Secure Boot enrollment is a task type. You schedule and track it like any other task.

    Plugins removed in 1.6

    These 1.5 plugins are removed. Do not copy them into a 1.6 server.

    accesscontrol: removed. Core role-based access control does its job. The upgrade removes its registration. site: removed. Core sites do its job. The upgrade copies its data into core and drops its tables only when the row counts match. persistentgroups: removed, with its database trigger. Core group grants do its job. Installing it again would re-create the trigger. fileintegrity and hoststatus: not carried into 1.6. No replacement ships.

    Every 1.5 plugin needs a 1.6 build to load. The 1.6 plugins are in https://github.com/FOGProject/fog-plugins .

    FOS 1.6.0 (kernel and init)

    Linux kernel 6.18.59 (LTS), Buildroot 2026.02.1. FOS verifies TLS when it talks to the server. x64 and x86 boot an initramfs, not an ext2 initrd. The file names stay init.xz and init_32.xz. Warning: a customized init can no longer be loop-mounted. Add your files as an overlay cpio appended to init.xz: https://docs.fogproject.org/en/latest/kb/how-tos/customize-the-fos-init/ . A post-init script needs no rebuild: https://docs.fogproject.org/en/latest/kb/how-tos/post-init-scripts/ master installs FOS release 1.6.0. stable and dev-branch installs download the Latest FOS release.

    What happens to your branch and update channel

    stable (--channel stable) now carries 1.6.0, then the monthly 1.6.0.N releases. dev-branch (--channel patches) now carries the 1.6 patches. rc-1.6.0 (--channel rc) and working-1.6 (--channel beta) are retired. Move to --channel stable. No branch was force-pushed. A 1.5 git checkout reaches 1.6 with an ordinary git pull.

    Before you upgrade

    Known issue: the upgrade from 1.5 fails at the schema update if the database holds multicast sessions (Truncated incorrect INTEGER value: ''). Do not upgrade a 1.5 server until 1.6.0.1 is released. Back up your database and /opt/fog/.fogsettings. Read the release notes: https://github.com/FOGProject/fogproject/blob/stable/docs/release/1.6.0-release-notes.md 1.6 removes Display Manager, Directory Cleaner, User Cleanup, Client Updater, Green FOG and the persistentgroups plugin. Their data is dropped. PHP 7.4 or later is required. Plugins built for 1.5 do not load.

    Install or update (as root)

    A 1.5 server installed from git: run bin/updatefog.sh --channel stable from the checkout. A 1.6 beta or RC server: run bin/updatefog.sh --channel stable. A new server, or a 1.5 server installed from a tarball:
    curl -fsSL https://raw.githubusercontent.com/FOGProject/fogproject/stable/bin/bootstrap.sh | bash -s -- --channel stable

    If an upgrade fails, bin/revertfog.sh in the 1.6 checkout restores the pre-upgrade backup. Run it with --list, then --dry-run, first.

    1.5 support

    1.5 is end of life. The last 1.5 release is 1.5.10.2534. It gets no further fixes, security fixes included. Every 1.5 tag stays in the repository. Details: https://github.com/FOGProject/fogproject/blob/stable/docs/release/1.5-support-statement.md

    Report problems

    Open an issue at https://github.com/FOGProject/fogproject/issues. Include your FOG version, your OS, and the installer log from bin/error_logs/. Report a security problem through “Report a vulnerability” on the same repository, not in a public issue.

  • View tutorials or talk about FOG in general.
    2k Topics
    19k Posts
    Tom ElliottT

    I just wanted to add some updates as 1.6 being released and with advent of Win 11 now.

    Wanted to have more capable set of scripts here. I know the thread is very old, but it’s still worth the update in my book.

    /images/postdownloadscripts/fog.postdownload

    #!/bin/bash . /usr/share/fog/lib/funcs.sh [[ -z $postdownpath ]] && postdownpath="/images/postdownloadscripts/" # FOS sets osClass from the image's OS (determineOS, FOS 20261110 or later). # On an older FOS kernel osClass is empty, so derive it from the osid. if [[ -z $osClass ]]; then case $osid in 5|6|7) osClass="win7" ;; 9|10|11) osClass="winmodern" ;; esac fi case $osClass in win7|winmodern) clear [[ ! -d /ntfs ]] && mkdir -p /ntfs getHardDisk if [[ -z $hd ]]; then handleError "Could not find hdd to use" fi getPartitions $hd for part in $parts; do umount /ntfs >/dev/null 2>&1 fsTypeSetting "$part" case $fstype in ntfs) dots "Testing partition $part" ntfs-3g -o force,rw $part /ntfs ntfsstatus="$?" if [[ ! $ntfsstatus -eq 0 ]]; then echo "Skipped" continue fi if [[ ! -d /ntfs/windows && ! -d /ntfs/Windows && ! -d /ntfs/WINDOWS ]]; then echo "Not found" umount /ntfs >/dev/null 2>&1 continue fi echo "Success" break ;; *) echo " * Partition $part not NTFS filesystem" ;; esac done if [[ ! $ntfsstatus -eq 0 ]]; then echo "Failed" debugPause handleError "Failed to mount $part ($0)\n Args: $*" fi echo "Done" debugPause . ${postdownpath}fog.copydrivers # . ${postdownpath}fog.updateunattend umount /ntfs ;; *) echo "Non-Windows Deployment" debugPause return ;; esac

    /images/postdownloadscripts/fog.copydrivers

    #!/bin/bash ceol=`tput el`; manu=`dmidecode -s system-manufacturer`; dots "Identifying hardware" case $manu in [Ll][Ee][Nn][Oo][Vv][Oo]) machine=$(dmidecode -s system-version) ;; *I[Nn][Tt][Ee][Ll]*) # For the Intel NUC and intel mobo pick up the system type from the # baseboard product name machine=$(dmidecode -s baseboard-product-name) ;; *) # Dell and everything else machine=$(dmidecode -s system-product-name) ;; esac # if the machine isn't identified then no need to continue with this script, just return to caller if [[ -z $machine ]]; then echo "Unable to identify the hardware for manufacturer ${manu}"; debugPause; return; fi echo "${machine} Identified"; # Remove spaces from the machine name, works better with path definitions machine="${machine//[[:space:]]/}" dots "Verifying we've found the OS disk" if [[ ! -d /ntfs/windows && ! -d /ntfs/Windows && ! -d /ntfs/WINDOWS ]]; then echo "! OS root Not found !"; debugPause return; fi echo "Found"; dots "Verifying target Arch" system64="/ntfs/Windows/SysWOW64/" [[ ! -d $system64 ]] && arch="x86" || arch="x64" echo "${arch} found"; debugPause # Build the OS folder name from osname, which FOS sets from the image's OS. # "Windows 10" becomes win10, "Windows 8.1" becomes win8.1, "Windows 11" # becomes win11. So the folder names from the old osid table still match. osn="${osname,,}" osn="${osn//[[:space:]()]/}" osn="${osn/windows/win}" dots "Preparing Drivers" clientdriverpath="/ntfs/Drivers" remotedriverpath="/images/drivers/$machine/$osn/$arch" debugPause if [[ ! -d "${remotedriverpath}" ]]; then echo "failed"; echo " ! Driver package not found for ${machine}/$osn/$arch ! "; debugPause; return; fi echo "Ready"; debugPause [[ ! -d $clientdriverpath ]] && mkdir -p "$clientdriverpath" >/dev/null 2>&1 echo -n "In Progress" rsync -aqz "$remotedriverpath" "$clientdriverpath" >/dev/null 2>&1 [[ ! $? -eq 0 ]] && handleError "Failed to download driver information for [$machine/$osn/$arch]" debugPause # Windows 7, 8 and 8.1 only. Windows 10 and later read the OEM driver path # from the unattend.xml, so they skip the registry edit. [[ $osClass != win7 ]] && return regfile="/ntfs/Windows/System32/config/SOFTWARE" key="\Microsoft\Windows\CurrentVersion\DevicePath" devpath="%SystemRoot%\DRV;%SystemRoot%\inf;"; reged -e "$regfile" &>/dev/null <<EOFREG ed $key $devpath q y EOFREG

    /images/postdownloadscripts/fog.updateunattend

    #!/bin/bash # Sets the computer name and, for a host with "Join Domain" on, the domain # join in the deployed image's unattend.xml. fog.postdownload sources this # file with the Windows partition mounted on /ntfs. # # FOG sends these from the host record (service/hostinfo.php): # hostname the host name # hostusead 1 when "Join Domain after deploy" is on # hostaddomain, hostadou the domain and the OU # hostaduser, hostadpass the join account and its password # hostadpass is empty unless FOG Configuration > FOG Settings > # FOG_HOSTKEY_ALLOWED_SOURCES lists the imaging network. The FOG Agent, # and the older FOG Client, join the domain without writing the password # to disk. Use this only where neither is installed. # Escape a value for XML text, then for the right side of a sed s|||. # The quoted replacements stop bash 5.2 from reading & as "the match". xmlEscape() { local s="$1" s="${s//&/'&amp;'}" s="${s//</'&lt;'}" s="${s//>/'&gt;'}" s="${s//\"/'&quot;'}" s="${s//\'/'&apos;'}" printf '%s' "$s" } sedEscape() { local s="$1" s="${s//\\/'\\'}" s="${s//|/'\|'}" s="${s//&/'\&'}" printf '%s' "$s" } xmlValue() { sedEscape "$(xmlEscape "$1")" } # The places Windows Setup reads an unattend.xml from after sysprep. The # [x] in each name makes it a glob, so nocaseglob matches any case of it. shopt -s nocaseglob nullglob unattends=( /ntfs/[w]indows/[p]anther/[u]nattend.xml /ntfs/[w]indows/[p]anther/[u]nattend/[u]nattend.xml /ntfs/[w]indows/[s]ystem32/[s]ysprep/[u]nattend.xml /ntfs/[u]nattend.xml ) shopt -u nocaseglob nullglob if [[ ${#unattends[@]} -eq 0 ]]; then echo " * No unattend.xml found, skipping" debugPause return fi # A PXE boot does not carry the AD values on the kernel command line. Ask the # server for them, the same way FOS does after a USB boot. Keep only the # host* lines of the reply, and delete the reply because it holds the password. if [[ -z $hostusead ]]; then curlcmd=(curl -Lks) command -v fog.curl >/dev/null 2>&1 && curlcmd=(fog.curl -Ls) sysuuid=$(dmidecode -s system-uuid) sysuuid=${sysuuid,,} [[ -z $mac ]] && mac=$(getMACAddresses) token=$("${curlcmd[@]}" --data "mac=$(echo $mac | base64)" "${web}status/hostgetkey.php") "${curlcmd[@]}" -o /tmp/hinfo.txt --data "sysuuid=${sysuuid}&mac=${mac}&hosttoken=${token}" "${web}service/hostinfo.php" -A '' if [[ -f /tmp/hinfo.txt ]]; then . <(grep -E '^\[\[ -z \$host[a-z]+ \]\] && export host[a-z]+=' /tmp/hinfo.txt) rm -f /tmp/hinfo.txt fi fi joinDomain="" if [[ $hostusead == 1 && -n $hostaddomain ]]; then if [[ -z $hostadpass ]]; then echo " * No domain join password received, so the domain join is skipped." echo " Set FOG_HOSTKEY_ALLOWED_SOURCES to the imaging network to send it." debugPause else joinDomain=1 # Qualify a bare user name with the domain: DOMAIN\user. joinUser="$hostaduser" [[ $joinUser != *\\* && $joinUser != *@* ]] && joinUser="${hostaddomain}\\${joinUser}" fi fi name=$(xmlValue "$hostname") for unattend in "${unattends[@]}"; do dots "Writing Computer Name to $unattend" sed -i "s|<ComputerName>[^<]*</ComputerName>|<ComputerName>${name}</ComputerName>|g" "$unattend" if [[ ! $? -eq 0 ]]; then echo "Failed" debugPause handleError "Failed to update $unattend" fi echo "Done" debugPause [[ -z $joinDomain ]] && continue dots "Setting the domain join in $unattend" # Username and Password change only inside <Credentials>. AutoLogon and # the local accounts use the same element names. sed -i \ -e "/<JoinWorkgroup>/d" \ -e "/<Credentials>/,/<\/Credentials>/s|<Password>[^<]*</Password>|<Password>$(xmlValue "$hostadpass")</Password>|" \ -e "/<Credentials>/,/<\/Credentials>/s|<Username>[^<]*</Username>|<Username>$(xmlValue "$joinUser")</Username>|" \ -e "s|<MachineObjectOU>[^<]*</MachineObjectOU>|<MachineObjectOU>$(xmlValue "$hostadou")</MachineObjectOU>|g" \ -e "s|<JoinDomain>[^<]*</JoinDomain>|<JoinDomain>$(xmlValue "$hostaddomain")</JoinDomain>|g" \ "$unattend" if [[ ! $? -eq 0 ]]; then echo "Failed" debugPause handleError "Failed to set the domain join in $unattend" fi echo "Done" debugPause done
  • Report bugs, request features, or get the latest progress.
    2k Topics
    21k Posts
    Tom ElliottT

    @lperoma Thank you for the detailed write-up. The detection notes made this quick to scope.

    FOG 1.6 covers this as follows:

    FOG Client cannot authorize. This was the real gap, and a pull request now adds it: https://github.com/FOGProject/fogproject/pull/2089. The server records each host whose client fails the handshake: since when, how many times, the source address and the reason. A new list report, Reports > Client Authentication Failures, shows them with the fix next to each one. A host leaves the list when its client authenticates again. Two reasons are recorded:

    The security token does not match. Reset Encryption Data fixes this. The server cannot decrypt the client’s key. This means the certificate the server publishes for clients does not pair with its private key, so every host fails at once. A reset does not help. Running the installer with --recreate-keys issues a new pair.

    One case stays invisible to the server. A client that trusts a different FOG CA refuses the server’s certificate and never sends a login, so there is nothing to record. Its fog.log shows “Certificate is not from FOG CA”.

    Rejected iPXE menu logins. 1.6 already records each rejected login in the audit trail, with the username and the address. Look under Reports > Audit Report.

    Hosts with no contact. 1.6 stores the last check-in and the last ping for each host. Fleet Report shows the last check-in, so you can sort on it.

    All of this is 1.6 only. 1.5.x does not get these changes, so your script remains the answer there until you move to 1.6.

41

Online

12.8k

Users

17.7k

Topics

157.4k

Posts