• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Fog 1.6.0.14 Master Storage Node not handing off to Slave Storage Nodes

    Scheduled Pinned Locked Moved Unsolved FOG Problems
    28 Posts 2 Posters 292 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      FlareImp @Tom Elliott
      last edited by

      @Tom-Elliott

      {"draw":0,"recordsTotal":1298,"recordsFiltered":1,"truncated":false,"data":[{"id":384,"DT_RowId":"row_384","hostID":383,"hostLink":"(383) - 5260-Test<\/a>","mac":"e4:b9:7a:f0:a2:df","description":"","pending":0,"primary":1,"clientIgnore":0,"imageIgnore":0,"mac_vendor":""}],"_searchtypes":{"id":"num","DT_RowId":"num","hostID":"num","hostLink":"num","mac":"string","description":"string","pending":"num","primary":"num","clientIgnore":"num","imageIgnore":"num","mac_vendor":"string"},"_unadjustednote":"recorded before or near the time this server moved to UTC; it can be in UTC time or in UTC","_lang":"macaddressassociation","recordsReturned":1,"firstUrl":null,"prevUrl":null,"nextUrl":null,"lastUrl":null}
      
      1. I got “This page isn’t working HTTP ERROR 500”

      PHP 8.5.4 (cli) (built: Oct 19 2026 14:35:18) (NTS)
      Copyright (c) The PHP Group
      Built by Ubuntu
      Zend Engine v4.5.4, Copyright (c) Zend Technologies
          with Zend OPcache v8.5.4, Copyright (c), by Zend Technologies
      mysql from 11.8.6-MariaDB, client 15.2 for debian-linux-gnu (x86_64) using  EditLine wrapper
      2026-11-05 15:24:47 upgrade php-curl:all 2:8.3+93ubuntu2 2:8.5+99ubuntu1
      2026-11-05 15:24:48 install php8.5-gd:amd64 <none> 8.5.4-0ubuntu1.5
      2026-11-05 15:24:48 upgrade php-gd:all 2:8.3+93ubuntu2 2:8.5+99ubuntu1
      2026-11-05 15:24:48 install php8.5-ldap:amd64 <none> 8.5.4-0ubuntu1.5
      2026-11-05 15:24:48 upgrade php-ldap:all 2:8.3+93ubuntu2 2:8.5+99ubuntu1
      2026-11-05 15:24:50 install php8.5-mbstring:amd64 <none> 8.5.4-0ubuntu1.5
      2026-11-05 15:24:50 upgrade php-mbstring:all 2:8.3+93ubuntu2 2:8.5+99ubuntu1
      2026-11-05 15:24:51 install php8.5-mysql:amd64 <none> 8.5.4-0ubuntu1.5
      2026-11-05 15:24:51 upgrade php-mysql:all 2:8.3+93ubuntu2 2:8.5+99ubuntu1
      2026-11-05 15:24:58 upgrade mariadb-common:all 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:24:59 upgrade mariadb-server:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:25:01 upgrade mariadb-server-core:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:25:02 upgrade mariadb-client-core:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:25:02 upgrade mariadb-client:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:25:03 upgrade libmariadb3:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:27:42 upgrade mariadb-plugin-provider-bzip2:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:27:42 upgrade mariadb-plugin-provider-lz4:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:27:42 upgrade mariadb-plugin-provider-lzma:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:27:42 upgrade mariadb-plugin-provider-lzo:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      2026-11-05 15:27:43 upgrade mariadb-plugin-provider-snappy:amd64 1:10.11.14-0ubuntu0.24.04.1 1:11.8.6-5ubuntu0.1
      
      Tom ElliottT 1 Reply Last reply Reply Quote 0
      • Tom ElliottT
        Tom Elliott @FlareImp
        last edited by

        @FlareImp Thank you. A correction first: I said my lab shows the primary MAC. I had checked the database and the data feed, not the page. On the page, my lab shows the same problem you see.

        The MAC tab: your primary MACs are saved correctly. The page fails to show them. When the grid measures its columns, it briefly adds a copy of the table, and that copy unchecks the real radio button. The fix is https://github.com/FOGProject/fogproject/pull/2105. It changes only the page. Your saved data needs no change.

        The PXE boot: this is a separate problem. Your boot.php page answered with HTTP 500, which means PHP stopped with an error. Please open that boot.php?mac= address again. Then, right away, post the last 20 lines of /var/log/apache2/error.log. The error should be there.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        F 1 Reply Last reply Reply Quote 0
        • F
          FlareImp @Tom Elliott
          last edited by FlareImp

          @Tom-Elliott

          de. Group Adamsville Storage Group (id 2); node id 4 chosen anyway. The probe is a reachability test, not a test of the image share, so tasking continues -- if the task fails, check that this node is                                       reachable.', referer: https://tv-fog-main.tvschools.local/fog/management/index.php?node=storagenode&sub=list
          [Wed Nov 18 08:25:58.622405 2026] [mpm_prefork:notice] [pid 1289:tid 1289] AH00170: caught SIGWINCH, shutting down gracefully
          AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
          [Wed Nov 18 08:26:18.208295 2026] [mpm_prefork:notice] [pid 6900:tid 6900] AH00163: Apache/2.4.66 (Ubuntu) OpenSSL/3.5.5 configured -- resuming normal operations
          [Wed Nov 18 08:26:18.208351 2026] [core:notice] [pid 6900:tid 6900] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'
          [Wed Nov 18 08:28:33.963128 2026] [proxy_fcgi:error] [pid 12762:tid 12762] [client 10.220.99.11:56139] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /va                                      r/www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html                                      /fog/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location                                      \\Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boo                                      t\\IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:36.157208 2026] [proxy_fcgi:error] [pid 6916:tid 6916] [client 10.220.99.11:49457] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:36.919035 2026] [proxy_fcgi:error] [pid 7092:tid 7092] [client 10.220.99.11:61872] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:37.413317 2026] [proxy_fcgi:error] [pid 13106:tid 13106] [client 10.220.99.11:58000] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /va                                      r/www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html                                      /fog/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location                                      \\Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boo                                      t\\IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:37.597657 2026] [proxy_fcgi:error] [pid 7092:tid 7092] [client 10.220.99.11:56075] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:37.814380 2026] [proxy_fcgi:error] [pid 6916:tid 6916] [client 10.220.99.11:50029] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:38.053148 2026] [proxy_fcgi:error] [pid 13122:tid 13122] [client 10.220.99.11:62517] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /va                                      r/www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html                                      /fog/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location                                      \\Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boo                                      t\\IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:38.282779 2026] [proxy_fcgi:error] [pid 6914:tid 6914] [client 10.220.99.11:63151] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:38.506414 2026] [proxy_fcgi:error] [pid 13106:tid 13106] [client 10.220.99.11:56121] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /va                                      r/www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html                                      /fog/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location                                      \\Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boo                                      t\\IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:38.883973 2026] [proxy_fcgi:error] [pid 13105:tid 13105] [client 10.220.99.11:53142] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /va                                      r/www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html                                      /fog/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location                                      \\Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boo                                      t\\IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          [Wed Nov 18 08:31:39.088506 2026] [proxy_fcgi:error] [pid 6916:tid 6916] [client 10.220.99.11:61844] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught Exception: No nodes available in /var/                                      www/html/fog/src/Items/StorageGroup.php:455\nStack trace:\n#0 /var/www/html/fog/lib/plugins/location/src/Items/Location.php(183): FOG\\Items\\StorageGroup->getOptimalStorageNode()\n#1 /var/www/html/f                                      og/lib/plugins/location/src/Hooks/LocationChangeItems.php(204): FOG\\Plugins\\Location\\Items\\Location->getStorageNode()\n#2 /var/www/html/fog/src/Base/HookManager.php(219): FOG\\Plugins\\Location\\                                      Hooks\\LocationChangeItems->bootItemSettings()\n#3 /var/www/html/fog/src/Boot/IpxeBootMenu.php(921): FOG\\Base\\HookManager->processEvent()\n#4 /var/www/html/fog/service/ipxe/boot.php(86): FOG\\Boot\                                      \IpxeBootMenu->__construct()\n#5 {main}\n  thrown in /var/www/html/fog/src/Items/StorageGroup.php on line 455'
          
          Tom ElliottT 1 Reply Last reply Reply Quote 0
          • Tom ElliottT
            Tom Elliott @FlareImp
            last edited by

            @FlareImp Thank you, that log shows the cause.

            What happens: your hosts belong to a location with “Boot files from” turned on. That location has no Storage Node set, so FOG picks a node from the location’s Storage Group. That group has no storage nodes in it, or it no longer exists. FOG finds no node, and the boot page stops with “No nodes available”. So the host cannot load its boot menu and shows as not registered. I made the same setup on my lab and got the same error.

            My earlier suggestion to clear the location’s Storage Node is what sent FOG down this path. I’m sorry for that.

            Fix it now: open Location Management, then the location those hosts use. Set its Storage Group to the group that holds the Adamsville nodes, then save. Or set its Storage Node back to the node it had before. Then PXE boot one host.

            What I’m changing: a location set up like this should not stop every PXE boot. I’m fixing the location plugin, so it skips that location’s node and writes one line to the error log instead.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

            F 1 Reply Last reply Reply Quote 0
            • F
              FlareImp @Tom Elliott
              last edited by FlareImp

              @Tom-Elliott YAY! that seems to have fixed the issue at Adamsville location! I will test my other locations and let you know if everything is 10/10.

              Tom ElliottT 1 Reply Last reply Reply Quote 0
              • Tom ElliottT
                Tom Elliott @FlareImp
                last edited by

                @FlareImp Great, thank you for testing it.

                Both fixes are now in dev-branch. Run updatefog.sh --branch dev-branch again on the main server to get them:

                • The MAC tab shows the primary MAC as selected.
                • A location whose Storage Group has no nodes no longer stops the PXE boot. FOG uses its own node and writes one line to the error log that names the location.

                Let me know how the other sites do, and whether the deploys spread across the nodes.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                F 1 Reply Last reply Reply Quote 0
                • F
                  FlareImp @Tom Elliott
                  last edited by

                  @Tom-Elliott Per the original post, I was able to image 12 of the computer at the Adamsville location. 10 of them kicked off using the Master node but the other two sat in the que rather than moving to one of the other two free nodes.

                  Tom ElliottT 2 Replies Last reply Reply Quote 0
                  • Tom ElliottT
                    Tom Elliott @FlareImp
                    last edited by

                    @FlareImp Thanks. That should not happen on 1.6.0.181: when a host PXE boots and the master already has 10 tasks, the host should get a free node in the same group. I ran the same setup on my lab, 12 hosts with Max Clients 10, booted one at a time and all at once. Every time, the extra hosts moved to the free node.

                    When FOG looks for a free node, it skips a node without a message if any of these is true:

                    • The node is not enabled, or its Max Clients is 0.
                    • The node is not in the location’s Storage Group.
                    • The main server cannot reach the node. FOG tests the node’s SSH port (22 unless you changed it) first, then port 443 (or 80 if you use http).

                    Please check these for the Adamsville storage group:

                    • In Storage Node Management, for each of the three nodes: Storage Group, Is Enabled, Is Master Node, Max Clients.
                    • In Location Management, for that location: Storage Group and Storage Node.
                    • On the main server, run this for each of the two other nodes (replace the IP):
                    nc -zv -w 1 <node-ip> 22; nc -zv -w 1 <node-ip> 443; nc -zv -w 1 <node-ip> 80
                    

                    If all three fail for a node, FOG treats that node as offline, and that is why the hosts stayed in the queue.

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    F 1 Reply Last reply Reply Quote 0
                    • F
                      FlareImp @Tom Elliott
                      last edited by

                      @Tom-Elliott
                      In Storage Node Management, everything is set correctly.
                      in Location Management, everything is set correctly.
                      On the main server, all 3 nodes at Adamsville location passed their checks.

                      Tom ElliottT 1 Reply Last reply Reply Quote 0
                      • F
                        FlareImp
                        last edited by

                        After updating dev-branch 1.6.0.188 with the fixes you listed above. I noticed that my replication logs on all nodes (Master and Slave) are reading everything as:

                        [11-18-26 4:14:16 pm]  * We are group ID: 1. We are group name: default
                        [11-18-26 4:14:16 pm]  * We are node ID: 1. We are node name: DefaultMember
                        [11-18-26 4:14:16 pm]  * Attempting to perform Group -> Group image replication.
                        [11-18-26 4:14:16 pm]  | There are no images available!.
                        
                        

                        Nothing has changed in my settings so everything is still a member of its own induvial locations.

                        1 Reply Last reply Reply Quote 0
                        • Tom ElliottT
                          Tom Elliott @FlareImp
                          last edited by

                          @FlareImp Thanks for checking all of that. With those settings, the two hosts should have moved, and I cannot make my lab hold them back. So I need FOG to tell us why.

                          Until now, when a host could not move, FOG wrote nothing about the reason. I added a log line for that case: https://github.com/FOGProject/fogproject/pull/2108. It names the node the host waits on, its task count and Max Clients, and each other node in the group with the reason it was passed over.

                          When it is in dev-branch:

                          • Run updatefog.sh --branch dev-branch on the main server.
                          • Deploy the same 12 hosts again.
                          • If any host waits, post the output of this, from the main server:
                          grep "FOG storage node overflow" /var/log/apache2/error.log | tail -5
                          
                          • Also post the Storage Node column from the Active Tasks page for the hosts that wait.

                          If a host waits and the grep finds nothing, that is useful too. It means FOG did not see the node as over its Max Clients at PXE boot.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          1 Reply Last reply Reply Quote 0
                          • Tom ElliottT
                            Tom Elliott @FlareImp
                            last edited by

                            @FlareImp Thanks. That line means the replicator on that server thinks it is node 1. On 11-13, after the fix in your other thread, your nodes showed their own IDs, for example node 13 TVNE-Fog. Nothing between 1.6.0.181 and 1.6.0.188 changed how a server finds its own node.

                            One detail stands out. On 11-13, node 1 was named TV-Fog-Main. Now the log calls node 1 “DefaultMember”, in group “default”. Those are the names FOG gives the first node and group when it creates them.

                            Please post:

                            • From Storage Node Management: the name, IP address, storage group and Is Master setting of node 1. If no node is named DefaultMember, say so.
                            • Which server you read each log on, and how: the Log Viewer page, or the file /opt/fog/log/fogreplicator.log on that server.
                            • On one storage node that shows node ID 1, the output of:
                            hostname; hostname -I; grep -v '^#' /etc/hosts
                            head -20 /opt/fog/log/fogreplicator.log
                            

                            Your log says group “default” has no images, so these runs copy nothing. As a precaution until we know why, stop the replicators on the storage nodes again, but not on the main server. On 11-13, a node that acted as the main server deleted files on the other nodes:

                            sudo systemctl stop FOGImageReplicator FOGSnapinReplicator
                            

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                            F 1 Reply Last reply Reply Quote 0
                            • F
                              FlareImp @Tom Elliott
                              last edited by FlareImp

                              @Tom-Elliott There is not a default group. The main fog server is TV-Fog-Main and its in a storage group A(Default) by itself. I’m reading the logs from /opt/fog/log/fogreplicator.log on each storage node.

                              head -20 /opt/fog/log/fogreplicator.log
                              TVMS-Fog
                              10.229.9.50
                              127.0.0.1 localhost
                              127.0.1.1 tvms-fog
                              
                              ::1     ip6-localhost ip6-loopback
                              fe00::0 ip6-localnet
                              ff00::0 ip6-mcastprefix
                              ff02::1 ip6-allnodes
                              ff02::2 ip6-allrouters
                              [11-17-26 8:08:06 pm] ===== FOG 1.6.0.181 -- ImageReplicator starting =====
                              [11-17-26 8:08:06 pm] Interface Ready with IP Address: 10.229.9.50
                              [11-17-26 8:08:06 pm] Interface Ready with IP Address: 127.0.0.1
                              [11-17-26 8:08:06 pm] Interface Ready with IP Address: 127.0.1.1
                              [11-17-26 8:08:06 pm] Interface Ready with IP Address: TVMS-Fog
                              [11-17-26 8:08:06 pm]  * Starting ImageReplicator Service
                              [11-17-26 8:08:06 pm]  * Checking for new items every 600 seconds
                              [11-17-26 8:08:06 pm]  * Starting service loop
                              [11-17-26 8:08:06 pm]  * Starting Image Replication.
                              [11-17-26 8:08:06 pm]  * We are group ID: 1. We are group name: default
                              [11-17-26 8:08:06 pm]  * We are node ID: 1. We are node name: DefaultMember
                              [11-17-26 8:08:06 pm]  * Attempting to perform Group -> Group image replication.
                              [11-17-26 8:08:06 pm]  | There are no images available!.
                              [11-17-26 8:31:31 pm] ===== FOG 1.6.0.181 -- ImageReplicator starting =====
                              [11-17-26 8:31:32 pm] Interface Ready with IP Address: 10.229.9.50
                              [11-17-26 8:31:32 pm] Interface Ready with IP Address: 127.0.0.1
                              [11-17-26 8:31:32 pm] Interface Ready with IP Address: 127.0.1.1
                              [11-17-26 8:31:32 pm] Interface Ready with IP Address: TVMS-Fog
                              [11-17-26 8:31:32 pm]  * Starting ImageReplicator Service
                              [11-17-26 8:31:32 pm]  * Checking for new items every 600 seconds
                              
                              Tom ElliottT 1 Reply Last reply Reply Quote 0
                              • Tom ElliottT
                                Tom Elliott @FlareImp
                                last edited by

                                @FlareImp Thanks, that shows it. Your real database has no group “default” and no node “DefaultMember”. So TVMS-Fog is not reading your main server’s database. It reads a database on TVMS-Fog itself. The FOG installer fills a new database with that group and node, and gives the node the server’s own IP. So TVMS-Fog matches node 1 in its own database, and that database has no images.

                                A storage node must use the main server’s database. Please run this on TVMS-Fog and post the output. It shows no passwords:

                                grep -E '^(installtype|snmysqlhost|snmysqluser)=' /opt/fog/.fogsettings
                                grep DATABASE_HOST /var/www/html/fog/commons/config.class.php
                                

                                On a storage node, installtype should be ‘S’, and snmysqlhost and DATABASE_HOST should be your main server’s IP. If they are not, that server was installed or updated as a full server. Please run the same two commands on your other storage nodes too.

                                Keep the replicators on the storage nodes stopped until this is fixed.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                F 1 Reply Last reply Reply Quote 0
                                • F
                                  FlareImp @Tom Elliott
                                  last edited by FlareImp

                                  @Tom-Elliott I figured it out and it is totally my bad. Two of my sites i forgot to delete the old answer files so when I set them up they got the information for the old server. after deleting the fog settings and going back through the setup they are working correctly. Apologies for the extra work Tom.

                                  Tom ElliottT 1 Reply Last reply Reply Quote 0
                                  • Tom ElliottT
                                    Tom Elliott @FlareImp
                                    last edited by

                                    @FlareImp No problem at all, thanks for tracking it down. Now that those two servers point at the main server’s database, you can start their replicators again:

                                    sudo systemctl enable --now FOGImageReplicator FOGSnapinReplicator
                                    

                                    After one cycle, /opt/fog/log/fogreplicator.log on each should show its own node and group, not node 1.

                                    For the two hosts that waited in the queue: the next time you deploy that lab, post the FOG storage node overflow lines from /var/log/apache2/error.log if any host waits.

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    F 1 Reply Last reply Reply Quote 0
                                    • F
                                      FlareImp @Tom Elliott
                                      last edited by

                                      @Tom-Elliott The fog replication log on both is showing the correct information. Hopefully tomorrow morning I can post about the queuing issues.

                                      1 Reply Last reply Reply Quote 1
                                      • 1
                                      • 2
                                      • 1 / 2
                                      • First post
                                        Last post

                                      58

                                      Online

                                      12.8k

                                      Users

                                      17.7k

                                      Topics

                                      157.5k

                                      Posts
                                      Copyright © 2012-2026 FOG Project