• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Multicast just hangs

Scheduled Pinned Locked Moved Solved
FOG Problems
4
30
4.2k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R
    rogalskij @george1421
    last edited by Sep 3, 2019, 2:08 PM

    @george1421 Yes, they are both on “vlan 1” for the moment. Both are on the same subnet.

    G 1 Reply Last reply Sep 3, 2019, 3:34 PM Reply Quote 0
    • S
      Sebastian Roth Moderator
      last edited by Sep 3, 2019, 2:46 PM

      @rogalskij Is your VLAN 1 bound to em1 on your FOG server?

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      R 1 Reply Last reply Sep 3, 2019, 3:27 PM Reply Quote 1
      • R
        rogalskij @Sebastian Roth
        last edited by Sep 3, 2019, 3:27 PM

        @Sebastian-Roth I checked the Apache log you mentioned, but all I see from that day doesn’t make a ton of sense to me:

        [Fri Aug 30 15:25:11.052318 2019] [mpm_prefork:notice] [pid 2739] AH00170: caught SIGWINCH, shutting down gracefully
        [Fri Aug 30 15:28:26.997884 2019] [core:notice] [pid 2727] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
        [Fri Aug 30 15:28:27.032137 2019] [suexec:notice] [pid 2727] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
        [Fri Aug 30 15:28:27.101535 2019] [lbmethod_heartbeat:notice] [pid 2727] AH02282: No slotmem from mod_heartmonitor
        PHP Warning: Module ‘ldap’ already loaded in Unknown on line 0
        [Fri Aug 30 15:28:27.268250 2019] [mpm_prefork:notice] [pid 2727] AH00163: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/7.2.21 configured – resuming normal operations
        [Fri Aug 30 15:28:27.268287 2019] [core:notice] [pid 2727] AH00094: Command line: ‘/usr/sbin/httpd -D FOREGROUND’

        Does this make sense? Am I looking at something wrong here?

        1 Reply Last reply Reply Quote 0
        • G
          george1421 Moderator @rogalskij
          last edited by Sep 3, 2019, 3:34 PM

          @rogalskij said in Multicast just hangs:

          Yes, they are both on “vlan 1” for the moment. Both are on the same subnet.

          Please clarify, you have 2 network adapters on the same subnet(vlan) do they have different IP address subnets? If they are on the same subnet, there may be an issue.

          When you have a multicast running if you run the following command from the fog server command prompt: sudo ps aux|grep udp-sender you will see the current syntax that called the multicast sender.

          So its getting all the way to partclone bits and then its failing? Can you/have you unicast this image before?

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

          R 1 Reply Last reply Sep 3, 2019, 3:43 PM Reply Quote 0
          • R
            rogalskij @george1421
            last edited by Sep 3, 2019, 3:43 PM

            @george1421 Wait, you aren’t allowed to have the FOG server on the same subnet as the clients?! This is how we do most everything right now. We plan to subnet our devices later on, but previously with Ghost and other multicast products we just multicast with devices and the server on the same subnet. Is this still possible?

            Additionally, the output of the command you specified “sudo ps aux|grep udp-sender” is:

            root 13864 0.0 0.0 115300 1480 ? S Aug30 0:00 sh -c /usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 1200 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p1.img;/usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 10 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p2.img;
            root 14393 0.0 0.0 8688 660 ? S Aug30 0:00 /usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 10 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p2.img
            root 31094 0.0 0.0 112708 992 pts/0 S+ 11:39 0:00 grep --color=auto udp-sender

            As you can see, it sees the interface em1, unless I am wrong and em1 isn’t the name of the interface, but that is what it says when I do an “ip addr” command on the server.

            G 1 Reply Last reply Sep 3, 2019, 3:46 PM Reply Quote 0
            • G
              george1421 Moderator @rogalskij
              last edited by Sep 3, 2019, 3:46 PM

              @rogalskij said in Multicast just hangs:

              Wait, you aren’t allowed to have the FOG server on the same subnet as the clients?!

              Just for clarity I read that you have 2 network interfaces on the same subnet. Is that accurate?

              So from your output command I see you have 2 multicasts running at the moment from 30-Aug.

              OK what do you get when you run the ip addr show command?

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

              R 1 Reply Last reply Sep 3, 2019, 3:49 PM Reply Quote 0
              • R
                rogalskij @george1421
                last edited by Sebastian Roth Sep 3, 2019, 2:08 PM Sep 3, 2019, 3:49 PM

                @george1421 Yes, I have the “em1” interface of the FOG server, and the network card of the Dell Computer I am trying to image on the same subnet.

                The output of the “ip addr show” command on the FOG server is:

                1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
                    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                    inet 127.0.0.1/8 scope host lo
                       valid_lft forever preferred_lft forever
                    inet6 ::1/128 scope host
                       valid_lft forever preferred_lft forever
                2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
                    link/ether d4:ae:52:af:b5:63 brd ff:ff:ff:ff:ff:ff
                    inet 150.155.1.70/20 brd 150.155.15.255 scope global noprefixroute dynamic em1
                       valid_lft 704372sec preferred_lft 704372sec
                    inet6 fe80::3d39:c85:7bf0:e61e/64 scope link noprefixroute
                       valid_lft forever preferred_lft forever
                3: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
                    link/ether d4:ae:52:af:b5:64 brd ff:ff:ff:ff:ff:ff
                
                G 1 Reply Last reply Sep 3, 2019, 3:51 PM Reply Quote 0
                • G
                  george1421 Moderator @rogalskij
                  last edited by Sep 3, 2019, 3:51 PM

                  @rogalskij ok now that we understand your hardware setup a bit more. To my question, have you ever imaged using your “BaseImage” image using unicast?

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                  R 1 Reply Last reply Sep 3, 2019, 3:53 PM Reply Quote 0
                  • R
                    rogalskij @george1421
                    last edited by Sep 3, 2019, 3:53 PM

                    @george1421 Yes, when I capture or deploy an image using Unicast, everything is happy ducky wonderful. Images capture and deploy without an issue what so ever.

                    G 1 Reply Last reply Sep 3, 2019, 4:00 PM Reply Quote 0
                    • G
                      george1421 Moderator @rogalskij
                      last edited by Sep 3, 2019, 4:00 PM

                      @rogalskij ok so here is where we are:

                      1. Its not the image because it deploys correctly using unicast
                      2. We know the installed network adapters and em1 is the correct network adapter, it has an ip address and is currently up
                      3. The ps command shows that udp-sender should be using network interface em1
                      4. The target computers and fog server is on the same vlan so no additional infrastructure work is needed.
                      5. At least some of the multicasts are getting through since the clients are able to check in and the stream starts.
                      6. It appears to hang at the partclone screen

                      We still don’t know if the infrastructure is setup correctly for multicasting (i.e. igmp snooping is enabled on vlan 1).
                      We don’t know if the multicast settings are right in the fog configuration.
                      We don’t know if the fog server’s firewall has been enabled but multicasts not allowed.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                      R 1 Reply Last reply Sep 3, 2019, 4:05 PM Reply Quote 1
                      • R
                        rogalskij @george1421
                        last edited by Sep 3, 2019, 4:05 PM

                        @george1421

                        Some additional information and questions:

                        I enabled “igmp snooping” on all of the switches and I verified that it is enabled on the switch that lab full of computers sits under.

                        I am happy to review the multicast settings. I put them in the main body of this post, do they look correct?

                        How do I check the firewall on the fog server (CentOS 7). I am pretty sure I disabled it entirely but can’t remember.

                        I did a config restore from my dev system which was a virtual machine. Could this be screwing something up? Something brought over incorrectly?

                        G 1 Reply Last reply Sep 3, 2019, 5:11 PM Reply Quote 0
                        • G
                          george1421 Moderator @rogalskij
                          last edited by Sep 3, 2019, 5:11 PM

                          @rogalskij What network switches do you use?

                          I’ve been looking back in my docs and I found a documented multicasting issue with meraki switches.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                          R 1 Reply Last reply Sep 3, 2019, 5:40 PM Reply Quote 0
                          • R
                            rogalskij @george1421
                            last edited by Sep 3, 2019, 5:40 PM

                            @george1421 We use Cisco C2960S switches in those labs. The core is also Cisco.

                            G 1 Reply Last reply Sep 3, 2019, 5:52 PM Reply Quote 0
                            • G
                              george1421 Moderator @rogalskij
                              last edited by Sep 3, 2019, 5:52 PM

                              @rogalskij Well then, lets assume the fog server is setup correctly. The firewall thing may not be an issue because the prerequisites for installing fog is the firewall being off. Some organizations, that isn’t allow so its turned on with specific rules fog needs to operate. If the multicast is not part of those rules then that function will be disabled. FWIW: systemctl stop firewalld and systemctl disable firewalld is what you need to stop and then disable the linux firewall.

                              So if you plug 2 clients into the same switch is the fog server and then schedule a multicast deployment job with max clients of 2, when that second client comes online does the multicast move forward? Now this is on the same switch as the FOG server.

                              Oh one other comment I found is to make sure you have port-fast or one of the other fast spanning tree protocols enabled on the switch.

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                              1 Reply Last reply Reply Quote 0
                              • S
                                Sebastian Roth Moderator
                                last edited by Sep 3, 2019, 8:13 PM

                                @rogalskij Sometimes old tasks can cause issues where partclone hangs on that screen. May I ask you to cancel all current tasks, reboot the FOG server and then schedule a fresh multicast task lets say for three machines (all on the same switch!). Let us know if the clients hang again.

                                If they do I ask you to run ps aux | grep sender again and post output here.

                                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                G 1 Reply Last reply Sep 3, 2019, 8:17 PM Reply Quote 0
                                • G
                                  george1421 Moderator @Sebastian Roth
                                  last edited by Sep 3, 2019, 8:17 PM

                                  @Sebastian-Roth from a previous post:

                                  Additionally, the output of the command you specified “sudo ps aux|grep udp-sender” is:
                                  
                                  root 13864 0.0 0.0 115300 1480 ? S Aug30 0:00 sh -c /usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 1200 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p1.img;/usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 10 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p2.img;
                                  root 14393 0.0 0.0 8688 660 ? S Aug30 0:00 /usr/local/sbin/udp-sender --interface em1 --min-receivers 3 --max-wait 10 --portbase 56590 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/BaseImage/d1p2.img
                                  root 31094 0.0 0.0 112708 992 pts/0 S+ 11:39 0:00 grep --color=auto udp-sender
                                  

                                  There appears to be some stale multicast tasks running since 30-Aug.

                                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                  R 1 Reply Last reply Sep 3, 2019, 9:09 PM Reply Quote 1
                                  • R
                                    rogalskij @george1421
                                    last edited by Sep 3, 2019, 9:09 PM

                                    @george1421 After a reboot there is no change. The computer is still deploying images via unicast without an issue. I updated the Kernel to the latest version, no change. One question, could I have the interfaces for multicast set wrong? Is there a way to check on the CentOS server what they are really named? Also inside the system can I check this? Just in case the WebUI is reporting it back incorrect?

                                    G 1 Reply Last reply Sep 3, 2019, 9:31 PM Reply Quote 0
                                    • G
                                      george1421 Moderator @rogalskij
                                      last edited by Sep 3, 2019, 9:31 PM

                                      @rogalskij said in Multicast just hangs:

                                      @george1421 One question, could I have the interfaces for multicast set wrong? Is there a way to check on the CentOS server what they are really named?

                                      We have already proved this out via the ip addr show what network address / adapters are in play here.

                                      Both Sebastian and I recommended to start with just 2 systems on the same switch as the fog server.

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        Sebastian Roth Moderator
                                        last edited by Sep 3, 2019, 10:20 PM

                                        @rogalskij If your new test with two hosts hangs again, may I ask you to run ps aux | grep sender again and post output here. I want to make sure it start with the correct parameters.

                                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          Sebastian Roth Moderator
                                          last edited by Sep 5, 2019, 8:46 PM

                                          @rogalskij As well I wonder if you’ve gone through our testing guide on multicast?! https://wiki.fogproject.org/wiki/index.php/Troubleshoot_Downloading_-_Multicast#Testing_Multicast

                                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                          R 2 Replies Last reply Sep 11, 2019, 12:57 PM Reply Quote 0
                                          • 1
                                          • 2
                                          • 1 / 2
                                          1 / 2
                                          • First post
                                            15/30
                                            Last post

                                          224

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project