• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Multicast Issues on Centos 7

    Scheduled Pinned Locked Moved Solved
    FOG Problems
    5
    61
    43.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      BedCruncher @Sebastian Roth
      last edited by

      @Sebastian-Roth said in Multicast Issues on Centos 7:

      exec(“/sbin/ip addr | awk -F’[ /]+’ ‘/global/ {print $3}’”,$IPs,$retVal);

      No worries Sebastian, I’m just trying to get the exact information you’re requesting. Want to make sure that you get the pertinent information for your question so that we can get this figured out 100%.

      I changed the function back to what you posted from current trunk. I had attempted to change it in relation to the
      FOG .32 Centos 7 Instructions, but to no avail.

      Keep the ideas coming. I’d love to put this one to bed and get back to using this awesome software!

      1 Reply Last reply Reply Quote 0
      • S
        Sebastian Roth Moderator
        last edited by

        @BedCruncher Ok, I finally had some time to spin up my test machine and give it a try. Setting ens4 instead of eth0 in the storage settings I get this in the logs:

        [04-26-16 9:46:58 pm]  | CMD: cat /images/foo/d1p1.img | /usr/local/sbin/udp-sender --interface ens4 --min-receivers 2 --max-wait 600 --portbase 58038 --full-duplex --ttl 32 --nokbd --nopointopoint...

        As well I see the command (options) in the process tree on the FOG server with ps ax | grep udp:

        3350 ?        S      0:00 sh -c cat /images/foo/d1p1.img | /usr/local/sbin/udp-sender --interface ens4 --min-receivers 2 --max-wait 600 --portbase 62258 --full-duplex --ttl 32 --nokbd --nopointopoint

        This is with FOG trunk unmodified. I don’t think you need to change fogbase.class.php! Please see what you have in your process list on the FOG server after starting the multicast task.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        B 1 Reply Last reply Reply Quote 0
        • B
          BedCruncher @Sebastian Roth
          last edited by BedCruncher

          @Sebastian-Roth

          So I started the multicast session from my server this morning and checked the Multicast Log from the webgui, but I am not seeing that that command has fired off in the logs. Also, I don’t see the udp task showing up in my process list. Excuse my ignorance, but did I do something incorrect?

          EDIT:
          The command to at least trigger the process has worked though as it starts the process of trying to do the imaging but hangs at the same spot as before. Any guidance would be helpful.

          Just realized maybe you were referring to have me run the UDP-Sender / UDP-Receiver commands manually?

          1 Reply Last reply Reply Quote 0
          • S
            Sebastian Roth Moderator
            last edited by

            @BedCruncher said:

            Just realized maybe you were referring to have me run the UDP-Sender / UDP-Receiver commands manually?

            Nope, I was asking you to run the command ps ax | grep udp and show the output to us. From what I get you don’t seem to have any udp-sender processes running after you started multicast task. Please make sure you have a valid image on your server: ls -al /images/<img-name> (please post the output here as well!)…

            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

            B 1 Reply Last reply Reply Quote 0
            • B
              BedCruncher @Sebastian Roth
              last edited by BedCruncher

              @Sebastian-Roth

              Here’s the result of the

              ls -al

              ls -al /images/
total 4
drwxrwxrwx.  5 root root   81 Apr 27 10:39 .
dr-xr-xr-x. 20 root root 4096 Apr 27 10:08 ..
drwxrwxrwx.  2 root root   22 Apr 27 10:39 dev
-rwxrwxrwx.  1 root root    0 Apr 27 08:48 .mntcheck
drwxrwxrwx.  2 root root   29 Apr 27 08:48 postdownloadscripts
drwxrwxrwx.  2 root root   69 Apr 27 10:39 W7Px64PreSysprep

              and

              ps -aux | grep udp

              ps aux | grep udp
root     25751  0.0  0.0 112644   960 pts/1    S+   10:45   0:00 grep --color=auto udp
              1 Reply Last reply Reply Quote 0
              • S
                Sebastian Roth Moderator
                last edited by Sebastian Roth

                @BedCruncher No UDP-sender processes… please check you multicast.log to see if it actually starts any…

                As stated in one of my earlier posts I see this in my multicast.log:

                [04-26-16 9:46:58 pm]  | CMD: cat /images/foo/d1p1.img | /usr/local/sbin/udp-sender --interface ens4 ...

                You might want to upgrade to the latest version as Tom has fixed a bug which truncated the log file every so often. So you might miss the output. Upgrade and then see what you have in the logfile!

                As well, show us what you have in the image directory: ls -al /images/W7Px64PreSysprep

                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                B 1 Reply Last reply Reply Quote 0
                • B
                  BedCruncher @Sebastian Roth
                  last edited by

                  @Sebastian-Roth
                  Results of

                   ls -al /images/W7Px64PreSysprep/
total 22338008
drwxrwxrwx. 2 root root          69 Apr 27 10:39 .
drwxrwxrwx. 5 root root          81 Apr 27 10:39 ..
-rwxrwxrwx. 1 root root     1048576 Apr 27 10:12 d1.mbr
-rwxrwxrwx. 1 root root     8561702 Apr 27 10:12 d1p1.img
-rwxrwxrwx. 1 root root 22864501047 Apr 27 10:39 d1p2.img
-rwxrwxrwx. 1 root root         190 Apr 27 10:12 d1.partitions

                  On a side note, I went in earlier and completely wiped out my current install according to Uninstall Fog and then reinstalled earlier this morning and the multicast portion didn’t show up at all until manually ran touch /opt/fog/log/multicast.log and the selection was also missing from the Log Viewer menu. I verified the file permissions of that file compared to the other longs under that directory.

                  Just downloaded build 7348 and am now getting
                  0_1461777264325_upload-aaa9be64-165d-4547-9748-779c94f5ade1

                  1 Reply Last reply Reply Quote 0
                  • B
                    BedCruncher
                    last edited by BedCruncher

                    Following that screen, I let it run for a bit, then while it was doing the invalid session scrolling, I deleted and reinitiated a multicast session and it’s hanging at the same spot as previous.
                    0_1461780367043_upload-d0a02c2d-9ddc-4537-a9ec-9211330574d4

                    1 Reply Last reply Reply Quote 0
                    • S
                      Sebastian Roth Moderator
                      last edited by Sebastian Roth

                      @BedCruncher I am very sorry but without information (log file for example) I am not able to help. Posting the partclone picture is nice but does not really help I am afraid. Hope this does not sound ignorant, it’s not meant to!

                      We need to get some log output from the FOGMulticastManager (multicast.log)! Or at least see the udp-sender processes on the FOG server. Otherwise multicast will not work. Please try this:

                      service FOGMulticastManager restart
tail -f /var/log/fog/multicast.log

                      Notice the log path! Multicast daemon writes the log to /var/log…

                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                      B 1 Reply Last reply Reply Quote 0
                      • B
                        BedCruncher @Sebastian Roth
                        last edited by BedCruncher

                        @Sebastian-Roth said in Multicast Issues on Centos 7:

                        service FOGMulticastManager restart
                        tail -f /var/log/fog/multicast.log

                        Below is what that shows.

                        tail -f /var/log/fog/multicast.log

                        Can you verify the permissions that you have on that logfile? such as user, group, and file permissions? No, I completely understand, I was just trying to post that in the hopes that it would provide at least some extra information for your troubleshooting.

                        On my system the path /var/log/fog is actually a symlink to /opt/fog/log and I didn’t create this manually. Had to have been put in place on the installation.

                        As near as I can tell fog is never actually writing to the multicast.log file. It was only getting what I had posted earlier when it was manually ran during the udp-sender/udp-receiver testing.

                        1 Reply Last reply Reply Quote 0
                        • S
                          Sebastian Roth Moderator
                          last edited by

                          @BedCruncher Did you tun the service restart??? What do you get from this command? ps ax | grep FOG

                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                          B 1 Reply Last reply Reply Quote 0
                          • B
                            BedCruncher @Sebastian Roth
                            last edited by

                            @Sebastian-Roth

                            Just finished the upgrade to 7356 and ran

                            ps ax | grep FOG
16001 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
16004 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
16020 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator &
16023 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator &
16039 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator &
16042 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator &
16058 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler &
16061 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler &
16073 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts &
16076 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts &
16388 pts/1    S+     0:00 grep --color=auto FOG

                            I can see that they are running as expected.

                            1 Reply Last reply Reply Quote 0
                            • S
                              Sebastian Roth Moderator
                              last edited by Sebastian Roth

                              @Tom-Elliott Turns out something is wrong with the services logs in current trunk! I don’t see multicast.log on a fresh latest install… Service is properly running. Probably something to do with the static/self switch I suppose?!?

                              FOGMulticastManager[16754]: PHP Warning:  fopen(): Filename cannot be empty in /var/www/html/fog/lib/service/fogservice.class.php on line 82
FOGMulticastManager[16754]: PHP Warning:  fopen(): Filename cannot be empty in /var/www/html/fog/lib/service/fogservice.class.php on line 91

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              Tom ElliottT 1 Reply Last reply Reply Quote 0
                              • Tom ElliottT
                                Tom Elliott @Sebastian Roth
                                last edited by

                                @Sebastian-Roth Should be fixed in latest, sorry.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                1 Reply Last reply Reply Quote 0
                                • B
                                  BedCruncher
                                  last edited by

                                  I’ve updated to version 7386 to try and get this part of things fixed. Will post back once I test this.

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    BedCruncher
                                    last edited by BedCruncher

                                    I am now getting logging working, but

                                    [04-28-16 7:46:38 am] 
 ___ ___ ___ 
 /\ \ /\ \ /\ \ 
 /::\ \ /::\ \ /::\ \ 
 /:/\:\ \ /:/\:\ \ /:/\:\ \ 
 /::\-\:\ \ /:/ \:\ \ /:/ \:\ \ 
 /:/\:\ \:\__\ /:/__/ \:\__\ /:/__/_\:\__\ 
 \/__\:\ \/__/ \:\ \ /:/ / \:\ /\ \/__/ 
 \:\__\ \:\ /:/ / \:\ \:\__\ 
 \/__/ \:\/:/ / \:\/:/ / 
 \::/ / \::/ / 
 \/__/ \/__/ 

 ###########################################
 # Free Computer Imaging Solution #
 # Credits: #
 # http://fogproject.org/credits #
 # GNU GPL Version 3 #
 ###########################################

[04-28-16 7:46:38 am] Interface Ready with IP Address: xx.xx.xx.xx
[04-28-16 7:46:38 am] Interface Ready with IP Address: xx.xx.xx.xx
[04-28-16 7:46:38 am] Interface Ready with IP Address: 192.168.240.10
[04-28-16 7:46:38 am] Interface Ready with IP Address: REMOVED
[04-28-16 7:46:38 am] Interface Ready with IP Address: REMOVED
[04-28-16 7:46:38 am] * Starting MulticastManager Service
[04-28-16 7:46:38 am] * Checking for new items every 10 seconds
[04-28-16 7:46:38 am] * Starting service loop
[04-28-16 7:46:38 am] * No tasks found!
[04-28-16 7:46:48 am] * No tasks found!
[04-28-16 7:46:58 am] * No tasks found!
[04-28-16 7:47:09 am] * No tasks found!
[04-28-16 7:47:19 am] * No tasks found!
[04-28-16 7:47:29 am] * No tasks found!
[04-28-16 7:47:39 am] * No tasks found!
[04-28-16 7:47:49 am] * No tasks found!
[04-28-16 7:48:00 am] * No tasks found!
[04-28-16 7:48:10 am] * No tasks found!
[04-28-16 7:48:20 am] * No tasks found!
[04-28-16 7:48:30 am] * No tasks found!
[04-28-16 7:48:40 am] * No tasks found!
[04-28-16 7:48:50 am] * No tasks found!
[04-28-16 7:49:01 am] * No tasks found!
[04-28-16 7:49:11 am] * No tasks found!
[04-28-16 7:49:21 am] * No tasks found!
[04-28-16 7:49:31 am] * No tasks found!
[04-28-16 7:49:41 am] * No tasks found!
[04-28-16 7:49:51 am] * No tasks found!
[04-28-16 7:50:02 am] | Sleeping for 10 seconds to ensure tasks are properly submitted
[04-28-16 7:50:12 am] | 0 tasks to be cleaned
[04-28-16 7:50:12 am] | 1 task found
[04-28-16 7:50:12 am] | Task (4) Multi-Cast Task is new!
[04-28-16 7:50:12 am] | Task (4) Multi-Cast Task has been cleaned.
Udp-sender 20120424
[04-28-16 7:50:12 am] | Task (4) /images/W7Px64PreSysprep image file found.
Using mcast address 232.168.240.10
UDP sender for (stdin) at 192.168.240.10 on em2 
Broadcasting control to 224.0.0.1
[04-28-16 7:50:12 am] | Task (4) 2 client(s) found.
[04-28-16 7:50:12 am] | Task (4) Multi-Cast Task sending on base port: 50028
[04-28-16 7:50:12 am] | CMD: cat /images/W7Px64PreSysprep/d1p1.img | /usr/local/sbin/udp-sender --interface em2 --min-receivers 2 --max-wait 600 --portbase 50028 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/W7Px64PreSysprep/d1p2.img | /usr/local/sbin/udp-sender --interface em2 --min-receivers 2 --max-wait 600 --portbase 50028 --full-duplex --ttl 32 --nokbd --nopointopoint;
[04-28-16 7:50:12 am] | Task (4) Multi-Cast Task has started.
[04-28-16 7:50:22 am] | 0 tasks to be cleaned
[04-28-16 7:50:22 am] | 1 task found
[04-28-16 7:50:22 am] | Task (4) Multi-Cast Task is already running PID 13546
[04-28-16 7:50:32 am] | 0 tasks to be cleaned
[04-28-16 7:50:32 am] | 1 task found
[04-28-16 7:50:32 am] | Task (4) Multi-Cast Task is already running PID 13546
[04-28-16 7:50:42 am] | 0 tasks to be cleaned
[04-28-16 7:50:42 am] | 1 task found

                                    the above after letting it for quite a few minutes the above ending to this log gets repeatedly entered. I also ran the ran the command ps aux | grep udp

                                    ps aux | grep udp
root     13546  0.0  0.0 115240  1456 ?        S    07:50   0:00 sh -c cat /images/W7Px64PreSysprep/d1p1.img | /usr/local/sbin/udp-sender --interface em2 --min-receivers 2 --max-wait 600 --portbase 50028 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/W7Px64PreSysprep/d1p2.img | /usr/local/sbin/udp-sender --interface em2 --min-receivers 2 --max-wait 600 --portbase 50028 --full-duplex --ttl 32 --nokbd --nopointopoint;
root     13548  0.0  0.0   8640   664 ?        S    07:50   0:00 /usr/local/sbin/udp-sender --interface em2 --min-receivers 2 --max-wait 600 --portbase 50028 --full-duplex --ttl 32 --nokbd --nopointopoint
root     13753  0.0  0.0 112644   964 pts/1    S+   07:57   0:00 grep --color=auto udp

                                    Still hangs at the same partclone screen as before.

                                    ps ax | grep FOG
13062 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
13065 ?        S      0:06 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
13081 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator &
13084 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGImageReplicator/FOGImageReplicator &
13100 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator &
13103 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGSnapinReplicator/FOGSnapinReplicator &
13119 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler &
13122 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGTaskScheduler/FOGTaskScheduler &
13134 ?        Ss     0:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts &
13137 ?        S      0:00 /usr/bin/php -q /opt/fog/service/FOGPingHosts/FOGPingHosts &

ps aux | grep Multicast
root     13062  0.0  0.8 323076 15852 ?        Ss   07:46   0:00 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
root     13065  0.4  0.6 411168 13028 ?        S    07:46   0:05 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager &
root     14024  0.0  0.0 112648   960 pts/1    S+   08:06   0:00 grep --color=auto Multicast
                                    Tom ElliottT 1 Reply Last reply Reply Quote 0
                                    • Tom ElliottT
                                      Tom Elliott @BedCruncher
                                      last edited by

                                      @BedCruncher What’s output if you go to (in the browser)

                                      http://ip.of.fog.here/fog/service/ipxe/boot.php?mac=macofhostwithcolons

                                      Where macofhostwithcolons is the mac address (with colons) of the host that’s setup to do this tasking?

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                      B 1 Reply Last reply Reply Quote 0
                                      • B
                                        BedCruncher @Tom Elliott
                                        last edited by

                                        @Tom-Elliott
                                        Here’s the info. I also compared it to a second host that I had added to a group for multitasking, and it appears identical, barring the mac address of course. Let me know what else you can recommend and I’ll be glad to accommodate.

                                        #!ipxe
set fog-ip 192.168.240.10
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=192.168.240.10/fog/ consoleblank=0 rootfstype=ext4 rootfstype=ext4 mac=00:26:b9:aa:14:ec ftp=192.168.240.10 storage=192.168.240.10:/images/ storageip=192.168.240.10 web=192.168.240.10/fog/ osid=5 consoleblank=0 irqpoll hostname=0026b9aa14ec chkdsk=0 img=W7Px64PreSysprep imgType=mps imgPartitionType=all imgid=1 imgFormat= PIGZ_COMP=-6 hostearly=1 port=50028 type=down mc=yes
imgfetch init_32.xz
boot
                                        Tom ElliottT 1 Reply Last reply Reply Quote 0
                                        • Tom ElliottT
                                          Tom Elliott @BedCruncher
                                          last edited by

                                          @BedCruncher When the client boots to begin the tasking, is it on a separate subnet from the 192.168.240.10 server?

                                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                          B 1 Reply Last reply Reply Quote 0
                                          • B
                                            BedCruncher @Tom Elliott
                                            last edited by BedCruncher

                                            @Tom-Elliott
                                            No it’s not. I have the em2 interface controlling the DHCP server and have a cable running to a 5 port Gig switch. It’s a dumb switch with no special routing or anything. I even checked under the dhcpd.leases file and can find it there.

                                            lease 192.168.240.30 {
  starts 4 2016/04/28 12:50:28;
  ends 4 2016/04/28 18:50:28;
  cltt 4 2016/04/28 12:50:28;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet 00:26:b9:aa:14:ec;
  uid "\001\000&\271\252\024\354";
}

                                            EDIT:
                                            Updated to build 7410 this morning to hopefully see if it might have been fixed.
                                            EDIT 2:
                                            Updated and initiated a new test and still getting the same results.

                                            B 1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 4
                                            • 2 / 4
                                            • First post
                                              Last post

                                            196

                                            Online

                                            12.0k

                                            Users

                                            17.3k

                                            Topics

                                            155.2k

                                            Posts
                                            Copyright © 2012-2024 FOG Project