• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    FOG nodes not in sync

    Scheduled Pinned Locked Moved Solved
    FOG Problems
    3
    20
    2.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kafluke @Sebastian Roth
      last edited by kafluke

      @Sebastian-Roth I’ll gather that info and post here. Just FYI I did a manual sync from Park City main server over to the Buffalo Node. I was going to do the same for the other two until it was working again.

      1 Reply Last reply Reply Quote 0
      • K
        kafluke @Sebastian Roth
        last edited by Sebastian Roth

        @Sebastian-Roth

        du -m --max-depth=1 /images

        Park City Master Node:

        3198    /images/TulsaPBX
        11681   /images/DellOptiplex7050Win10Pro3-1-18
        18792   /images/DellPrec5530Win10ProUEFI9-27-18
        10986   /images/DellOptiplex7040Win10Pro3-5-18
        2386    /images/DenverPBX14
        52357   /images/DellLatitude7280Win10Pro3-1-18
        16665   /images/DellLatitudeE55709-12-18
        8675    /images/DellLatitudeE6420Win7Pro3-20-18
        24192   /images/LAPBX12Backup
        4868    /images/OrlandoFreePBX12Backup
        21612   /images/DellLatitudeE7250Win10Pro3-13-18
        17532   /images/DellLatitude7290Win10Pro8-3-18
        2770    /images/ClevelandFreePBX13Backup
        956     /images/FreePBX12FreshInstall
        6514    /images/RaleighPBX12
        1730    /images/FresnoPBX14
        11903   /images/DellLatitude52902-in1Win10Pro4-19-18-UEFI
        12237   /images/LenovoX240Win10Pro4-9-18
        1679    /images/FreePBX13Generic
        5338    /images/SuperMicroServer2012R2forNAS
        3192    /images/Windows7ProGeneric64-bit
        954     /images/FreshFreePBX12
        1226    /images/FreshFreePBX13
        2342    /images/SeattlePBX14
        13003   /images/DellLatitude7290GenericTest
        8444    /images/SuperMicroWin10ProVol1709forNAS
        18261   /images/DellLatitude7490Win10Pro8-1-18
        1259    /images/St.LouisPBX-12
        1511    /images/FreePBX14Generic
        3201    /images/bigsnapins
        6650    /images/BostonPBX12backup
        12387   /images/DellLatitude5290Win10Pro3-16-18
        12979   /images/LenovoX230Updated
        2594    /images/FreePBX14-1805-3
        13403   /images/DellLatitudeE7270Win10Pro3-1-18
        24049   /images/DellLatitude7280Win10Pro3-20-18
        2520    /images/OrlandoFreePBX14
        14793   /images/DellLatitude5480Win10Pro3-1-18
        2501    /images/LAPBX14
        2426    /images/RaleighPBX14
        2645    /images/SeattlePBX12backup
        3562    /images/DetroitPBXBackup13
        4224    /images/AtlantaPBX14
        1       /images/postdownloadscripts
        2512    /images/BostonPBX14
        19018   /images/DellAlienwareBaseWin10Pro9-21-18
        26821   /images/DellLatitude5480Win10Pro6-4-18
        31185   /images/Win-Pak
        3980    /images/Win10ProGeneric
        2584    /images/St.LouisPBX-14
        3256    /images/13to14PBXupgradetest
        5874    /images/Windows10Pro64bit1709Vol
        10914   /images/LenovoX240Win7Pro3-28-18
        2673    /images/dev
        1948    /images/FreePBX13-Ext4-Base
        1       /images/lost+found
        18475   /images/DellLatitudeE5470Win10Pro3-8-18
        7164    /images/Windows10Pro64bit1803Vol
        25542   /images/DellLatitudeE5450Win10Pro3-1-18
        4452    /images/WindowsServer2016Generic
        2824    /images/PortlandPBX14
        3436    /images/PortlandPBX12Backup
        28037   /images/DellAlienwareDevelopersWin10Pro9-27-18
        16440   /images/DellOptiplex9020Win10Pro5-25-18
        2495    /images/DaytonPBX14
        4695    /images/DaytonPBX12
        618490  /images
        

        Park City Storage Node:

        3256    /images/13to14PBXupgradetest
        11587   /images/DellLatitude7280Win10Pro3-20-18
        8675    /images/DellLatitudeE6420Win7Pro3-20-18
        1       /images/postdownloadscripts
        9491    /images/DellOptiplex7050Win10Pro3-1-18
        4452    /images/WindowsServer2016Generic
        3980    /images/Win10ProGeneric
        8444    /images/SuperMicroWin10ProVol1709forNAS
        2485    /images/DaytonPBX14
        1511    /images/FreePBX14Generic
        1948    /images/FreePBX13-Ext4-Base
        12237   /images/LenovoX240Win10Pro4-9-18
        20576   /images/DellOptiplex9020Win10Pro5-25-18
        4695    /images/DaytonPBX12
        2645    /images/SeattlePBX12backup
        1831    /images/FreePBX14-1805-3
        5874    /images/Windows10Pro64bit1709Vol
        12420   /images/DellLatitudeE5450Win10Pro3-1-18
        956     /images/FreePBX12FreshInstall
        2520    /images/OrlandoFreePBX14
        31185   /images/Win-Pak
        11903   /images/DellLatitude52902-in1Win10Pro4-19-18-UEFI
        11648   /images/DellLatitudeE7270Win10Pro3-1-18
        5039    /images/AtlantaPBX14
        2349    /images/RaleighPBX14
        6650    /images/BostonPBX12backup
        5338    /images/SuperMicroServer2012R2forNAS
        3192    /images/Windows7ProGeneric64-bit
        2770    /images/ClevelandFreePBX13Backup
        3436    /images/PortlandPBX12Backup
        2512    /images/BostonPBX14
        3578    /images/DetroitPBXBackup13
        11500   /images/DellOptiplex7040Win10Pro3-5-18
        14565   /images/DellLatitudeE5470Win10Pro3-8-18
        3509    /images/St.LouisPBX-14
        2342    /images/SeattlePBX14
        2501    /images/LAPBX14
        1259    /images/St.LouisPBX-12
        1       /images/dev
        11372   /images/DellLatitudeE7250Win10Pro3-13-18
        10914   /images/LenovoX240Win7Pro3-28-18
        4868    /images/OrlandoFreePBX12Backup
        6514    /images/RaleighPBX12
        2573    /images/DenverPBX14
        14793   /images/DellLatitude5480Win10Pro3-1-18
        2824    /images/PortlandPBX14
        298697  /images
        

        Buffalo Storage Node:

        12237   /images/LenovoX240Win10Pro4-9-18
        956     /images/FreePBX12FreshInstall
        4695    /images/DaytonPBX12
        52357   /images/DellLatitude7280Win10Pro3-1-18
        11681   /images/DellOptiplex7050Win10Pro3-1-18
        2594    /images/FreePBX14-1805-3
        2520    /images/OrlandoFreePBX14
        2645    /images/SeattlePBX12backup
        1226    /images/FreshFreePBX13
        3562    /images/DetroitPBXBackup13
        24192   /images/LAPBX12Backup
        2584    /images/St.LouisPBX-14
        1259    /images/St.LouisPBX-12
        1730    /images/FresnoPBX14
        10914   /images/LenovoX240Win7Pro3-28-18
        5874    /images/Windows10Pro64bit1709Vol
        10986   /images/DellOptiplex7040Win10Pro3-5-18
        5338    /images/SuperMicroServer2012R2forNAS
        2342    /images/SeattlePBX14
        18261   /images/DellLatitude7490Win10Pro8-1-18
        2386    /images/DenverPBX14
        4868    /images/OrlandoFreePBX12Backup
        16665   /images/DellLatitudeE55709-12-18
        3198    /images/TulsaPBX
        12979   /images/LenovoX230Updated
        13403   /images/DellLatitudeE7270Win10Pro3-1-18
        1679    /images/FreePBX13Generic
        1       /images/bigsnapins
        3436    /images/PortlandPBX12Backup
        3192    /images/Windows7ProGeneric64-bit
        14793   /images/DellLatitude5480Win10Pro3-1-18
        6514    /images/RaleighPBX12
        16440   /images/DellOptiplex9020Win10Pro5-25-18
        2495    /images/DaytonPBX14
        7607    /images/WindowsServer2016Generic
        25542   /images/DellLatitudeE5450Win10Pro3-1-18
        1       /images/dev
        18475   /images/DellLatitudeE5470Win10Pro3-8-18
        8675    /images/DellLatitudeE6420Win7Pro3-20-18
        19018   /images/DellAlienwareBaseWin10Pro9-21-18
        1       /images/lost+found
        1511    /images/FreePBX14Generic
        17532   /images/DellLatitude7290Win10Pro8-3-18
        12387   /images/DellLatitude5290Win10Pro3-16-18
        7164    /images/Windows10Pro64bit1803Vol
        4224    /images/AtlantaPBX14
        3980    /images/Win10ProGeneric
        2512    /images/BostonPBX14
        8444    /images/SuperMicroWin10ProVol1709forNAS
        1948    /images/FreePBX13-Ext4-Base
        2501    /images/LAPBX14
        11903   /images/DellLatitude52902-in1Win10Pro4-19-18-UEFI
        6650    /images/BostonPBX12backup
        21612   /images/DellLatitudeE7250Win10Pro3-13-18
        2770    /images/ClevelandFreePBX13Backup
        2426    /images/RaleighPBX14
        954     /images/FreshFreePBX12
        26821   /images/DellLatitude5480Win10Pro6-4-18
        31185   /images/Win-Pak
        2824    /images/PortlandPBX14
        24049   /images/DellLatitude7280Win10Pro3-20-18
        18792   /images/DellPrec5530Win10ProUEFI9-27-18
        3256    /images/13to14PBXupgradetest
        13003   /images/DellLatitude7290GenericTest
        28037   /images/DellAlienwareDevelopersWin10Pro9-27-18
        1       /images/postdownloadscripts
        615772  /images
        

        Dallas Storage Node:

        12387   /images/DellLatitude5290Win10Pro3-16-18
        1       /images/postdownloadscripts
        14565   /images/DellLatitudeE5470Win10Pro3-8-18
        12237   /images/LenovoX240Win10Pro4-9-18
        19018   /images/DellAlienwareBaseWin10Pro9-21-18
        1948    /images/FreePBX13-Ext4-Base
        11372   /images/DellLatitudeE7250Win10Pro3-13-18
        6514    /images/RaleighPBX12
        5338    /images/SuperMicroServer2012R2forNAS
        956     /images/FreePBX12FreshInstall
        2594    /images/FreePBX14-1805-3
        2501    /images/LAPBX14
        3980    /images/Win10ProGeneric
        20576   /images/DellOptiplex9020Win10Pro5-25-18
        8444    /images/SuperMicroWin10ProVol1709forNAS
        2386    /images/DenverPBX14
        4868    /images/OrlandoFreePBX12Backup
        24495   /images/DellLatitudeE5450Win10Pro3-1-18
        11903   /images/DellLatitude52902-in1Win10Pro4-19-18-UEFI
        11587   /images/DellLatitude7280Win10Pro3-20-18
        6650    /images/BostonPBX12backup
        31185   /images/Win-Pak
        21268   /images/DellLatitude7280Win10Pro3-1-18
        5874    /images/Windows10Pro64bit1709Vol
        28037   /images/DellAlienwareDevelopersWin10Pro9-27-18
        9491    /images/DellOptiplex7050Win10Pro3-1-18
        2584    /images/St.LouisPBX-14
        2512    /images/BostonPBX14
        11500   /images/DellOptiplex7040Win10Pro3-5-18
        8675    /images/DellLatitudeE6420Win7Pro3-20-18
        2520    /images/OrlandoFreePBX14
        2342    /images/SeattlePBX14
        4452    /images/WindowsServer2016Generic
        14793   /images/DellLatitude5480Win10Pro3-1-18
        3198    /images/TulsaPBX
        3436    /images/PortlandPBX12Backup
        11648   /images/DellLatitudeE7270Win10Pro3-1-18
        2495    /images/DaytonPBX14
        3192    /images/Windows7ProGeneric64-bit
        1       /images/dev
        2824    /images/PortlandPBX14
        7164    /images/Windows10Pro64bit1803Vol
        3256    /images/13to14PBXupgradetest
        2770    /images/ClevelandFreePBX13Backup
        4695    /images/DaytonPBX12
        1259    /images/St.LouisPBX-12
        3578    /images/DetroitPBXBackup13
        2426    /images/RaleighPBX14
        2645    /images/SeattlePBX12backup
        10914   /images/LenovoX240Win7Pro3-28-18
        4224    /images/AtlantaPBX14
        1511    /images/FreePBX14Generic
        400764  /images
        

        md5sum /images/DaytonPBX14/*

        Park City Master Node:

        1e94d361fbba4d85b29d96f712e32fd9  /images/DaytonPBX14/d1.fixed_size_partitions
        d41d8cd98f00b204e9800998ecf8427e  /images/DaytonPBX14/d1.has_grub
        4993d426db2dee227e855e92d71ccaa8  /images/DaytonPBX14/d1.mbr
        12858c05d8f5d17d91923f128196688e  /images/DaytonPBX14/d1.minimum.partitions
        917677f5661fd3fe50c286f345fccbad  /images/DaytonPBX14/d1.original.fstypes
        80bd7d28edabbd6f8bf1b3a1f57c4642  /images/DaytonPBX14/d1.original.swapuuids
        85ba8c44919a82093f77daa4f41daba6  /images/DaytonPBX14/d1p1.img
        9a666c1cf93b22530ea018bfddf88b68  /images/DaytonPBX14/d1p2.img
        b57dd15de0e672a846db773a62e59f5d  /images/DaytonPBX14/d1p3.img
        af82072fe7eda65d7cacd34dd792f6fb  /images/DaytonPBX14/d1p4.ebr
        bf619eac0cdf3f68d496ea9344137e8b  /images/DaytonPBX14/d1p5.ebr
        fd0c4ed9f768eebfbe4a8461063083de  /images/DaytonPBX14/d1.partitions
        

        Park City Storage Node:

        1e94d361fbba4d85b29d96f712e32fd9  /images/DaytonPBX14/d1.fixed_size_partitions
        d41d8cd98f00b204e9800998ecf8427e  /images/DaytonPBX14/d1.has_grub
        9c6c7cfabd9ff42fb6913268284f626b  /images/DaytonPBX14/d1.mbr
        1ed8ce4056cf877bb780d8dfd67ebc4c  /images/DaytonPBX14/d1.minimum.partitions
        917677f5661fd3fe50c286f345fccbad  /images/DaytonPBX14/d1.original.fstypes
        80bd7d28edabbd6f8bf1b3a1f57c4642  /images/DaytonPBX14/d1.original.swapuuids
        a52b9eb48ce72559c3bc402b8b4926dc  /images/DaytonPBX14/d1p1.img
        27e0d1cd1f2b3cc07eee026bb5c20a77  /images/DaytonPBX14/d1p2.img
        cfd9d3831190fd003f586bd2da483c7e  /images/DaytonPBX14/d1p3.img
        af82072fe7eda65d7cacd34dd792f6fb  /images/DaytonPBX14/d1p4.ebr
        bf619eac0cdf3f68d496ea9344137e8b  /images/DaytonPBX14/d1p5.ebr
        fd0c4ed9f768eebfbe4a8461063083de  /images/DaytonPBX14/d1.partitions
        

        Buffalo Storage Node:

        1e94d361fbba4d85b29d96f712e32fd9  /images/DaytonPBX14/d1.fixed_size_partitions
        d41d8cd98f00b204e9800998ecf8427e  /images/DaytonPBX14/d1.has_grub
        4993d426db2dee227e855e92d71ccaa8  /images/DaytonPBX14/d1.mbr
        12858c05d8f5d17d91923f128196688e  /images/DaytonPBX14/d1.minimum.partitions
        917677f5661fd3fe50c286f345fccbad  /images/DaytonPBX14/d1.original.fstypes
        80bd7d28edabbd6f8bf1b3a1f57c4642  /images/DaytonPBX14/d1.original.swapuuids
        85ba8c44919a82093f77daa4f41daba6  /images/DaytonPBX14/d1p1.img
        9a666c1cf93b22530ea018bfddf88b68  /images/DaytonPBX14/d1p2.img
        b57dd15de0e672a846db773a62e59f5d  /images/DaytonPBX14/d1p3.img
        af82072fe7eda65d7cacd34dd792f6fb  /images/DaytonPBX14/d1p4.ebr
        bf619eac0cdf3f68d496ea9344137e8b  /images/DaytonPBX14/d1p5.ebr
        fd0c4ed9f768eebfbe4a8461063083de  /images/DaytonPBX14/d1.partitions
        

        Dallas Storage Node:

        1e94d361fbba4d85b29d96f712e32fd9  /images/DaytonPBX14/d1.fixed_size_partitions
        d41d8cd98f00b204e9800998ecf8427e  /images/DaytonPBX14/d1.has_grub
        9c6c7cfabd9ff42fb6913268284f626b  /images/DaytonPBX14/d1.mbr
        1ed8ce4056cf877bb780d8dfd67ebc4c  /images/DaytonPBX14/d1.minimum.partitions
        917677f5661fd3fe50c286f345fccbad  /images/DaytonPBX14/d1.original.fstypes
        80bd7d28edabbd6f8bf1b3a1f57c4642  /images/DaytonPBX14/d1.original.swapuuids
        b0d9859b58422eb99ea09444ee6129dd  /images/DaytonPBX14/d1p1.img
        9a666c1cf93b22530ea018bfddf88b68  /images/DaytonPBX14/d1p2.img
        b57dd15de0e672a846db773a62e59f5d  /images/DaytonPBX14/d1p3.img
        af82072fe7eda65d7cacd34dd792f6fb  /images/DaytonPBX14/d1p4.ebr
        bf619eac0cdf3f68d496ea9344137e8b  /images/DaytonPBX14/d1p5.ebr
        fd0c4ed9f768eebfbe4a8461063083de  /images/DaytonPBX14/d1.partitions
        
        1 Reply Last reply Reply Quote 0
        • S
          Sebastian Roth Moderator
          last edited by

          @kafluke Looking through the information you posted it seems like the Buffalo storage is pretty much in sync. Only difference at least in the size output is image WindowsServer2016Generic and bigsnapins - the later might not be an image anyways. But the other two nodes are definitely not in sync.

          While trying to figure out some other replication hickup I found and fixed a couple of issues within the code. All that will be in the next release. Hopefully coming soon. Would be interesting to test those changes in an environment like yours. Are you keen to give it a try?

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          1 Reply Last reply Reply Quote 0
          • K
            kafluke
            last edited by

            You bet! Just let me know when the update is out! Yeah, it looks like buffalo is the only one syncing fully. I had to manually sync the other two nodes.

            1 Reply Last reply Reply Quote 0
            • S
              Sebastian Roth Moderator
              last edited by

              @mronh The current changes are on a new branch replication (link) which I will merge into working after a first round of feedback. It will all be in the next official release.

              Not sure if you have ever installed FOG unstable/testing. This is done using git to checkout the current code and install from that.

              git clone https://github.com/FOGProject/fogproject/
              cd fogproject
              git checkout replication
              cd bin
              ./installfog.sh
              

              Important notice: I had to change some of the hashing code too and therefore nodes being on different versions (1.5.4 or working VS. replication branch) will end up replicating images over and over again. So you need to have all nodes on the replication branch or setup up a separate test environment!!

              Please make sure you stop replication first (systemctl stop FOGImageReplicator), then update the storage node and after that update master node.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              K 5 Replies Last reply Reply Quote 0
              • K
                kafluke @Sebastian Roth
                last edited by

                @Sebastian-Roth This is what I get when I try and checkout replication:

                error: pathspec ‘replication’ did not match any file(s) known to git.

                1 Reply Last reply Reply Quote 0
                • K
                  kafluke @Sebastian Roth
                  last edited by

                  @Sebastian-Roth never mind. I had to remove what I had checked out already “rm -rf /fogproject”

                  1 Reply Last reply Reply Quote 0
                  • K
                    kafluke @Sebastian Roth
                    last edited by

                    @Sebastian-Roth Okay I updated all 3 nodes to the “replication” branch and then I updated the master and restarted the sync service.

                    1 Reply Last reply Reply Quote 1
                    • K
                      kafluke @Sebastian Roth
                      last edited by

                      @Sebastian-Roth It seems to be properly sync-ing now. I’ll check it tomorrow after a day of sinc-ing and let you know the results. The Dallas FOG Node is definitely growing in data

                      1 Reply Last reply Reply Quote 0
                      • S
                        Sebastian Roth Moderator
                        last edited by

                        @kafluke Thanks for the feedback so far. Keep an eye on the logs and let us know if you feel this is working properly.

                        You might want to do a quick file size check on all your nodes tomorrow:

                        du -k --max-depth=1 /images | sort -k2 > /tmp/imagelist_xxx_node.txt
                        

                        Do that on all four nodes and copy those files over to your PC. Then use some kind of diff tool (e.g. www.quickdiff.com) to compare those file contents.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        K 2 Replies Last reply Reply Quote 0
                        • K
                          kafluke @Sebastian Roth
                          last edited by

                          @Sebastian-Roth It’s definitely working now. I’m watching the hard drive space on my Dallas node steadily go up. Thanks for fixing this!

                          1 Reply Last reply Reply Quote 0
                          • K
                            kafluke @Sebastian Roth
                            last edited by

                            @Sebastian-Roth Ok it worked. All nodes are now in sync with the master node! Thanks again for the help! Looking forward to the next release

                            1 Reply Last reply Reply Quote 0
                            • S
                              Sebastian Roth Moderator
                              last edited by

                              @kafluke Thanks for the feedback! Do you see “file mismatches” anymore in the logs now?

                              Will mark this solved now. Those fixes will be merged into the working branch and from there go into the next release.

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              K 1 Reply Last reply Reply Quote 0
                              • K
                                kafluke @Sebastian Roth
                                last edited by

                                @Sebastian-Roth No more “file mismatches”. Thanks again.

                                1 Reply Last reply Reply Quote 0
                                • K
                                  kafluke @Sebastian Roth
                                  last edited by

                                  @Sebastian-Roth Did you ever merge this into working branch? I had to start over from scratch on a new FOG server and wanted to know if I need to check out “replication” or “working”

                                  Tom ElliottT 1 Reply Last reply Reply Quote 0
                                  • Tom ElliottT
                                    Tom Elliott @kafluke
                                    last edited by

                                    @kafluke You’d want to checkout dev-branch.

                                    Working is more or less a stand in for Alpha RC’s now.

                                    Dev-branch is a bug fix branch

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    1 Reply Last reply Reply Quote 0
                                    • 1 / 1
                                    • First post
                                      Last post

                                    162

                                    Online

                                    12.1k

                                    Users

                                    17.3k

                                    Topics

                                    155.3k

                                    Posts
                                    Copyright © 2012-2024 FOG Project