• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS

Scheduled Pinned Locked Moved Unsolved
FOG Problems
4
35
1.0k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • F
    Fog_Newb @JJ Fullmer
    last edited by Sep 16, 2024, 6:03 PM

    @JJ-Fullmer said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

    @Fog_Newb I am able to recreate the problem in the latest stable version (1.5.10.1615). Working it out. it looks like the image does stay in dev on my end.
    I’ll debug this when I can this week.

    Awesome. Thanks.

    Then you could at least manually move the image from dev to images to workaround the issue while we work this out.

    I have another FOG server set up where /images and /images/dev are on the same drive so I can continue using that until things get sorted.

    T 1 Reply Last reply Sep 16, 2024, 7:22 PM Reply Quote 0
    • J
      JJ Fullmer Testers @JJ Fullmer
      last edited by Sep 16, 2024, 7:12 PM

      @Fog_Newb I think I found the problem.

      Normally the ftp_rename command successfully handles moving the captured image from /images/dev to /images but for some reason that command ( a built in php/ftp thing) doesn’t like it when the directories are mounted on separate disks. Not sure why, doesn’t make a ton of sense, but that’s the issue.

      In Fog, when this fails, we attempt an ftp_put command aka an upload.
      This should work, but there’s a path mismatch at the moment.
      During capture the “local” file where it is captured is at /images/{mac} instead of /images/dev/{mac} and the put command is getting the latter because that’s what we already have for the rename command that happens completely remotely.
      So I’m testing some stuff to fix this in the dev-branch.

      Basically I think you may have found a bug in a failsafe method for uploading a captured image.

      Have you tried the FogApi powershell module? It's pretty cool IMHO
      https://github.com/darksidemilk/FogApi
      https://fogapi.readthedocs.io/en/latest/
      https://www.powershellgallery.com/packages/FogApi
      https://forums.fogproject.org/topic/12026/powershell-api-module

      F 1 Reply Last reply Sep 16, 2024, 11:18 PM Reply Quote 0
      • T
        Tom Elliott @Fog_Newb
        last edited by Sep 16, 2024, 7:22 PM

        @Fog_Newb @JJ-Fullmer I wonder if this would work better separated out:

        mkdir /images1
        mkdir /imagesdev1
        
        mv /images/dev/* /imagesdev1 #everything here to imagesdev1 for example
        rm -rf /images/dev #only after moving everything first (including . files/directories)
        mv /images/* /images1
        
        rm -rf /images #only after moving everything first of course)
        
        ln -s /images1 /images
        ln -s /imagesdev1 /images/dev
        

        Does this work?

        Of course your fstab would need adjusting if it does, but jus trying to think what may be causing issue is that a root system (/images for example) doesn’t know how to handle the inode movement occurring from another filesystem (/images/dev for example) If you were to move /imagesdev1/file to /images1/file it should work just fine because you’re crossing filesystems which doesn’t pose a problem due to the root structures actually being at 2 different physical points. I believe this is a limitation of linux, not necessary a limitation of FOG persay.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

        F J 3 Replies Last reply Sep 16, 2024, 11:14 PM Reply Quote 0
        • F
          Fog_Newb @Tom Elliott
          last edited by Fog_Newb Sep 16, 2024, 8:27 PM Sep 16, 2024, 11:14 PM

          @Tom-Elliott said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

          I will definitely try this. Just to make sure I understand how you want these new directories mounted…

          sudo mkdir /images1
          sudo mkdir /imagesdev1
          

          Then adjust fstab so ‘/images1’ is a mount point for one drive (where /images was mounted), and ‘/imagesdev1’ is a mount point for the other drive (where /images/dev was mounted. Mount them, once mounted then I do…

          After mounting these, new mount points, everything that was in /images and /images/dev is now in /images1 and /imagesdev1

          sudo rm -rf  /images
          sudo rm -rf  /images/dev
          sudo ln -s /images1 /images
          sudo ln -s /imagesdev1 /images/dev
          

          Then capture an image and see what happens

          1 Reply Last reply Reply Quote 0
          • F
            Fog_Newb @JJ Fullmer
            last edited by Fog_Newb Sep 16, 2024, 8:05 PM Sep 16, 2024, 11:18 PM

            @JJ-Fullmer said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

            @Fog_Newb I think I found the problem.

            Normally the ftp_rename command successfully handles moving the captured image from /images/dev to /images but for some reason that command ( a built in php/ftp thing) doesn’t like it when the directories are mounted on separate disks. Not sure why, doesn’t make a ton of sense, but that’s the issue.

            That makes sense to me especially if that command is only able to rename.

            Basically I think you may have found a bug in a failsafe method for uploading a captured image.

            Cool cool?

            J 1 Reply Last reply Sep 17, 2024, 5:35 PM Reply Quote 0
            • F
              Fog_Newb @Tom Elliott
              last edited by Fog_Newb Sep 16, 2024, 6:27 PM Sep 17, 2024, 12:06 AM

              @Tom-Elliott

              Ok I think I have it set up how you asked

              New mount points

              see

              see1

              see2

              Symlinks

              see3

              see4

              Gonna try to capture will report back

              Same thing.

              err

              1 Reply Last reply Reply Quote 0
              • J
                JJ Fullmer Testers @Tom Elliott
                last edited by Sep 17, 2024, 5:32 PM

                @Tom-Elliott @Fog_Newb

                I tried this symlink method on my test server and didn’t have success sadly. Seems nfs doesn’t follow symlinks.
                My theory on the put function was also wrong.

                How big of an issue is this disk size inflation for you? Typically you’re only capturing 1 image at a time, so you need your /images disk to have
                (enough space for all your images) + (the size of your biggest image) + (maybe 10% of that for flexibility)
                So if your current images take up say 500 GB, and the biggest of them was 100GB (which would be a very big image)
                Then provisioning 660 GB should be sufficient if you didn’t want to just double the provisioned space.
                If you want to have space to capture them all at the same time, then 1.1 TB would do the trick.
                If your images are just really really big, then consider using snapins or other software deployment methods to keep your images to just the base os and a limited number of default apps so that they stay on the smaller side. For examples, my Windows 11 images take up just under 20 GB on the server and around 30 GB on my hosts and then I add the rest per use case within my environment.

                You can also play with the different compression levels of the image. I’ve found a zstd compression level of 11 to be a good balance of speed and size.
                You could also throw caution to the wind and not care about the speed of deploy and capture and just put the compression to max if you’re more concerned about taking less server disk space.

                The only thing I can think of to make this work natively would require database schema changes so you can define separate paths for /images and /images/dev and reworking the install script to make sure the /etc/exports is created correctly. Sadly I don’t think that amount of effort would benefit any other use case and this is an uncommon use case that’s not in our realm of support right now, especially since it’s mostly due to limitations in linux.

                The is one other possibility, and that would be a postinit script, but those run before imaging starts not after like postdownload scripts. But there’s other security issues there with how you’d get the ro /images share to be mounted as rw when your task only has permissions to open /images/dev which does have rw. So you’d have to open an ssh or sftp session and send the command to move the folder from /images/dev to /images that way. But securely creating that connection at that point is a different beast. It’s odd that linux/php are having this limitation but that’s where we’re at.

                Have you tried the FogApi powershell module? It's pretty cool IMHO
                https://github.com/darksidemilk/FogApi
                https://fogapi.readthedocs.io/en/latest/
                https://www.powershellgallery.com/packages/FogApi
                https://forums.fogproject.org/topic/12026/powershell-api-module

                F 1 Reply Last reply Sep 17, 2024, 10:00 PM Reply Quote 0
                • J
                  JJ Fullmer Testers @Fog_Newb
                  last edited by Sep 17, 2024, 5:35 PM

                  @Fog_Newb said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                  That makes sense to me especially if that command is only able to rename.

                  Just for clarity. Rename in this context also handles moving the file. By “renaming” the file from its old path of /images/dev/$mac to /images/$imagename it’s really moving the file. But since those paths are on different filesystems, it doesn’t work.
                  The php ftp_put attempts to use an upload of the file from the server to the other path, but that is when it’s actually making the empty file because it gets confused by the paths being on different disks.

                  Have you tried the FogApi powershell module? It's pretty cool IMHO
                  https://github.com/darksidemilk/FogApi
                  https://fogapi.readthedocs.io/en/latest/
                  https://www.powershellgallery.com/packages/FogApi
                  https://forums.fogproject.org/topic/12026/powershell-api-module

                  F 1 Reply Last reply Sep 29, 2024, 10:30 PM Reply Quote 0
                  • F
                    Fog_Newb @JJ Fullmer
                    last edited by Fog_Newb Sep 17, 2024, 4:01 PM Sep 17, 2024, 10:00 PM

                    @JJ-Fullmer said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                    How big of an issue is this disk size inflation for you?

                    It’s not a big deal. I’ve since stopped capturing non-OS drives and now use file sync / mirror program to back those up. I actually like that better than huge image captures once a week. Currently, FOG is just capturing 2 relatively small OS drive images.

                    All this is running on a QNAP box someone gave me and QNAP’s Virtualization Station software is lacking features to cleanup/shrink or compact virtual machine disks.

                    This all started out as a Feature Request

                    https://forums.fogproject.org/topic/17651/local-time-and-alternate-temp-directory-for-capture-option

                    It’s odd that linux/php are having this limitation but that’s where we’re at.

                    Is this something the devs would change if it was brought to their attention?

                    Thanks

                    1 Reply Last reply Reply Quote 0
                    • F
                      Fog_Newb @JJ Fullmer
                      last edited by Fog_Newb Sep 29, 2024, 4:35 PM Sep 29, 2024, 10:30 PM

                      @JJ-Fullmer @Tom-Elliott

                      I’ve found a work around. I now have /images on USB HDD passed through the to VM. When I run a standard back up of the VM via the VM software (Virtualization Station), it only back-ups the OS VM drive.

                      For the /images drive - I use a sync program to back-up the files, not the disk .

                      1 Reply Last reply Reply Quote 0
                      • 1
                      • 2
                      • 2 / 2
                      • First post
                        Last post

                      199

                      Online

                      12.0k

                      Users

                      17.3k

                      Topics

                      155.2k

                      Posts
                      Copyright © 2012-2024 FOG Project