• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS

Scheduled Pinned Locked Moved Unsolved
FOG Problems
4
35
1.0k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • F
    Fog_Newb
    last edited by Fog_Newb Sep 12, 2024, 9:33 AM Sep 12, 2024, 10:57 AM

    This is long. My FOG server has /images as a mount point for a 2nd drive. Everything was working fine. The /images mount was set up prior to installing FOG.

    With FOG storing temp images in /images/dev, it was really inflating the disk space on my VM drive. As some of you know, with VM drives, the size of the drive increases as data is added to it.

    So I decided to make /images/dev a mount point on a 3rd drive. I made sure to copy all the exiting files and permissions from the original /images/dev.

    After re-capturing an image, I noticed something peculiar, The temp image remained in /images/dev and the directory where that image was initially stored (/images/NAMEOFIMAGE), was no longer seen as directory. But it wasn’t a symbolic link either. This only happened after trying to capture the images using the new mount.

    Just to make sure it wasn’t something I messed up copying and setting permissions on the newly created /images/dev mount, I spun up a new VM specifying the mount points during install. Then installed FOG

    But… the same thing happened.

    See

    Another

    Another

    It’s looking like FOG has issues if the temp storage location is on a different volume than the images.

    I know this isn’t a big deal in the grand scheme of things and most users will never run into this but would be cool if there was an easy fix/change that could be implemented in the next release.

    Quick TLDR. incase my word salad OP doesn’t make sense.

    If /images/dev is on a different mount point than /images, captured images will remain in their temporary location (/images/dev) and a directory for these newly captured images will not be created in /images

    Thank you

    T 1 Reply Last reply Sep 12, 2024, 11:44 AM Reply Quote 0
    • T
      Tom Elliott @Fog_Newb
      last edited by Tom Elliott Sep 12, 2024, 5:46 AM Sep 12, 2024, 11:44 AM

      @Fog_Newb I’m not sure I follow th eproblem.

      Your directory structure looks correct to me?

      Now, there is a piece to be thinking of first:

      Your mounts seem to be correct and even a df -h seems to show up properly (lsblk if you want use that which is displayed)

      The only way I could think of how this should occur.

      You should:

      1. Unmount vdc1
      2. move vdb’s dev folder to your /root/folder for backup purposes temporarily (completely remove it from the /images
      3. create a new dev folder in your /images
      4. Mount vdc1 to /images/dev
      5. Copy back the items in /root/dev/ to the new /images/dev
      6. Make sure the requisite .mntcheck file exists touch /images/.mntcheck; touch /images/dev/.mntcheck
      7. Change the ownership back to chown -R fogproject:fogproject /images; chmod 775 -R /images

      This should allow your system to work without issues.

      Your FSTAB needs to have the mount set correctly too so that dev gets mounted appropriately on subsequent boots which I presume is already happening?

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      F 2 Replies Last reply Sep 12, 2024, 12:26 PM Reply Quote 0
      • F
        Fog_Newb @Tom Elliott
        last edited by Fog_Newb Sep 12, 2024, 6:36 AM Sep 12, 2024, 12:26 PM

        @Tom-Elliott

        Quick TLDR. incase my word salad OP doesn’t make sense.

        If /images/dev is on a different mount point than /images, captured images will remain in their temporary location in /images/dev and a directory for the newly captured images with not be created in /images

        Thanks Tom,

        Everything you mentioned is what I did on my existing FOG server. And the same problem occurs on a freshly spun VM I set up just to make sure I didn’t miss any files or permissions from the original /images/dev location

        The images in the OP are from the new VM. All mount points were specified at the beginning of the Ubuntu install - fstab created by the Ubuntu installer (see code below). The files and permissions on the /images/dev mount were created by FOG.

        I still have it up if any logs could be of help.

        # /etc/fstab: static file system information.
        #
        # Use 'blkid' to print the universally unique identifier for a
        # device; this may be used with UUID= as a more robust way to name devices
        # that works even if disks are added and removed. See fstab(5).
        #
        # <file system> <mount point>   <type>  <options>       <dump>  <pass>
        # / was on /dev/vda2 during curtin installation
        /dev/disk/by-uuid/078f54eb-340b-41fc-ab34-1494cd59e4e4 / ext4 defaults 0 1
        # /boot/efi was on /dev/vda1 during curtin installation
        /dev/disk/by-uuid/DD22-44A9 /boot/efi vfat defaults 0 1
        # /images was on /dev/vdb1 during curtin installation
        /dev/disk/by-uuid/d61ab2ae-b79a-4b07-bfc5-4678ab0902f4 /images ext4 defaults 0 1
        # /images/dev was on /dev/vdc1 during curtin installation
        /dev/disk/by-uuid/3d7874cb-8c59-4e6d-8735-fb8361994590 /images/dev ext4 defaults 0 1
        /swap.img       none    swap    sw      0       0
        
        
        A 2 Replies Last reply Sep 12, 2024, 12:51 PM Reply Quote 0
        • A
          AUTH IT Center @Fog_Newb
          last edited by Sep 12, 2024, 12:51 PM

          @Fog_Newb

          I believe the issue relates to /etc/fstab parameters because the mounts must reside in order.

          Check the comment in reddit and try the proper parameters.

          F 1 Reply Last reply Sep 12, 2024, 1:54 PM Reply Quote 0
          • F
            Fog_Newb @AUTH IT Center
            last edited by Fog_Newb Sep 12, 2024, 8:11 AM Sep 12, 2024, 1:54 PM

            @AUTH-IT-Center

            Thanks -The fstab paste is below. The parameters are correct as far as I can tell and they were added by Ubuntu during the install. And it appears both /images and /images/dev are mounting correctly and accessible since FOG was able to capture to /images/dev and create a file in /images. I was also able to manually create files and directories in each mount location without any problem. If you see something out of place feel free to let me know.

            On my other FOG server, where there is only the 2nd drive, and it’s is a mount point for /images, it uses the same fstab parameters (albiet different UUID) and it has been working fine.

            1 Reply Last reply Reply Quote 0
            • A
              AUTH IT Center @Fog_Newb
              last edited by Sep 12, 2024, 3:06 PM

              @Fog_Newb said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

              /dev/disk/by-uuid/d61ab2ae-b79a-4b07-bfc5-4678ab0902f4 /images ext4 defaults 0 1
              /dev/disk/by-uuid/3d7874cb-8c59-4e6d-8735-fb8361994590 /images/dev ext4 defaults 0 1

              based on the reddit post you should try

              /dev/disk/by-uuid/d61ab2ae-b79a-4b07-bfc5-4678ab0902f4 /images ext4 defaults 0 1
              /dev/disk/by-uuid/3d7874cb-8c59-4e6d-8735-fb8361994590 /images/dev auto bind,x-systemd.requires=/images 0 0
              
              F 1 Reply Last reply Sep 12, 2024, 3:56 PM Reply Quote 0
              • F
                Fog_Newb @Tom Elliott
                last edited by Fog_Newb Sep 12, 2024, 9:18 AM Sep 12, 2024, 3:14 PM

                @Tom-Elliott said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                1. Change the ownership back to chown -R fogproject:fogproject /images; chmod 775 -R /images

                This should allow your system to work without issues.

                I ran each command separately, this time on the new FOG server. The one where FOG created the permissions and files during the install. However I’m still experiencing the same issue

                Your FSTAB needs to have the mount set correctly too so that dev gets mounted appropriately on subsequent boots which I presume is already happening?

                Yes, in both cases - on the original FOG server and new test FOG server fstab was correct, the mount points mounted fine at boot and I was able to manually write directories and files to each mount (/images /images/dev). I’ve pasted the contents of the fstab in an earlier comment

                So strange. As long as both /images and /images/dev are on the same mount/drive everything goes off without a hitch

                1 Reply Last reply Reply Quote 0
                • F
                  Fog_Newb @AUTH IT Center
                  last edited by Fog_Newb Sep 12, 2024, 10:44 AM Sep 12, 2024, 3:56 PM

                  @AUTH-IT-Center

                  Unfortunately, making those change to fstab, saving then rebooting resulted in errors during boot when it attempted to mount vdc (/images/dev) and ultimately led to “emergency mode”.

                  fstab

                  error during boot

                  Emergency mode

                  No worries though, in emergency mode I was able to edit fstab

                  I know when the box is booted, mount points are correct vdb is mounted as /images, and vdc is mounted as /images/dev

                  To test this I moved a 20 something GB directory from /images/dev to /images. The size of the VM disk image for vdb grew 20 something GB. The directory I coped from, /images/dev had to have been on vdc since the OS drive (vda) doesn’t have that much space. And it couldn’t have been on vdb since it would have moved instantly and the VM disk image for vdb wouldn’t have increased in size.

                  edit - just for the heck of it, I removed /images and /images/dev from fstab and once booted, I manually mounted /images (vdb) once I confirmed that was mounted and working I manually mounted /images/dev (vdc) - still the same problem with FOG

                  A T 2 Replies Last reply Sep 12, 2024, 4:45 PM Reply Quote 0
                  • A
                    AUTH IT Center @Fog_Newb
                    last edited by Sep 12, 2024, 4:45 PM

                    @Fog_Newb

                    try the below

                    /dev/disk/by-uuid/d61ab2ae-b79a-4b07-bfc5-4678ab0902f4 /images ext4 defaults 0 1
                    /dev/disk/by-uuid/3d7874cb-8c59-4e6d-8735-fb8361994590 /imagesdev ext4 defaults 0 1
                    /imagesdev  /images/dev   none    defaults,bind   0   0
                    

                    Sources:

                    • https://man.archlinux.org/man/systemd.mount.5
                    • https://wiki.archlinux.org/title/Fstab#Bind_mount
                    F 1 Reply Last reply Sep 12, 2024, 7:40 PM Reply Quote 1
                    • T
                      Tom Elliott @Fog_Newb
                      last edited by Sep 12, 2024, 6:33 PM

                      @Fog_Newb You seem to have the same uuid for both vdb and vdc

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                      F 2 Replies Last reply Sep 12, 2024, 6:52 PM Reply Quote 0
                      • F
                        Fog_Newb @Tom Elliott
                        last edited by Fog_Newb Sep 12, 2024, 1:07 PM Sep 12, 2024, 6:52 PM

                        @Tom-Elliott

                        If you’re looking at the Storage Configuration screen cap, those numbers are something else and are slightly different. The end numbers - 01, 02, 03

                        The UUID’s for vda1, vda2, vdb, and vdc are different

                        See

                        1 Reply Last reply Reply Quote 0
                        • F
                          Fog_Newb @AUTH IT Center
                          last edited by Fog_Newb Sep 12, 2024, 1:42 PM Sep 12, 2024, 7:40 PM

                          @AUTH-IT-Center said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                          @Fog_Newb

                          try the below

                          /dev/disk/by-uuid/d61ab2ae-b79a-4b07-bfc5-4678ab0902f4 /images ext4 defaults 0 1
                          /dev/disk/by-uuid/3d7874cb-8c59-4e6d-8735-fb8361994590 /imagesdev ext4 defaults 0 1
                          /imagesdev  /images/dev   none    defaults,bind   0   0
                          

                          Sources:

                          • https://man.archlinux.org/man/systemd.mount.5
                          • https://wiki.archlinux.org/title/Fstab#Bind_mount

                          Same results only - the changes you suggested making to fstab created /imagesdev along with /images/dev on vcd. Both are bound together.

                          Create a file in /imagesdev it is seen in /images/dev and vis-a vers-a.

                          here

                          see

                          1 Reply Last reply Reply Quote 0
                          • F
                            Fog_Newb @Tom Elliott
                            last edited by Fog_Newb Sep 12, 2024, 2:55 PM Sep 12, 2024, 8:31 PM

                            @Tom-Elliott

                            I think this thread has gotten a bit off track, partly due to my explanation about trying to modify an existing FOG server.

                            So, if I could - Let’s forget about the old FOG server - a re-cap - the new ‘test’ FOG server.

                            Fresh Ubuntu 24.04.1 install with 3 drives. During the install process, I created mount points for /images (vdb), and /images/dev (vdc)

                            each drive has a unique UUID

                            here

                            mount points correct

                            here

                            fstab created by the Ubuntu installer

                            here

                            I installed FOG 1.5.10.1612 without any issues—no errors at all. During the installation, the FOG installer successfully wrote files and set permissions on both mount points (/images/dev (vdc) and /images (vdb)). And all the files that should be in each directory/mount are present when compared to a functioning FOG server.

                            When capturing an image, it gets stuck in the temporary location /images/dev/[randomnumbers] and never moves to the /images directory.

                            executing both

                            sudo chown -R fogproject:fogproject /images
                            

                            and

                            sudo chmod 775 -R /images
                            

                            then re-testing/capturing - no change

                            Manually moving a directory (which couldn’t possibly fit on vda) from /images/dev (vdc) to /images (vdb) caused the virtual disk size on vdb to increase. IMHO, it seems clear the directory was initially stored on vdc. Am I wrong in assuming this confirms that the mounts are correctly configured and functioning?

                            Anyway, it’s not a big deal. I just wanted to make sure I’ve at least clarified everything

                            Thanks

                            T J 2 Replies Last reply Sep 13, 2024, 9:56 AM Reply Quote 0
                            • T
                              Tom Elliott @Fog_Newb
                              last edited by Tom Elliott Sep 13, 2024, 3:57 AM Sep 13, 2024, 9:56 AM

                              @Fog_Newb

                              Manually moving a directory (which couldn’t possibly fit on vda) from /images/dev (vdc) to /images (vdb) caused the virtual disk size on vdb to increase. IMHO, it seems clear the directory was initially stored on vdc. Am I wrong in assuming this confirms that the mounts are correctly configured and functioning?

                              Yes. vdb is /images, so moving a file from /images/dev (vdc) to /images (vdb) should cause the disk to increase on vdb? The sice on vdc wouldn’t decrease but the amount of usable space off vdc should be increased by the amount of the file that was moved to now vdb?

                              That’s what I’m understanding here. I don’t know if this means there is an issue or if this is working as intended, but it seems like it is?

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              F 1 Reply Last reply Sep 13, 2024, 6:57 PM Reply Quote 0
                              • F
                                Fog_Newb @Tom Elliott
                                last edited by Fog_Newb Sep 14, 2024, 2:30 AM Sep 13, 2024, 6:57 PM

                                @Tom-Elliott said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                                @Fog_Newb

                                Manually moving a directory (which couldn’t possibly fit on vda) from /images/dev (vdc) to /images (vdb) caused the virtual disk size on vdb to increase. IMHO, it seems clear the directory was initially stored on vdc. Am I wrong in assuming this confirms that the mounts are correctly configured and functioning?

                                Yes. vdb is /images, so moving a file from /images/dev (vdc) to /images (vdb) should cause the disk to increase on vdb? The sice on vdc wouldn’t decrease but the amount of usable space off vdc should be increased by the amount of the file that was moved to now vdb?

                                That’s what I’m understanding here. I don’t know if this means there is an issue or if this is working as intended, but it seems like it is?

                                That is what I was thinking - I was trying to establish that the mount points are set up correctly and that it’s not an issue with fstab.

                                It only takes a few minutes or so to spin one of these VM’s. I spun up another VM, same parameters and mount points. The only things added other than the ssh server during the Ubuntu install were - the Ubuntu updates, dnsmasq, pip, tzlocal, tz… and of course the latest FOG dev version 1.5.10.1614. All cloud services, disabled

                                I exported it to OVF right before registering a host and capturing an image… which exhibited the same problems.

                                If you or anyone else wants to check it out

                                https://drive.google.com/file/d/1GPJH0ZuJFZaTqmGvIe1EM2HBCVlde4FI/view?usp=sharing

                                username: test
                                password: 123456

                                All 3 VM drive images are included with the OVF. A 2.8 GB zip since the drives have barely anything on them.

                                Fog web interface - user and password are defaults

                                It takes a ridiculously long time to import the OVF into VMWare Workstation. But it will import

                                If you want it quick and dirty create a new UEFI based Linux VM and chose to use the existing vmdk drives contained in the zip.

                                A copy of the conf file used by dnsmasq is in the home directory

                                1 Reply Last reply Reply Quote 0
                                • J
                                  JJ Fullmer Testers @Fog_Newb
                                  last edited by Sep 13, 2024, 8:13 PM

                                  @Fog_Newb
                                  I think this might be a limitation on the NFS side and the purposely limited permissions of the host when booted into FOS (Fog Operating System) for a task.
                                  I know I once tried to do a separate disk within the /images NFS mount to have drivers I inject on a separate disk. I ran into similar issues, in the end I just made a bigger disks for /images.

                                  Also, just an FYI, it’s not random numbers, it’s the mac address of the host being captured.

                                  So anyway it sounds like.

                                  • FOS is mounting both nfs shares correctly.
                                  • FOS is writing to the /images/dev share
                                  • FOS can’t move the files from one share to another when the shares are on different disks.

                                  There could be an issue with the nfs exports

                                  If you have another linux machine you could try mounting /images and /images/dev via nfs and see if you can copy in those use cases

                                  Have you tried the FogApi powershell module? It's pretty cool IMHO
                                  https://github.com/darksidemilk/FogApi
                                  https://fogapi.readthedocs.io/en/latest/
                                  https://www.powershellgallery.com/packages/FogApi
                                  https://forums.fogproject.org/topic/12026/powershell-api-module

                                  J F 2 Replies Last reply Sep 13, 2024, 8:19 PM Reply Quote 0
                                  • J
                                    JJ Fullmer Testers @JJ Fullmer
                                    last edited by Sep 13, 2024, 8:19 PM

                                    @JJ-Fullmer Maybe if we mounted the shares in fos differently so that they are in different mount paths and not the same on the client/host then this might work, but that’s a bigger change than it sounds like I think.

                                    Have you tried the FogApi powershell module? It's pretty cool IMHO
                                    https://github.com/darksidemilk/FogApi
                                    https://fogapi.readthedocs.io/en/latest/
                                    https://www.powershellgallery.com/packages/FogApi
                                    https://forums.fogproject.org/topic/12026/powershell-api-module

                                    1 Reply Last reply Reply Quote 0
                                    • F
                                      Fog_Newb @JJ Fullmer
                                      last edited by Fog_Newb Sep 13, 2024, 4:37 PM Sep 13, 2024, 9:49 PM

                                      @JJ-Fullmer said in FOG has issues if the temp image location is on another drive. FOG 1.5.10.1612 Ubuntu Server24.04.1 LTS:

                                      @Fog_Newb

                                      • FOS is writing to the /images/dev share
                                      • FOS can’t move the files from one share to another when the shares are on different disks.

                                      exactly

                                      FOS is mounting both nfs shares correctly.

                                      In my case, there’s no NFS shares involved, unless I’m misunderstanding.

                                      /images and /images/dev are ‘mount points’ of the 2nd and 3rd local hard-drives on the VM. Maybe I’ve been using the term ‘mount points’ incorrectly

                                      Also, just an FYI, it’s not random numbers, it’s the mac address of the host being captured.

                                      When I mentioned random numbers I was talking about the numbers listed during the ubuntu install for the disk drive device numbers.

                                      They weren’t the UUID’s of the drives or partitions and looked too long to be MAC addresses

                                      these

                                      J 1 Reply Last reply Sep 14, 2024, 9:47 PM Reply Quote 0
                                      • J
                                        JJ Fullmer Testers @Fog_Newb
                                        last edited by Sep 14, 2024, 9:47 PM

                                        @Fog_Newb Oh those are random, I thought you were talking about the /images/dev/{mac} that is used when capturing.

                                        So /images and /images/dev are 2 nfs shares on your FOG server. You’ll see this defined in /etc/exports

                                        If you aren’t able to manually move the file on the FOG server from /images/dev to /images then there’s some other weird permissions issue on the server.

                                        When you are capturing an image though, the host boots to FOS, and part of the capture process is mounting the 2 NFS shares from the FOG server, /images and /images/dev. The capture process has partclone writing the captured image files to /images/dev/{mac}/*
                                        Once complete, it moves them over to /images (technically it deletes the old version if any in /images and then moves the new version into its place).
                                        I don’t recall if in the stable/dev branch if we are using sftp yet, so it may be doing ftp (1.6 for sure uses sftp). Point being it does technically remote into the server to do the move operation because the /images nfs share is a read only share.
                                        When you have nfs shares mounted in the same folder that are actually on different disks, I’ve seen weirdness like you’re experiencing is what I’m saying.

                                        So all of that is to say

                                        • Try moving/copying a file from /images/dev to /images on the server directly, confirm that works.
                                        • Try booting into a debug capture session, capture the image (use the command fog and hit enter to step through. Capture any errors.

                                        Technically if you’ve captured the image, you can manually move it on the server to the /images/{imageName} folder and you should be able to deploy as a workaround while we figure this out.

                                        You could also try installing 1.6-beta.

                                        Have you tried the FogApi powershell module? It's pretty cool IMHO
                                        https://github.com/darksidemilk/FogApi
                                        https://fogapi.readthedocs.io/en/latest/
                                        https://www.powershellgallery.com/packages/FogApi
                                        https://forums.fogproject.org/topic/12026/powershell-api-module

                                        F 2 Replies Last reply Sep 15, 2024, 12:46 AM Reply Quote 0
                                        • F
                                          Fog_Newb @JJ Fullmer
                                          last edited by Fog_Newb Sep 14, 2024, 10:48 PM Sep 15, 2024, 12:46 AM

                                          This post is deleted!
                                          1 Reply Last reply Reply Quote 0
                                          • 1
                                          • 2
                                          • 1 / 2
                                          1 / 2
                                          • First post
                                            7/35
                                            Last post

                                          150

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project