Ubuntu Help - Boot Problem #N00b

[Sebastian Roth]

@RogerBrownTDL Ok, your /images directory is on the root partition and just filled it up. Not too much trouble - let’s hope it didn’t break the database.

Run ls -al /images/dev and post output here. Pretty sure there will be old broken image captures that we can delete.

[RogerBrownTDL]

@Sebastian-Roth said in Ubuntu Help - Boot Problem #N00b:

ls -al /images/dev

root@tie-fogdeploy-01:~# ls -al /images/dev
total 16
drwxrwxrwx 4 fogproject root 4096 Mar  3 15:43 .
drwxrwxrwx 7 fogproject root 4096 Mar  1 19:31 ..
drwxrwxrwx 2 root       root 4096 Mar  3 15:45 c8d9d2d4c8ac
-rwxrwxrwx 1 fogproject root    0 Sep  7 20:42 .mntcheck
drwxrwxrwx 2 fogproject root 4096 Sep  7 20:42 postinitscripts
root@tie-fogdeploy-01:~#

[Sebastian Roth]

@RogerBrownTDL Ok, there is really only one. Probably the one that failed to upload because it filled the disk. I suggest you remove that because with a full disk that is not going to be a valid image in any case.

rm -rf /images/dev/c8d9d2d4c8ac

Make sure you don’t mess with this command (e.g. use spaces in other places or *) because it can do a lot of harm if used the wrong way. Just wanted to get out this beforehand as you told us you are a Linux beginner.

Now run df -h to check the disk space available and reboot the whole VM.

[RogerBrownTDL]

@Sebastian-Roth said in Ubuntu Help - Boot Problem #N00b:

df -h

Okay, removed the offending one using rm -rf /images/dev/c8d9d2d4c8ac df -h shows:

root@tie-fogdeploy-01:~# df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                               5.7G     0  5.7G   0% /dev
tmpfs                              1.2G  112M  1.1G  10% /run
/dev/mapper/ubuntu--vg-ubuntu--lv   98G   86G  7.8G  92% /
tmpfs                              5.8G     0  5.8G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                              5.8G     0  5.8G   0% /sys/fs/cgroup
/dev/loop0                          92M   92M     0 100% /snap/lxd/23991
/dev/loop1                          64M   64M     0 100% /snap/core20/1822
/dev/loop2                          92M   92M     0 100% /snap/lxd/24061
/dev/loop3                          50M   50M     0 100% /snap/snapd/17950
/dev/loop5                          50M   50M     0 100% /snap/snapd/18357
/dev/loop4                          64M   64M     0 100% /snap/core20/1778
/dev/sda2                          2.0G  207M  1.6G  12% /boot
root@tie-fogdeploy-01:~#

Rebooted VM. df -h shows

root@tie-fogdeploy-01:~# df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                               5.7G     0  5.7G   0% /dev
tmpfs                              1.2G  1.1M  1.2G   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv   98G   86G  7.8G  92% /
tmpfs                              5.8G     0  5.8G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                              5.8G     0  5.8G   0% /sys/fs/cgroup
/dev/loop1                          64M   64M     0 100% /snap/core20/1778
/dev/loop0                          64M   64M     0 100% /snap/core20/1822
/dev/loop2                          50M   50M     0 100% /snap/snapd/18357
/dev/loop5                          92M   92M     0 100% /snap/lxd/24061
/dev/loop3                          50M   50M     0 100% /snap/snapd/17950
/dev/loop4                          92M   92M     0 100% /snap/lxd/23991
/dev/sda2                          2.0G  207M  1.6G  12% /boot
tmpfs                              1.2G     0  1.2G   0% /run/user/0
root@tie-fogdeploy-01:~#

It’s a 500gb VHDX though, surely this shouldnt be full?

[Sebastian Roth]

@RogerBrownTDL On the long run you need to look into purging some of the images and/or extending the disk space.

The other outputs you posted show that the root partition (LVM) really is only 100 GB in size while the whole VMDK container is 500 GB.

Extending the space is a bit more advanced but should be possible for you to achieve as well because things seem to be prepared already. After the reboot of the VM run the following commands and post output here:

pvs
vgs
lvs

[RogerBrownTDL]

@Sebastian-Roth said in Ubuntu Help - Boot Problem #N00b:

lvs
Output is as follows:

root@tie-fogdeploy-01:~# pvs
  PV         VG        Fmt  Attr PSize    PFree
  /dev/sda3  ubuntu-vg lvm2 a--  <498.00g <398.00g

root@tie-fogdeploy-01:~# lvs
  LV        VG        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  ubuntu-lv ubuntu-vg -wi-ao---- 100.00g                                        

root@tie-fogdeploy-01:~# vgs
  VG        #PV #LV #SN Attr   VSize    VFree
  ubuntu-vg   1   1   0 wz--n- <498.00g <398.00g
root@tie-fogdeploy-01:~#

[lukebarone]

@RogerBrownTDL

lvextend -L +398g ubuntu-lv --resizefs

You may need to reboot for it to take effect. This will use all the remaining LVM volume space for your logical volume. You can’t go smaller after the fact, so be sure you want to use it all first.

[RogerBrownTDL]

@lukebarone said in Ubuntu Help - Boot Problem #N00b:

lvextend -L +398g ubuntu-lv --resizefs

to confirm, this will mean I can actually use the box to it’s full 500gb drive allocation?

[lukebarone]

@RogerBrownTDL Yes.

lvextend - Increase the volume size
-L - Specify the size to increase it by
ubuntu-lv - The name of your logical volume (as reported)
--resizefs - Resize the EXT{2|3|4} file system

[RogerBrownTDL]

@lukebarone

In which case based on the output here:
root@tie-fogdeploy-01:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda3 ubuntu-vg lvm2 a-- <498.00g <398.00g
root@tie-fogdeploy-01:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
ubuntu-lv ubuntu-vg -wi-ao---- 100.00g
root@tie-fogdeploy-01:~# vgs
VG #PV #LV #SN Attr VSize VFree
ubuntu-vg 1 1 0 wz–n- <498.00g <398.00g
root@tie-fogdeploy-01:~# lvextend -L +398g ubuntu-lv --resizefs
Please specify a logical volume path.
Run `lvextend --help’ for more information.
root@tie-fogdeploy-01:~#

Would it be lvextend -L +398g ubuntu-vg --resizefs ?

[lukebarone]

@RogerBrownTDL Try this instead:

lvextend -L +398g ubuntu-vg/ubuntu-lv --resizefs

[RogerBrownTDL]

@lukebarone response: root@tie-fogdeploy-01:~# lvextend -L +398g ubuntu-vg/ubuntu-lv --resizefs
Insufficient free space: 101888 extents needed, but only 101887 available
root@tie-fogdeploy-01:~#

[RogerBrownTDL]

@lukebarone thoughts?

[Sebastian Roth]

@lukebarone I am wondering if it’s wise to create another volume instead of extending the existing one. This way the images could be put on the separate volume/partition and would not be able to fill the root partition. But that’s just me trying to prevent this from happening again.

The downside of my proposal is that the existing 100 GB root partition can’t be used by images and therefore are kind of wasted because the FOG server itself will never use 100 GB disk space with the images on a separate partition.

So maybe it’s actually wise to just extend as proposed and @RogerBrownTDL needs to keep an eye on the disk space from now on.

So if you wanna go this path, do:

lvextend --extents +100%FREE ubuntu-vg/ubuntu-lv --resizefs

[RogerBrownTDL]

@Sebastian-Roth So obviously as a Linux noob, I know very little… I’m purely (I know, insert puke emoji here) a Windows Sysadmin. If I understand it correctly the way i’ve got it setup is like having a single big C:\ and all i’m wanting to do is get the entire fog server to see it all rather than just a partition of it… If I follow that command
“lvextend --extents +100%FREE ubuntu-vg/ubuntu-lv --resizefs”
Will it do that so that the full 500gb would be visible and the images stored remain intact?

[Sebastian Roth]

@RogerBrownTDL said in Ubuntu Help - Boot Problem #N00b:

Will it do that so that the full 500gb would be visible and the images stored remain intact?

Yes! Although I am not a 100% sure the filesystem resize (to full size) will work while the system is running. Should work but I can’t promise you it will. As well just for safety reasons I always suggest for people to take a backup copy before doing these kind of operations on a production server. Should be really easy taking a snapshot in Hyper-V before going ahead.

The other option I was talking about would be like adding a D:\ drive and moving all the images to that new partition. As I said it’s wise on the one hand so images can’t fill up your important C:\ drive (with the database on it) but on the other side you waste free space on C:…

[RogerBrownTDL]

@Sebastian-Roth further info print i’ve got if this makes more sense:
root@tie-fogdeploy-01:~# sudo parted /dev/sda unit MiB print
Model: Msft Virtual Disk (scsi)
Disk /dev/sda: 512000MiB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 1.00MiB 2.00MiB 1.00MiB bios_grub
2 2.00MiB 2050MiB 2048MiB ext4
3 2050MiB 511999MiB 509949MiB

If the system isnt running, surely I wouldnt be able to do anything, so it needs to be online?

[Sebastian Roth]

@RogerBrownTDL said in Ubuntu Help - Boot Problem #N00b:

If the system isnt running, surely I wouldnt be able to do anything, so it needs to be online?

Well, you can boot into a Live Linux ISO (Ubuntu or many others) to get to a command shell and do such things with the OS on disk not running. But I think you are fine doing it live after taking a snapshot backup.

[RogerBrownTDL]

@Sebastian-Roth Thanks my friend, snapshot taken in hyper-v and command run… Stupid Q now. How do I check if it’s worked

[Sebastian Roth]

@RogerBrownTDL Reboot and take a look at the disk space with df -h.