Posts made by george1421

@brakcounty Lets find out a bit more about your setup run these commands and capture the output.

ip addr show
ip route show

@astrugatch It never made it to the repo. It was a skunkworks project I was working on. I just went and grabbed the centos version of cctk https://downloads.dell.com/FOLDER05519675M/1/command-configure-4.2.0-553.el7.x86_64.tar.gz to see if I could rebuild the test environment.

@astrugatch said in Secureboot issues:

Isn’t that what command and configure is for (if you’re a dell shop at least)?

Yes, we use command and configure (CCTK) to set default bios settings. I started looking into building cctk into FOS Linux so those bios settings could be made via a post install script. I think that project fell off the table a while ago.

If you are seeing bzImage being transferred you can remove iPXE out of the problem. The default you should have for uefi is ipxe.efi.

Now you should focus on the FOS Linux engine (bzImage). What version is the FOS Linux kernel? I know I have a Dell 7400 2 in 1 with a Dell usb c dongle (as well as a WD15/WD19 docks) that boot into FOS Linux and pick up a dhcp address.

I find it strange that it finds the network adapter, and sees the link come up but can’t communicate over that link. Lets start with what version of kernel are you using? The next part is to schedule a new capture/deploy task but tick the debug check box before you schedule the task. Then pxe boot the target computer that will but the target computer in debug mode so we can key in some additional commands.

@Peter365 The only comment I can make is watch the permissions on the /images directory now. If you run into upload issues issue this command (note is a bit loose with the security, but its not too big of a risk) chmod -R 777 /images

@rocksteve69 This may be too much to ask, but if you swapped the WD drive for the Samsung drive in the same model laptop, does the problem move with the drive? That would then point directly to the Samsung drive.

@Quazz do you think the updated version of partclone would solve this issue ?

@f-an Did you install FOG using the git method or the tarball method? Because if you use the git method you can switch to the working branch and install fog 1.5.7.2 which addresses some issues since 1.5.7 GA was released. I’m not saying that solves anything also @Quazz was working on updated inits (FOS Linux virtual hard drive) with the updated version of partclone. That updated version I seem to remember fixed a slow nvme issue. Let me see if I can find the link he posted.

Edit: Read the entire thread so you can see how to install the inits without breaking your fog deployment: https://forums.fogproject.org/topic/13620/very-slow-cloning-speed-on-specific-model/12 Upgrading to 1.5.7.2 is not required to test these inits.

@f-an said in Deploying images from virtual FOG-server to VM's on the same host takes forever:

The FOG server is right now running with 48GB RAM and 8 vCPU

Just for reference 4GB ram and 2 vCPU is all that is required, a bit more if you have many fog client checking in.

With 1.5.7 your FOS Linux kernel and inits are pretty new so that probably isn’t the issue. So if you introduce a physical machine to capture or deploy to what are your transfer rates? The physical machine doesn’t need to boot for this test, only to deploy the image to get the speed.

As for your VM is it a type 1 or type 2 (not the right words) hyper-v vm? Is the vm in uefi or bios mode? If we find out its the VM (based on the results of the physical machine deploy) we can run some tests to see if its the target machine disk or network that’s causing the issue.

Also just for reference what OS is the hyper-v host running on?

@paat So just to be clear, you have a Dell branded usbc cable?

I have a dell branded usb c ethernet adapter on a 7400 and I can pxe boot into the fog compatibility menu (I don’t want to image my workstation at the moment) and ping devices. So I know the network adapter works. Also when we image we typically use the WD15 dock (the WD19 docks are flaky some times with pxe booting). That way we charge the laptop as well as image at the same time.

This is the dell usbc dongle we are using: https://www.dell.com/en-us/shop/dell-adapter-usb-c-to-ethernet-pxe-boot/apd/470-abnd/pc-accessories

Also what version of FOG are you using as well as what FOS Linux kernel version do you have?

What version of FOG are you using? I remember a while ago the FOS Linux engine would stall out during the partclone process then take off again.

As for the speed, I’ve heard about hyper-v being slow before, but not by first hand experience.

In can say in our environment we use vSphere and capture from a vsphere vm to a vshpere FOG server is about 6GB/m and deployment is in the area of 13GB/min. I can push out a 25GB (fat) golden image in about 2 minutes. I’m only saying this to show what is possible.

So what would be interesting to know in your setup is the hyper-v FOG server slow or is it the target computer (vm)? I can say during a fog image deployment cycle the target computer does almost all of the work in imaging, the FOG server only manages the process. In my home lab I have a fog server running on a raspberry PI3 and I get 4.5 to 5.0GB/min transfer rates so the point is the fog server isn’t a major factor in imaging speed its the target computer.

@Wolfbane8653 said in rcu_sched self detected stall on CPU when Deploying:

tsc=unstable – works with Kernel 4.19.64
clocksource=hpet – works with Kernel 4.19.64

Great on fixing it with the timing source. That is the solution. As for my noacpi I figured that would happen because I also removed the acpi boot device drivers too. It was a risk, but the right answer is with the kernel parameters with the stock kernel. Well done!

@fry_p ok now I think is time for some hokus-pokus magic without breaking much.

From your production server export the image definitions and import them into your dev environment.

From your production server export and then import the host definitions into the test setup. This will create a clone of your production environment without the image files.

Temporarily move the production FOG server’s IP address (i.e. just swing it out of the way) and move the dev FOG client server’s IP address in place of the production fog server (understand this may impact pxe booting if you have them boot through FOG). The goal here is to apply real world loads to the FOG client server. We should watch the CPU usage on the FOG client server (we may need to increase the number of php-fpm workers to satisfy all of the requests) as well as the mariadb service on the dev imaging server. We are looking for high cpu usages on both systems. This might tell us if we need to spin off the sql server onto its own server. We should also gauge the web ui performance on the imaging server, because that was a pain point when you had the check in times set to the default.

I’m not sure I’d leave the check in time at 30 seconds in a production environment, but it would be interesting to see the load.

When you are done testing then swing the dev FOG client server out of the way and put the production fog server back in place.

@Peter365 ok here is how we will proceed.

1. Move everything from /images to /Images
   mv /image/* /Images
2. Do an ls -la /images and ensure the directory is blank.
3. Unmount /Images
   umount /Images
4. Now edit your fstab and change the line where the mountpoint is /Images (the line with centos-Images in it) to the /images (note the lower case i) What we are doing is mounting the 931GiB partition onto /images instead of /Images. I know it sounds a bit confusing, but we are putting the disks into the configuration standard for FOG. This will save you headache in the future. Now save the fstab file.
5. From the linux command prompt key in mount -a
6. Now see what is in /images ls -la /images It now should contain everything it had originally before we moved it.
7. If you issue the lsblk command again you will now see that centos-Images is mounted to /images.
8. Done and done

Here is a test kernel with no acpi functions supported: https://drive.google.com/open?id=1siERUC9h8MfQIXbqrQShKOHc55h5xK3q

Download it as bzImageNoACPI and move it to /var/www/html/fog/service/ipxe directory on your fog server. Then go into the host definition for this specific host with the rcu_sched error and enter bzImageNoACPI (watch the case) into the kernel field and save the host configuration. Then pxe boot the target computer into imaging to see if we can get past the cpu stall.

@Wolfbane8653 I’m currently building a FOS Linux kernel without acpi support to see if we can get past the rcu_sched issue. I can say tracking down this type of issue does take time because its hardware/model specific. If you can get one of the kernel parameters that Quazz mentioned to work is the preferable route. I’ll post a link to the noacpi generated kernel when its done building.

@Sebastian-Roth said in Secureboot issues:

@bgbgbgbg There is money und time involved to make this work. Check out the information on the iPXE website:
https://ipxe.org/appnote/etoken

The best part of that article I found was the footnote “Microsoft is special”

Not only does iPXE need to be signed, but the FOG Project would have to go through the same process MS boot image signing for each FOS Linux kernel that was released. This would most assuredly impact the “free-ness” and responsiveness to changing hardware demands for FOG imaging.

@oliver just so you know the linux user fog or fogproject should never be used for system administration. These accounts are owned by the fog program and will reset their passwords each time you install FOG.

We are glad you have it working now.

@rocksteve69 other than the export errors are you experiencing any issue? The export errors are understandable because of the dot in the parameter name.

@Peter365 Do you need both storage locations?

What does the output of ls -la /images and ls -la /Images look like?

@Deimos When you installed fog, did/do you have the environment variables setup so that the fog installer can download the needed OS modules from your linux distro’s repo repository?