It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.

Basavarajnc

Dear Sir,

This is Basavaraj From India ( I am the beginner stage of learning This Fog Server), Heartily Request you to help me in this Regard as early as possible. When Deploying Image to Client Machine, It takes more than 5 Minutes for Erasing Current MBR/GPT Tables. That moment client machine looks like ideal or stuck…
After Some Time, It completes its Work Successfully.

My Question is : Why it is taking longer time for this section ?

Could you Please Help me

Heartily Thank You,
Basavaraj From India,

Tywyn

Hi,

https://forums.fogproject.org/topic/11989/erasing-current-mbr-gpt-tables-taking-about-5-10-minutes

Tom Elliot describes there, that there are some new NTFS-Progs in place, that cause the delay.

Cheers, Rainer

Basavarajnc

@tywyn Dear Sir,

Actually, I have read through your link but understood little. mine is ubuntu 16.04.3 Desktop and Fog Version 1.5.4 (New Version). Where Exactly need to configure OR need to install Any Extra Packages.

Sir Little Bit I am confusion… could you tell me Where things need to be configured and Where needs to be installed any Extra Packages…

Could you kindly explain it in Detailed Way?

Heartily Thank you of Your Reply.

Basavaraj From India,

george1421

@basavarajnc From the FOG Configuration pages, downgrade your FOG Kernel (not FOG server ubuntu linux kernel) to version 4.15.2. This version images GPT disks without the long delay. The FOG kernel developers are looking into the reason for the slow erasure. An answer to this problem is not easy to find at the moment.

Basavarajnc

@george1421 Dear Sir (george1421),

Heartily Thank You For Your Kindly Reply Without making Delay. Sir, Actually I am Beginner Now. So I am Facing Some Difficulties… I will check out Now Where This Section Comes Up —>>>****“From the FOG Configuration pages, downgrade your FOG Kernel (not FOG server ubuntu linux kernel) to version 4.15.2”.

I will reply to you Sir Once It has been completed.

Basavarajnc

@Basavarajnc Dear Sir (george1421),

Where Exactly Comes Downgrade FOG Kernel Option to Version 4.15.2 … Because I looked into it and went in FOG Configuration Section Pages… Here a lot of options are there including Kernel Update… Herewith I enclosed the Screen-Short-Picture.

Seeing this Picture, You will come to know… **

My Sincere and Honest thanks to you and Every One For helping to those people Who are trouble in Fog Server Settings… Extra Extra…

I am really Grateful for This…!

Warm Regards,
Basavaraj From India,

george1421

@Basavarajnc In the FOG Configuration page. Along the left side there is a menu called “kernel update”. You can see this in the picture you posted. When you select that menu there will be a list of kernels, scroll down to 4.15.2, expand both the 32 bit and 64 bit kernels and hit the download button along the right side

Basavarajnc

@george1421 Dear Sir,

As per your guidance, I have done it to 4.15.2 64 bit Kernel. Wholeheartedly Thank you for your Support.

But One thing… I observed that… capturing Image process is Slow (Approx… 800 MB to 830 MB/Minute) Before It was 3GB to 4GB/Minute… It is Because of Changing the Kernel Version??

I also made some change while taking image. The change is that Compression feature I kept 20. In Previous Image, I did nothing(Default Compression -> 6)
is this reason or anything Else ?

Please Kindly Explain it in Detailed Manner…

Thanking you Sir,

Warm Regards,

Basavaraj From India,

Tom Elliott

@Basavarajnc higher compression rating = slower capture. This also means smaller disk space used and typically translates to faster deploy as it means less network traffic to send the data.

That said this is very subjective. We’ve found gzip compression at 6 is the goldilocks range between compression and deploy speeds. Gzip only has a max compression of 11 and is extremely slow without a significant difference in space used meaning no faster deploy either. ZSTD is really quite amazing and I’ve found setting to it’s normal max of 19 is still fast enough for uploads and size on disk. ZSTD at 20 thru 22 takes too much memory and often freezes.

Your mileage may vary buy hopefully this helps you make a decision.

compman

zstd would be even better if it was possible to use its new --long option, which can be associated with level -19 and multithreading for amazing compression ratio and speed, without the known issue of too large memory usage from --ultra modes (levels 20+). Downside: it requires a fairly recent version of zstd (v1.3.4+).

Basavarajnc

@compman Dear Sir (compman),

I am New Comer to this… But One Thing is that Clear: I Love Linux and This Kind of Stuffs ( One more Important is that: Fog Server is Part of My Life. I am so Happy about this Fog Server).

possible to use its new –long option : Where this option Comes Up ?

Compression level -19 : is it your recommendations ? am I Correct Sir?

Downside: it requires a fairly recent version of zstd >> Is it This Version of zstd-v1.3.4+) required to have Best Result — Is it Correct Sir,

Actually I have Fog-1.5.4 Version(It is latest Version). So it means that zstd also Latest One - Am I correct?

Please Kindly to rectify about What assumption I have made or written is Correct?

Heartily Thank You,

Basavaraj From India,

george1421

@Basavarajnc said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

But One thing… I observed that… capturing Image process is Slow (Approx… 800 MB to 830 MB/Minute) Before It was 3GB to 4GB/Minute… It is Because of Changing the Kernel Version??

This would be interesting to know, if changing the kernel between 4.15.2 and 4.18.x has an impact on transfer speeds if the only thing that changes is the kernel. If you are willing to help us test this concept it would be appreciated.

You can manually download the FOG kernels from here: https://fogproject.org/kernels/

Download Kernel.TomElliott.4.18.3.64 kernal and save as bzImage4183.
Now transfer bzImage4183 to the fog server using winscp or pscp utilties into the following path: /var/www/html/fog/service/ipxe
Go into the host definition in the fog gui to the same host that gave you 800MB/s transfer rates.
Add the following to the Host Kernel field: bzImage4183
Now schedule a deployment.
Monitor the transfer rates.

I also have to say that 800MB/min is suspiciously close to the theoretical max transfer speeds of 100MB/s (not 1GbE) network speeds.

george1421

@Basavarajnc said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

possible to use its new –long option : Where this option Comes Up ?\

That is a feature the developers would need to add because it requires an adjustment to the php code. You can do this if you understand php programming.

Downside: it requires a fairly recent version of zstd >> Is it This Version of zstd-v1.3.4+) required to have Best Result — Is it Correct Sir,

I might suspect that zstd in FOG is not the latest, because FOG uses buildroot to create FOS (the linux OS that runs on the target computer). If buildroot from February 2018 has zstd 1.3.4 then so does FOS.

Basavarajnc

@george1421 Dear Sir,

Definitely I will do Help Without Second Thought to test this Concept…

I need little Time for making changes to …

Little confusion in below Section 2 and 5

-2. transfer means copying the bzImage4183 To This /var/www/html/fog/service/ipxe … is it Correct way of understanding myselft?

-5. Now Scedule a deployment -->> Sir, it is Not During Deployment Of Image…

It is the time of Capturing Image From Client… (It took 800MB/s Transfer Rates).

Thanking You Sir,

Basavaraj From India,

george1421

@Basavarajnc said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

-2. transfer means copying

Yes trasfer == copy == move however you get the file to that directory.

-5. Now Scedule a deployment

I have seen capture as low as 800MB/min depending on your compression setting and the performance specifications of your target computer. The target computer’s CPU is used heavily during image capture to compress the hard drive image before its sent to the fog server. If you have a slow target computer, with an old CPU, or slow hard drive, or too high image compression will cause slow capture speeds. But for image deployment to same computer it should be faster.

Sebastian Roth

@Nicolas-T @phil-dosi @Basavarajnc @compman @Tywyn Good news to all of you, we have figured out what is causing the slowness and fixed it. It’s been a longer story [ref] - thanks again to @Quazz for doing lots of testing and finding very helpful stuff to figure this out!

As we have not had enough time to push for a new release yet you can simply update to the latest kernel and inits like this:

sudo -i
cd /var/www/fog/service/ipxe
mv bzImage bzImage.orig
wget https://fogproject.org/kernels/bzImage
mv bzImage32 bzImage32.orig
wget https://fogproject.org/kernels/bzImage32
mv init.xz init.xz.orig
wget https://fogproject.org/inits/init.xz
mv init_32.xz init_32.xz.orig
wget https://fogproject.org/inits/init_32.xz

Basavarajnc

This post is deleted!

george1421

@Basavarajnc Please follow Sebastian’s instructions to solve your slow partition creation issue. He found the root of the problem. Please confirm its fixed on your system so the developers can mark this issue as closed.

Basavarajnc

@george1421 said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

@Basavarajnc said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

But One thing… I observed that… capturing Image process is Slow (Approx… 800 MB to 830 MB/Minute) Before It was 3GB to 4GB/Minute… It is Because of Changing the Kernel Version??

This would be interesting to know, if changing the kernel between 4.15.2 and 4.18.x has an impact on transfer speeds if the only thing that changes is the kernel. If you are willing to help us test this concept it would be appreciated.

You can manually download the FOG kernels from here: https://fogproject.org/kernels/

Download Kernel.TomElliott.4.18.3.64 kernal and save as bzImage4183.

Now transfer bzImage4183 to the fog server using winscp or pscp utilties into the following path: /var/www/html/fog/service/ipxe

Go into the host definition in the fog gui to the same host that gave you 800MB/s transfer rates.

Add the following to the Host Kernel field: bzImage4183

Now schedule a deployment.

Monitor the transfer rates.

I also have to say that 800MB/min is suspiciously close to the theoretical max transfer speeds of 100MB/s (not 1GbE) network speeds.

Dear sir(george1421),
1 Downloaded the Kernel.TomElliot.4.18.3.64 Kernal and Renamed it to bzImage4183. Taken that file In Pendrive and Copied it in Fog Server Machine in the Following Path-> /var/www/html/fog/service/ipxe

I went into Host Definition in the FOG GUI to the Same Host …
Added the File “bzImage4183” by Writing**(manually Written)** in the Field of Host Kernel… and then Updated Successfully by clicking Update Button in Make Changes? Field.

I scheduled a Deployment for Same Image Which captured from same Computer at Rate of 800mb/s During Capture.

I monitored the Transfer Rates At Rate of Transfer Speed is 7.5 GB to 8.2 Gb while deploying the Image…

Now My Question is … Manually Written the file ->“bzImage4183” in the Host Kernel Field… is it Correct Sir?

Thereafter, I updated by using Update Button in the Make Changes? Field… is it Correct Sir?

It is for my Satisfaction and Confirmation myself.
My Intention is …What you asked Questions should be matched with What I understood …

Honestly Thankful to you Sir,![alt text]
Basavaraj From India,
0_1541407968385_Reqsted-From-George1341_Deploying-Image-ScreenShort.jpg

george1421

@Basavarajnc said in It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.:

Now My Question is … Manually Written the file ->“bzImage4183” in the Host Kernel Field… is it Correct Sir?
Thereafter, I updated by using Update Button in the Make Changes? Field… is it Correct Sir?

Yes this is correct. This field allows you to assign a specific kernel to a specific host. That is the design of FOG.
No this is not correct. You have hard defined for that host it will only use bzImage4183. The main and updatable kernel is called bzImage. When the next version of FOG is released (1.5.5) it will only update bzImage. The host where you statically defined bzImage4183 will continue to us that kernel forever.

Also Sebastian’s fix also requires you to update the init.xz files to completely eliminate the delay during disk format creation. So both the kernel (bzImage) and the virtual hard drive (init.xz) need to be updated to correct this problem.

It takes longer Time (More Than 5 Minutes) Erasing current MBR/GPT tables.

236

11.6k

17.1k

154.6k