Another upload/download speed issue

the_duke · Mar 1, 2016, 9:55 PM

My colleges and I have noticed over the last couple of months that 2 of our images seem to upload at a faster speed than the others do and then in turn, deploy at a slower rate than the others. My FOG_PIGZ_COMP set at 9 and has always been set at this since I’ve had it. Ordinarily, my upload speeds are under 1GB/min, about 800 MB/min, and download speeds at around 3-4.5 GB/min. Right now, I have a computer uploading a 1.6-1.7 GB/min and will download at about 500-800 MB/min. What is causing these 2 images to be a different speeds than the many others that I have done. These 2 are some of the first ones that I created on my fog server.

Tom Elliott · Mar 1, 2016, 10:25 PM

My guess is my stupidity, though I can’t be sure.

Look on the image definitions themselves. My guess is you’re running Trunk, and the image’s have their own compression value defined (even if you didn’t define it) and it’s returning as “0”. Because of this, the uploads are actually sitting at 0 compression ration (which still has compression yes), and in turn the download is slower as it has to transfer more data across the network.

For kicks and giggles, I’d recommend setting all your images to compression ration 5 or 6 and see if that helps you out. 9 is not as nice because while the data across network is much more minimized, the cpu has to take its time to decompress the data to put it on the hdd.

Our own community has thrown together some basic rundowns of performance it it always seems to be a compression ration of 5-6 as the “best”.

the_duke · Mar 1, 2016, 10:34 PM

@Tom-Elliott Sorry, should’ve said so before, I’m running 1.2.0…I did try trunk for a brief moment to test something, but came back to this. However I never uploaded any of the images while I was in trunk. It could be possible that while I was on trunk that it changed the settings by itself. I have a good idea that it will, but want to confirm, will running it at 5 or 6 make my download speeds slower? This would not be something that we would want, after all, we do deploy more than we upload and time is of the essence when in a classroom full of kids.

Tom Elliott · Mar 1, 2016, 10:36 PM

@the_duke 1.2.0 didn’t have the speed improvements natively in the installer.

What I can do is crack open the 1.2.0 init’s and host them for you to download with the “nice” changes if you’d like. By nice I mean the speed tweaks that make trunk a much speedier beast.

Tom Elliott · Mar 2, 2016, 1:46 PM

@the_duke Speedier init’s are located at:

http://mastacontrola.com/init_1.2.0_speed.xz
http://mastacontrola.com/init_32_1.2.0_speed.xz

To download to your fog server you should run as root:

If on redhat or /var/www/html exists on your installation and fog is located at /var/www/html/fog

wget http://mastacontrola.com/init_1.2.0_speed.xz -O /var/www/html/fog/service/ipxe/init.xz
wget http://mastacontrola.com/init_32_1.2.0_speed.xz -O /var/www/html/fog/service/init_32.xz

If on ubuntu and/or /var/www/fog is the location:

wget http://mastacontrola.com/init_1.2.0_speed.xz -O /var/www/fog/service/ipxe/init.xz
wget http://mastacontrola.com/init_32_1.2.0_speed.xz -O /var/www/fog/service/init_32.xz

Wayne Workman · Mar 2, 2016, 2:03 AM

@the_duke Consider yourself lucky.

the_duke · Mar 2, 2016, 1:46 PM

@Tom-Elliott Ok, I just ran this and now trying to see how it goes.

the_duke · Mar 2, 2016, 3:02 PM

@Tom-Elliott Ok, I tried it once and it was still downloading at about 457 MB/min. I then tried to re-upload an image for it and then push it out. I was uploading at about 1.65 GB/min. While it is a little better this time around, I’m still only downloading at about 1.4 GB/min. I had a couple of other computers with a different image download at over 5 GB/min since I made this change. These computers that are going slower are custom built computers that were built 2.5 years ago and have much better hardware than what is on the rest of my systems as they are either refurbished units or 5yr old laptops.

Wayne Workman · Mar 2, 2016, 3:05 PM

@the_duke Sounds like it could be your network, or maybe a hardware issue on the fog server. Like a dying HDD or maybe a bad stick of ram, or maybe the RAM just needs reseated.

If you have a 1Gbps unmanaged switch you could hook your FOG server to, and then hook one target computer to that switch, and try to image through that, you will be able to eliminate your network as a variable and see if things improve or not.

the_duke · Mar 2, 2016, 3:12 PM

@Wayne-Workman I can try that, but it doesn’t make sense to me as to why on a different computer with a different image being downloaded to it that it will be a lot faster. In my mind it would seem that if it was my network or a hardware issue on the fog server, it would be this slow with all of my images, not just this one.

Wayne Workman · Mar 2, 2016, 3:18 PM

@the_duke Could be just a bad port, a bad few ports, a bad switch… a number of things that I’m not even thinking of right now.

Troubleshooting is about eliminating variables.

the_duke · Mar 2, 2016, 4:35 PM

@Wayne-Workman @Tom-Elliott Ok, I’ve got 2 different custom built computers with a separate image for each model. Both of those computers are having the same slow speeds. On every other one I’ve tried so far, I’m getting between 4-5 GB/min download speed. I’ve even tried using the same port for each computer and same results. Is there something in the kernel or possibly the boot file that could be doing this? All of my switches in my system are brocade icx6430s. These are enterprise grade switches so if something was going bad on one, I’d know. I know it’s not a coincidence that everywhere I have one of these 2 custom built computers its slower on the download side.

Wayne Workman · Mar 2, 2016, 4:38 PM

@the_duke said:

On every other one I’ve tried so far, I’m getting between 4-5 GB/min download speed.

Can you elaborate on that?

Is this with different computers, or the same computer? Different models or the same model?

the_duke · Mar 2, 2016, 4:47 PM

@Wayne-Workman A different computer with an image for that particular model. I don’t have a master image, just an image for all the different computers that I have.

Wayne Workman · Mar 2, 2016, 4:51 PM

@the_duke Ok so for these tests, you’re using the same model and same image, but different machines, right? Just want to make sure I’m reading it right.

the_duke · Mar 2, 2016, 4:59 PM

@Wayne-Workman no…I first reimaged one of my custom built computers with its own image and it ran about 500 MB/min. I then tried one of our refurbished units with its own image, using the same network port and had an average download speeds of 4 GB/min.

Hope that clarifies it a lot

Wayne Workman · Mar 2, 2016, 5:00 PM

@the_duke Are the two images you tested with the same image? Are the two computers the same model?

I’m not trying to be dense, believe me.

the_duke · Mar 2, 2016, 5:04 PM

@Wayne-Workman no, different computers, different images.

Wayne Workman · Mar 2, 2016, 5:55 PM

@the_duke Ok. The model that has higher transfer speeds, are those higher speeds consistent with this model no matter where it is on your network?

And, the model that has the slower speeds, are those slower speeds consistent with this model no matter where it is on your network?

the_duke · Mar 2, 2016, 6:07 PM

@Wayne-Workman correct

Another upload/download speed issue

169

12.0k

17.3k

155.2k