Posts made by benc

@Sebastian-Roth I’m building a server that has to be ready first thing tomorrow morning so I haven’t been able to put as much time into this as I would like. Based on your suggestion, I tried using a hard drive that was => the original source drive. This worked perfectly, just like pretty much every time I’ve tried to do this type of thing before. Looks like something to do with where the partitions were on the original drive not being properly mapped to the new SSD…? I’m going to take a closer look at all the partitions and where they are located on the drive. It might be as simple as getting rid of a little recovery partition at the end of the drive or something like that. I’ve never had to use a drive that was => the original drive. It has always captured the images and as long as total contents of each partition is less than the capacity of the destination drive it works fine.

@Quazz Here is a picture of the error.

@Wayne-Workman I will snapshot the VM now and get Fog upgraded to the working branch. I’ve never done that but I remember reading about it. Shouldn’t be too hard to figure out. Would you suggest doing anything to Ubuntu as far as upgrading or updating? It’s been a few months since any OS updates have been applied.

It’s a VM running on Server 2008 R2 with VirtualBox.

Fog Server 1.4.4
Ubuntu Server 16.04

Re: "Seems like you are trying to restore to an empty disk" issue

I seem to be having this issue when deploying a certain image. In this case the image is from a normal EFI Windows 10 install. I’m trying to use Fog to make an image of a mechanical hard drive so I can turn around and deploy the image to an SSD. I’ve done this probably a 100 times over the last year or two with no issues (once I figured out that sometimes a partition just can’t be resized and the image type in Fog needs to be set to Multi Partition, No Resize). Capturing the image from the original hard drive works normally and succeeds. I’ve got ~150GB of image files in the /images folder associated with that image, so everything looks normal there. When deploying the image to the SSD, it goes through all the normal steps until it’s time to launch partclone, just as the picture shows. I’ve tried capturing the image again by putting the hard drive in another more trustworthy machine, and it captures just fine but still won’t deploy at all, even using a different machine and a different SSD. I’ve even tried deploying to another hard drive.

I’ve ruled out the original PC, the new SSD/hard drive, and even tried it on a different network using a different Fog server. I’m thinking there’s something weird about the original hard drive and/or the way it is partitioned. I plan to try a debug deploy to see what info that may reveal. I’ve never done a debug task before so that will be a new experience. Any ideas what is causing this?

@george1421 Thanks, I will give that a try later today.

Please forgive my ignorance, but could someone point me to a guide on how to change kernels? I’m pretty sure I understand at least at a basic level what the kernel is and what it does, but I’ve never done this before and the googling I’ve done on installing new kernels has only confused me. In my case the step where FOG erases the MBR/GPT tables is taking 6+ minutes which is longer than the actual imaging process takes. As soon as I get past this current batch of laptops that must be imaged ASAP, I’ll be starting from scratch with a new hard drive in one of my fog servers and probably starting with Ubuntu Server 14 and not going any higher. So far, in my experience, upgrading one thing breaks other things and it just snowballs from there. Sorry for the long post but this has been a very difficult day.

@sebastian-roth The switches at each location are identical, and the configuration is fundamentally the same except that some locations have two switches stacked together to provide enough ports. One VLAN, same addressing scheme, same types of devices connected. Right now I’m really combing through the details of the configs, comparing the working locations to the ones that don’t. There could also be something with the fact that some locations have two switches and others have just one. That shouldn’t matter, but who knows. I’ll check back in with my findings.

I’ve tried putting new drives in 2 PCs and trying a multicast again. Same results. I tried the same thing at another location and actually got up to 98% before it got stuck. I tried a couple more times and it hung randomly around 90%.

I’m starting to think that I made a mistake by trying to keep our FOG servers up to date. I’m relatively new to the Linux world and I just assumed that running apt-get update / apt-get upgrade / apt-get dist-upgrade / do-release-upgrade every now and then was probably a good idea to keep security tight. I have not had time to rebuild any of my FOG servers yet to see if that fixes my issues. When I do rebuild, I’ll most likely just throw a new drive in and start over. For long-term stability and reliability, what distro/version should I go with? Most of my experience in Linux has been with Ubuntu so I’d like to stay with that, but I’m open to suggestions.

The last 3 of my FOG servers I’ve been working with have successfully completed all multicasts. It looks like I’ve got the issue with about half of my servers. Haven’t yet found the issue or the difference between the working servers and the non-working servers. I may just try to reinstall Ubuntu Server 16 and start there. One thing I did try on the last server with the issue was to reinstall FOG. It failed on every package that had curl in it. Don’t know anything about curl but maybe that’s a clue.

I am working in a different location today, and both of the multicasts that I tried have worked all the way through. I copied the same image I have been using all along from the last location’s server to this location’s server, deployed it to 1 PC, changed a few settings, captured, and used multicast to deploy it to 5 PCs and then 3 PCs. I have attached one of the successful multicast logs from the server at this location.

0_1533155793310_multicast.log.udpcast.12.log

@sebastian-roth I will try putting different hard drives in the clients, and if that shows the same results I’ll probably just reinstall Win10 on one of the machines, capture that, and use that as my smaller test image.

@sebastian-roth My guess is that the problem, whatever it is, only shows up on a partition that is over a certain size. Or perhaps it has to do with the time elapsed. I wouldn’t think it has anything to do with the type of partition or the data it contains. This image is a pretty straightforward UEFI Windows 10 install. The first 3 partitions are whatever Windows puts there during install. The last partition is NTFS. I am thinking about finding another smaller image to test with and see if maybe the smaller image multicasts successfully.

0_1533058838262_d1.fixed_size_partitions.log
0_1533058851863_d1.minimum.partitions.log
0_1533058865275_d1.partitions.log

@snaggel I’ve run into this issue quite a lot in my environment. It took a ton of work to figure it out. I sometimes copy images from one fog server to another, and in the process the permissions on the folder/containing files were getting changed. In my case, running <code>sudo chmod 777 -R /images</code> fixes my issue.

0_1532980312008_multicast.log.udpcast.1.log

I pulled one of the multicast logs from the last server I used. Had to add .log to the end of the file to upload it here. Hope this helps.

@george1421 I haven’t had a chance to dig much deeper into this issue since my last post but I did look through the logs of the last server I was using. I see a lot of timeouts, but I’m not familiar enough with the logs to be able to identify the issue. I can post a log here if that would help. Which log(s) would be relevant?

@george1421 I edited www.conf and set the values you specified. There was only one copy, located in /etc/php/7.1/fpm/pool.d/. The server at this location is running Ubuntu Server 18.04.1 and FOG 1.4.4. Tried a multicast with 5 machines and it failed the first time at 86%, then I tried it a second time and it failed at 93%. My third attempt was with 2 clients, and it failed at 90%. I attached a pic of a client after each attempt just after it hung. After it hangs, elapsed time on the clients is still counting up, and GB/min is slowly decreasing since there is no activity. I looked at each client to confirm they all showed the same thing. They all stop at the same block and the Partclone screens are identical. I checked the hard drive LEDs on each client to make sure none of them are hung or showing signs of drive failure. All clients have a 120GB SSD. During the first 3 or 4 minutes, the speed is around 11-12 GB/min, but I can usually tell when it’s about to fail because the speed will start dropping to around 8-9 GB/min. HD LEDs on each client flash as expected, and none of them are staying lit constantly. I have also tried using different images in case it is something related to the image. Also, when I use unicast to deploy, it works fine on all machines and runs near 12 GB/min the whole way through.

@george1421 Something I forgot to mention is that I’m running FOG 1.4.4 at all of my locations except one. I upgraded one of them to 1.5.4 to see what would happen. Same results. I am going to try editing www.conf as you suggested. Will report back shortly.

@sebastian-roth Yes, I have tried multicasting only two machines, and I’ve tried several different pairs in case it was a hard drive like you said. I’ve seen a failed drive twice before, and I’ve seen that if one machine drops off the network or dies for some reason, the others will hang because they only go as fast as the slowest member.

Server specs:
Lenovo ThinkCentre M90
Intel Core i5 @ 2.67 GHz
4GB RAM
2TB mechanical SATA hard drive
onboard gigabit network card

Client specs:
ByteSpeed mini desktop
Intel Core i3 @ 3.9GHz
8GB RAM
120GB SATA SSD
onboard gigabit network card

Hello. Let me start by saying FOG is freaking awesome. I use it every day in our environment. Most of what I do is unicast, but occasionally it would be beneficial to use multicast. We have 15 libraries with a FOG server at each location. Each one is standalone, and they are not replicating images or anything like that. Only one library is able to successfully finish a multicast. When trying to multicast at any other location, the task starts just fine and every machine waits until everyone has joined and is ready to begin. The first 3 partitions image ok, then somewhere towards the end of the 4th and final partition every machine hangs. The 4th partition is a little over 40GB, so it runs for probably 3 or 4 minutes before hanging. This behavior is consistent across every other location except for that one. I have not been able to figure out what is different at that location. The server, switches, router, and even the client machines at every location are identical. All fog servers were running Ubuntu Server 16.04 when we started using fog about a year ago, but I’ve tried 16.10, 17.04, 17.10, and 18.04 in the process of troubleshooting just to see what would happen. Right now the fog server at the location that is working is running Ubuntu 18.04. This past week I’ve tried at 4 other locations and the multicast always hangs before it finishes that last partition.

I’m good with networking and Windows environments, but not so much Linux.

Thanks,
Ben

edit: I forgot to mention that I’ve tried the multicast on 2 machines at a time all the way up to 16 at a time. Our switches are Juniper EX-3300, 48 ports with PoE. Fog server and all clients are on the same VLAN.

@sebastian-roth Success! I read through /var/www/fog/service/ipxe/refind.conf and changed the default scanfor internal to scanfor internal,hdbios and now everything is booting whether it has a UEFI or BIOS based image. Thanks for pointing me in that direction. I have not tried converting a BIOS image to a UEFI image, so I don’t know if that would have worked. I would prefer to just change an option in a config file though. Thanks again for the help.