Latest FOG 0.33b
-
Tested the new revision, get an error from fog service, that the file size is zero.
In file snapins.file.php on line 53 is not given the full path to the file in filesize. After fixing that, the snapin could be downloaded.
Another problem now, the task for the snapin is not get remoed after the installation has finished, means the fog service download/runs the file after it finished again. -
Added the file location to the snapins.file.php file.
Went back to getId() rather than get(‘id’) just as test point. I’m thinking it can’t find the task, therefore can’t remove the task.
r1023.
-
Hi! I’m on revision 1023 and I have two problems when I’m trying to upload an image:
[LIST=1]
[]This is not really important, I have an annoying message that stays for 4-5 minutes with this information. After that the system continues with the image upload. [IMG]https://lh4.googleusercontent.com/-XzXBiI-hhHE/UqjvRCumKuI/AAAAAAAAZIM/X460BXOk748/w929-h697-no/IMG_20131211_183223.jpg[/IMG]
[][S]Sorry, this problem is solved. My fault[/S]
[*]There are still problems with logical partitions and swap?
[/LIST]
Thank you,
Joan -
This is normal behaviour with the kernel. The kernel has to load all the drivers, and the drivers it thinks it needs, when the client is booting into the system. Some systems take longer, some systems take much less time. I don’t know where the differences lie within all the possible types of systems, so this is likely to be something I’m not going to be trying to correct for. The error’s you’re seeing are, in some way’s, to be expected, as they’re basically just telling you the kernel loaded the driver, but there’s no equipment or item to relate it to. They’re generally, as I like to call them, “false” errors.
I don’t know if there are still problems with logical partitions and swap. The r1023 -> and up, revisions should have mkswap/swapon/swapoff and should read the partitions, though I don’t know your particular setups. It’s also using partclone now, so creating/deploying the image shouldn’t be a problem in and of itself.
The one thing I haven’t played with yet, is just using partclone to make an exact replica of the drive vs. the partitions.
I imagine the partitions should only be needed specifically for resizable disks, but could forsee an issue if we copy the drive as is in disk size issues later on. What I mean by this, is if we copy the drive “as is” will it affect drives that might be larger than the original image created drive in limiting the actual size available? These are unknown to me which is why I tried leaving things in, more or less, the same state as before.
I hope this helps. If you’re noticing issues with logical partitions and swap, please try to describe what you’re seeing.
Thank you,
-
Ok, so the error message isn’t the cause for the delay.
Logical volumes seems to be the other problem, I tried today with this setup:
[LIST=1]
[]Primary ext4 --> / of ubuntu 12.04
[]Primary partition --> extended
[LIST=1]
[]Logical ext4 --> /home of ubuntu 12.04
[]Logical swap
[/LIST]
[/LIST]
When I tried to restore, only partitions 1 (primary) and 2 (primary) were restored, the extended partition was empty. I always try to backup and restore full disk, not independent partitions.
Thank you very much. -
My only guess, as to the issue, is that primary partitions are labeled, generally, as : /dev/sda1, /dev/sda2 etc…
I’m guessing, if you were to perform fdisk -l on a working system at that point, you’d see partitions labeled as:
/dev/sda1
/dev/sda2
/dev/sda2p1
/dev/sda2p2Or very similar.
FOG doesn’t search for partitions within partitions, it only searches, to my understanding, for primary partitions.
I don’t know how I can try to make logical partitioning work. My question, though, is why couldn’t you make /home a primary partition on the device. Swap as well?
I only ask this because I’ve tested ext4 imaging on my systems at home, with a similar setup:
/dev/sda1 = / partition
/dev/sda2 = swap partition
/dev/sda3 = /home
/dev/sda4 = /tmp -
[quote=“Tom Elliott, post: 20899, member: 7271”]My only guess, as to the issue, is that primary partitions are labeled, generally, as : /dev/sda1, /dev/sda2 etc…
I’m guessing, if you were to perform fdisk -l on a working system at that point, you’d see partitions labeled as:
/dev/sda1
/dev/sda2
/dev/sda2p1
/dev/sda2p2Or very similar.
FOG doesn’t search for partitions within partitions, it only searches, to my understanding, for primary partitions.
I don’t know how I can try to make logical partitioning work. My question, though, is why couldn’t you make /home a primary partition on the device. Swap as well?
I only ask this because I’ve tested ext4 imaging on my systems at home, with a similar setup:
/dev/sda1 = / partition
/dev/sda2 = swap partition
/dev/sda3 = /home
/dev/sda4 = /tmp[/quote]Yes, your configuration works for my “test”, I was only testing if logical partitions were working.
I need for example this scheme:
[LIST=1]
[]ntfs Windows 7 (I force to 1 partition install as for default it takes 2 partitions)
[]ext4 / ubuntu
[]ext4 /home ubuntu
[]swap
[]If I would like to make a data partition I can’t because all primary partitions are full
[]If I would like to install another OS I can’t because all primary partitions are full
[/LIST] -
Just doing some tests and I’ve found the following: (Latest Debian 64 bit, Test VMs 512mb RAM with Single/Dual vCPUs)
- When you manually enter a MAC, and then do a hardware inventory it gives an invalid MAC address format at the end of the inventory.
- NTFS resizable Image only does the first 100mb partition on windows 7
- When viewing one of my hosts I get the following error:
FOG DEBUG:Image:Database Load Failed:ID:0:Row Not Found
Fix - Give hosts an image association. - When the mouse goes over the deploy button, it changes to garbled words.
- Error 1 can occur after fog registers the host as well as manual entry.
- FOGImageReplicator - didn’t seem to work on it’s own… needed Manual restart to push image. (Possible use Bitorrent Sync instead)
- Deploy seems to error when using resizable image with a single partition windows 7?
- Total block progress only ever gets to 47% and then jumps to 100% when imaging using multi partition image
- Is Partclone CPU intensive? quad core VM managed 1gb/m but single core only 310mb/m upload speeds. or is compression enabled by default?
Great Improvements so far and nice to see that Tom is still going strong. Hopefully we can get all the tiny bugs fixed and we could have the next release of FOG
Some suggestions:
On the Hardware inventory it displays what the host is doing… would it be possible to display an image and have a progress bar? (Some users like to try and do strange things like unplug PCs when they see them doing something they don’t know)
Compression settings in the GUI or in the installer?
Bandwidth graph shows both send a receive instead of picking one?
Partclone displays Block xxx of xxx… could it display progress in MB/GB?
Could logs be sent to the server? -
VincentJ,
Thank you very much for the suggestions/bugs.
I’ll try explaining/showing my assistance where I can.
First Question: What revision of FOG 0.33b are you using? I ask is it is pertinent to your 2nd tip/find there.
In response to 1:
I will try to take a look at this as I’ve never done a manual add, then an inventory. Maybe the MAC entered is trying to impede on a database entry it thinks doesn’t exists, but at the end it finds it? I don’t know, I’ll test and see if I can replicate the results.In response to 2:
If you’re on the latest version of FOG, and this issue is still happening, I don’t know where else to go. Basically, I’ve rewritten the fog script to take into count different size Hard Drives. Parted doesn’t like the use of kB mB, etc… for disk size. Nor do all drives like to use 100% for the layout. This doesn’t seem right to me, but it’s up to the creator of parted how they want to do things. I’m looking to test with -1s for the end to use the full size of drive in place, but don’t know what repercussions they may pose. With that said, what size is the Hard Drive you’re using? 50Gb, 80Gb, etc…In response to 3:
This is expected behavior. I think the “DEBUG” is only enabled as this is a beta release. If you disable debug on the OS, Image, and Host pages, this clear’s itself right up. That said, I believe the latest version (r1024) has this, issue, taken care of, though I could very well be wrong.In response to 4:
I don’t see any change on my side. Can you try being more specific? Meaning, are you using a different language?In response to 5:
I’m assuming you mean, basically, doing quick host registration after full inventory of the host?In response to 6:
I don’t have FOGImageReplicator on my end, so I may need to setup a storage node and test out some more. Sorry about that.In response to 7:
This sounds like my response to number 2. I’m guessing parted is actually errorring out and not creating the partition as required.In response to 8:
That’s actually a piece of partclone. It’s nothing I can change directly. I suppose I could look at their source, but I’d probably cause more problems. My guess for this is partclone is only copying the data, not the free space, so blocks not being at the same pace as the rest of the drive actually makes sense. For example, Let’s just say we have a 10GB hard drive, with only 2GB of data. The blocks progress, should be less than the main progress. It’s only copying used blocks of data, not free. So when it encounters the free blocks, it can zoom right past. Just my theory.In response to 9:
Compression is enabled by default. It’s pigz compression, so it uses the systems total number of cores to speed up progress. With that, it’s a part of the FOG scripts built into the init.gz. If you don’t need/want compression, you’ll have to decompress the init.gz, mount the init file, and edit the bin/fog script. The compression ratio, I’ve set, is -9. If you don’t want any compression, I’d think putting it as -0 would work.In response to the suggestions.
I don’t know what you mean about the hardware inventory showing what’s happening with the system. Maybe there’s something beyond what I’m aware of. I’ll take a look to try to see what you mean. Adding the image and progress bar, I don’t know exactly what I’m doing in that respect. I’m trying to compare the progress indicator from 0.32 to what we’ve got in 0.33 so I could, potentially, get something like that going, but it’s not a guarantee.
Compression settings could be added to the GUI, now that I understand, better, what’s actually happening. It would require a minor rewrite in how the system picks up and builds the PXE file for the tasks. It would also require adding another field to the FOG Settings table on the database. Probably called Compression Ratio.
Bandwidth graph, I’ve noticed, does have a few bugs, but I don’t know where they lie yet. I’ve been looking at it for quite some time.
Partclone, unfortunately, is it’s own thing. I’ll try seeing of there’s command line arguments that can provide that for us.
What do you mean about logs being sent to the server? You mean essentially creating a log on the FOG Server, for all tasks? I don’t think this is impossible, but imagine it causing issues in filtering and reporting. -
It was downloaded yesterday, i’ll try and go into it later and find the revision number.
Usually when i purchase new desktops/laptops the MAC is written on the side of the box so i can pre-stage the computers in FOG. Then when putting them together they only need a BIOS password setting and changing to PXE boot and they sort themselves out.
Usually once we have put in a load i’ll set a hardware inventory job for later so we can have more basic info in fog. (I have a program that does a more complex inventory)
If i were you… I wouldn’t bother with the FOGImageReplicator. Now that i know i have a working deploy i am going to test out the storage server and try using other methods of replication. Allowing it to be done in a custom manner will probably suit those running more complex setups with storage nodes. If they have a lot of them, Many to Many replication may suit better. They could configure something as simple or as complex as they needed.
It’s good that compression is enabled by default… most PCs are a lot more capable than the single core VMs i’m testing on and it shrank my 12gb image to 5gb.
For the logs thing… when partclone was giving the error, it was saying where it stored the error. but i couldn’t get to a command line to look at the log file. maybe since fog has FTP you could add a script to copy the logs directory after an error to the server’s images directory?
The VMs i were using had 25gb HDDs and 512mb RAM. (i know low power, but i only have 2 8GB hosts to play with at home)
I was impressed with FOG running nicely on 512 and even 386mb RAM in debian without using too much swap.
I now have access to a 72GB RAM Test server online, so i’m planning some bigger tests and VPN tests. (Uploading ISOs has taken quite a while but i should be able to do some beefy tests over the Christmas period.)The GUI button error just seems to be in the debian browser… works fine in IE. (Image Attached)
The Image thing over the inventory - When fog loads it runs through what it’s doing, such as the contents of the inventory and that it is sending the inventory to the server. If a static image could be put in front of that so the user wasn’t ‘confused’ by the strange things on their PC it might make things a bit better. Also people could customize it to their company logo and their employees would know something is supposed to be happening. A progress bar would just mean they could have some indication that something was getting done.
[url=“/_imported_xf_attachments/0/454_GUIButtonError.png?:”]GUIButtonError.png[/url]
-
This may sound weird. I’m testing with, rather than size or 100% on the parted side of the house, just setting 99%. It means, theoretically, that you’re not getting 100% of the drive. However, it should work across all hard-drives. If there’s a major reason we shouldn’t be using 99% of the drives, please let me know and I’ll try to figure out another solution.
-
r1025 released. Should address the iceweasel browser hover issue on deploy as noted by VincentJ. Also, parted uses 99% for layout rather than 100% or the actual available disksize. This seems unusual, but Not all disks like 100% use and not all like the diskSize for the value. I figure lose 1% of the drive and you can recoup later on as needed.
-
Vincent,
If it’s of any help, the logs are located at /var/log on the init.gz. If you’re trying to get to a point where you’re able to type linux commands, you’ll need to have the system setup in debug mode.
-
Did you test to start windows xp after downloading the image back to the hard disk? I tested it and it works on my side, but after I try to start windows xp I get this error: Corrupt boot.ini file
-
I’ve tested it, and it works for me. I will try again but it will be a little later tonight as I am not at home to test okay?
-
No problem, ty
-
All,
Just an update. I know I’ve been not quite updating you all as frequently as I like to, but it is the holiday’s so please forgive me.
I am still working diligently and have released a couple new revisions. We’re now on 1027 in case you’re wondering. My tarball is still on 1026. The only difference between the two is the php-mysql to php-mysqlnd stuff for centos is now working properly. The variables didn’t like spaces which I was unaware of a while ago. It would try running the command then set the command as a variable which is just wrong.
I’m currently making a class to do the pxe file creation for us. It uses better variable management for easier adding if you needed to.
Hopefully this helps.
Thank you,
-
r1028 is out with default PXE file creation in class.
Please take a look at the {fogwebdir}/lib/fog/PXEFileGenerator.class.php file as it should be much easier for people to incorporate such setups as ISO’s and other customizations. I’ll try documenting it in the next few days to try to have some sort of sense out of it.
The intention of this, all-be-it, small change is to try getting to a point where the “legacy” items are not as needed as they once were. Though OOP seems to create more files, this actually makes things easier to manage/understand, so please bare with me on this and I hope you all enjoy.
If you need comparison open up the file: {fogwebdir}/commons/functions.include.php and search for generatePXEMenu and scroll through the where the strMenu variable is creating the many, many, Class reference variables. This new method is much neater, and makes much more sense in my eyes at least.
Thank you,
-
Tom,
Since 0.33b is getting more stable and getting more features… have you planned a feature freeze or time to work on just stability so that there could be a release soon?
-
Vincent,
I don’t know that I’m the source to ask for this. I’m trying to get it as close to finished as possible, but ultimately I think the release table should be up to Chuck and/or Jian as it is their program.
If it were my choice, a feature freeze, beyond simple mechanics of the system, should already be in place.
Some of the features I’ve added are:
Group Management Page, can create groups, can add hosts/set of hosts.
Host Management Page, can see groups associated to host, can add host to group/set of groups.
Printer Management, Stores type of printer in database, informs user of the type of printer it is. Editing printer gives you the same choices as creating. (Should minimize guess work when needing to edit a printer.)
Configuration Save in FOG Settings, basically just creates a backup copy of the current database. Useful if you have to change servers quickly.
Installation of fog checks for both mysql and mariadb, as well as php-mysql and php-mysqlnd *FOR REDHAT INSTALLS FOR NOWI haven’t really added much else in the way of features and don’t know that I should work of feature additions at this point. I’m still trying to narrow down/fix snapin deployments.
