Posts made by Jim Graczyk

@wayne-workman All nodes are already up-to-date (as reported by git pull). We had updated to v24 at 10 am this morning and retested replication before posting this issue to the forum.

Jim

@wayne-workman

I was able to connect to each node’s FTP service with the credentials stored in the storage definition in FOG and transfer files into and out of each server.

Jim

@wayne-workman

I’m thinking the FTP errors are the result of failed replication, not the cause - again, but what do I know…
I’m thinking this only because the FTP errors are from FOG client machines attempting to access a snapin that hasn’t replicated.

I’ll test the FTP connection of each node for files I know are actually on each node and report back…

Jim

@wayne-workman

I did the tail command you posted with ’ | grep error’ on the end. Here’s what matched:

[root@Sal1FOGV1 bin]# tail -n 500 /var/log/httpd/error_log | grep error
[Mon Sep 04 12:06:49.751559 2017] [:error] [pid 110308] [client 192.168.100.171:49805] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Mon Sep 04 12:25:33.023607 2017] [:error] [pid 110150] [client 192.168.100.171:49721] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Mon Sep 04 13:00:58.193160 2017] [:error] [pid 86532] [client 192.168.100.171:49742] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Mon Sep 04 13:02:51.022016 2017] [:error] [pid 93233] [client 192.168.100.171:49766] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_PP.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Tue Sep 05 14:31:15.976881 2017] [:error] [pid 89324] [client 192.168.100.196:50414] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
[Tue Sep 05 15:00:08.147464 2017] [:error] [pid 86534] [client 10.179.100.156:49891] PHP Warning: array_filter() expects parameter 1 to be array, null given in /var/www/html/fog/lib/fog/image.class.php on line 164, referer: http://fogserver/fog/management/index.php?node=image&sub=membership&id=23
[Tue Sep 05 15:30:25.281071 2017] [:error] [pid 86534] [client 10.179.100.177:58270] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
[Tue Sep 05 16:20:21.716391 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.716447 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_delete(): Delete operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 244
[Tue Sep 05 16:20:21.717251 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.717739 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.718304 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.718808 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.719324 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.719736 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.720228 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.720734 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823
[Tue Sep 05 16:20:21.868833 2017] [:error] [pid 86534] [client 10.179.100.177:42560] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
[Tue Sep 05 22:18:12.376965 2017] [:error] [pid 6299] [client 192.168.100.196:49729] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Tue Sep 05 22:23:05.179099 2017] [:error] [pid 6299] [client 192.168.100.196:49690] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_Std.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 10:06:51.808428 2017] [:error] [pid 101284] [client 192.168.100.196:50025] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 14:23:46.601396 2017] [:error] [pid 12985] [client 10.179.100.156:54027] PHP Warning: array_filter() expects parameter 1 to be array, null given in /var/www/html/fog/lib/fog/image.class.php on line 164, referer: http://fogserver/fog/management/index.php?node=image&sub=membership&id=24
[Wed Sep 06 15:01:43.531662 2017] [:error] [pid 12985] [client 10.179.100.176:57724] PHP Warning: ftp_chmod(): SITE CHMOD command failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 137
[Wed Sep 06 15:44:05.209565 2017] [:error] [pid 188731] [client 192.168.100.171:49720] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 16:07:37.877958 2017] [:error] [pid 12985] [client 192.168.100.171:49720] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Malwarebytes.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 16:10:13.754976 2017] [:error] [pid 12985] [client 192.168.100.171:49749] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/Office_v2013_PP.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 17:23:14.101811 2017] [:error] [pid 158225] [client 192.168.100.171:51205] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 18:06:31.179787 2017] [:error] [pid 8849] [client 192.168.100.171:49767] PHP Warning: fopen(ftp://...@192.168.100.30/opt/fog/snapins/1-Win10NET35.exe failed to open stream: FTP server reports 550 Could not get file size.\r\n in /var/www/html/fog/lib/client/snapinclient.class.php on line 618
[Wed Sep 06 18:29:18.857955 2017] [:error] [pid 243543] [client 10.179.100.156:56865] PHP Warning: ftp_rmdir(): Remove directory operation failed. in /var/www/html/fog/lib/fog/fogftp.class.php on line 823, referer: http://fogserver/fog/management/index.php?node=snapin&sub=edit&id=42
[root@Sal1FOGV1 bin]#

Jim

@wayne-workman

Disregard my confusion about group to group and group to node replication. I see that now in the log, accounting for each image and snapin appear twice in each log. I will assume that group to group replication replicates from node to node between groups and group to node replicates from node to node within a group. If I have that wrong, let me know.

Jim

@wayne-workman

I did a wget http://myfognode/fog/service/getversion.php from my main FOG server where I substituted the IPs of my 3 FOG servers (main and 2 storage) in for myfognode.

Each returned a file and each response was the correct version number, 24. Also, FOG Settings reports the same, as does the Dashboard page. This implies to me there is no db connection issue, but what do I know.

I’m not clear on the group->group replication verses the group->node issue you’re talking about. Sooner or later the system needs to replicate to a node, something that the replication service fails to initiate, as far as I can tell. To be clear, I see logs up to Aug 26th describing replication actions, but after that, there are no logs, like from that time on, the replication process never saw a need to start any replication actions.

I didn’t try to delete all the storage groups and recreate them. Is that the next step?

Jim

Server

• FOG Version: v1.5.0 RC-8 v24 w/ Location Plugin
• OS: CEntOS 7

Description

We have 2 Storage Nodes with same as above, 3 Storage Groups, 3 Locations, Each Location has 1 storage node in 1 storage group.

Snapin and Image Replication has stopped as of Aug 26th Working Update. We’ve removed and re-connected the storage nodes and we’ve removed, reinstalled and reconfigured the Locations Plugin, but the problem persists.

All images and all snapins are configured to replicate to all storage groups. The main FOG server is the primary for all images and snapins. On the dashboard, Storage nodes are shown to be online and Storage Groups report as expected.

Image Replicator Log from the main FOG server (the Primary for all Images) shows 2 sorts of issue in a single pass:
It shows images fail to replicate because the storage nodes are offline (but they aren’t):
[09-07-17 11:23:23 am] * Starting Image Replication.
[09-07-17 11:23:23 am] * We are group ID: 1. We are group name: default
[09-07-17 11:23:23 am] * We are node ID: 1. We are node name: DefaultMember
[09-07-17 11:23:23 am] * Attempting to perform Group -> Group image replication.
[09-07-17 11:23:23 am] | Replicating postdownloadscripts
[09-07-17 11:23:23 am] * Not syncing Image between nodes
[09-07-17 11:23:23 am] | Image Name:
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] | Replicating postinitscripts
[09-07-17 11:23:23 am] * Not syncing Image between nodes
[09-07-17 11:23:23 am] | Image Name:
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] * Found Image to transfer to 3 groups
[09-07-17 11:23:23 am] | Image Name: W10Prox64BIOSSysprep
[09-07-17 11:23:23 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:23 am] sal2fogsnl01 Server does not appear to be online.
[09-07-17 11:23:23 am] * Found Image to transfer to 3 groups
[09-07-17 11:23:23 am] | Image Name: W7ProSp1x32ReamDrivers
[09-07-17 11:23:23 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:23 am] sal2fogsnl01 Server does not appear to be online.
[09-07-17 11:23:23 am] * Found Image to transfer to 3 groups
[09-07-17 11:23:23 am] | Image Name: W7ProSP1x64ReArmDrivers
[09-07-17 11:23:23 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:23 am] sal2fogsnl01 Server does not appear to be online.
.
.
.
Second, it shows some images are not configured to replicate:
[09-07-17 11:23:23 am] | Image Name: Win7ProSP1x64DriversRearm
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] * Attempting to perform Group -> Nodes image replication.
[09-07-17 11:23:23 am] * Not syncing Image between nodes
[09-07-17 11:23:23 am] | Image Name: W10Prox64BIOSSysprep
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] * Not syncing Image between nodes
[09-07-17 11:23:23 am] | Image Name: W7ProSp1x32ReamDrivers
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] * Not syncing Image between nodes
[09-07-17 11:23:23 am] | Image Name: W7ProSP1x64ReArmDrivers
[09-07-17 11:23:23 am] | There are no other members to sync to.
[09-07-17 11:23:23 am] * Not syncing Image between nodes
.
.
.

Note that some of the images are listed twice in one replication pass.

Similarly, the Snapin Replication Log from the Main FOG Server (Primary for all Snapins) shows the same two issues:
First that the Storage Nodes are offline:
[09-07-17 11:23:26 am] * Starting Snapin Replication.
[09-07-17 11:23:26 am] * We are group ID: 1. We are group name: default
[09-07-17 11:23:26 am] * We are node ID: 1. We are node name: DefaultMember
[09-07-17 11:23:26 am] * Attempting to perform Group -> Group snapin replication.
[09-07-17 11:23:26 am] | Replicating ssl less private key
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name:
[09-07-17 11:23:26 am] | There are no other members to sync to.
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name:
[09-07-17 11:23:26 am] | There are no other members to sync to.
[09-07-17 11:23:26 am] * Found Snapin to transfer to 3 groups
[09-07-17 11:23:26 am] | Snapin Name: -DeliverFogExe
[09-07-17 11:23:26 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] sal2fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] * Found Snapin to transfer to 3 groups
[09-07-17 11:23:26 am] | Snapin Name: -ExtendDisk
[09-07-17 11:23:26 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] sal2fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] * Found Snapin to transfer to 3 groups
[09-07-17 11:23:26 am] | Snapin Name: -Timeout
[09-07-17 11:23:26 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] sal2fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] * Found Snapin to transfer to 3 groups
[09-07-17 11:23:26 am] | Snapin Name: 0-AdminSet
[09-07-17 11:23:26 am] roa1fogsnl01 Server does not appear to be online.
[09-07-17 11:23:26 am] sal2fogsnl01 Server does not appear to be online.
.
.
.

and Second that the snapin isn’t configured for replications:
09-07-17 11:23:26 am] * Attempting to perform Group -> Nodes snapin replication.
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name: -DeliverFogExe
[09-07-17 11:23:26 am] | There are no other members to sync to.
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name: -ExtendDisk
[09-07-17 11:23:26 am] | There are no other members to sync to.
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name: -Timeout
[09-07-17 11:23:26 am] | There are no other members to sync to.
[09-07-17 11:23:26 am] * Not syncing Snapin between nodes
[09-07-17 11:23:26 am] | Snapin Name: 0-AdminSet
[09-07-17 11:23:26 am] | There are no other members to sync to.

And, again, some, if not all, Snapins are listed twice in the single log pass.

This all worked in previous versions of the working branch of v1.5.0 at the end of August.

In the current system we have, Images and snapins fail from storage nodes and work from the main FOG server. It appears the only problem is replication. Our next step is to manually copy files around and test deployment to verify the problem is limited to replication alone.

Any idea how to proceed?

Any suggestions would be appreciated.

Thanks,
Jim

@wayne-workman Thanks for the help. Just wanted to post this as Solved. Checkboxes now work in the Web UI.

Jim

@tom-elliott

Thanks Tom. The two fixes worked, so the image protection and manual add of a host are working.

We still cannot delete any snapins associated with any host, so this isn’t Solved for us. Should I post this issue separately in a different section of the forum? Can you give us any idea what to look at?

Thanks for your help,

Jim

@wayne-workman

Agreed on all counts, plus I don’t have images that need deleting at present to test, nor time to create images just to test deletion (at this point). I’ll check this as the opportunity presents itself.

thanks to all.

Jim

@sebastian-roth
I’m ok with closing this.

I’ll do more testing and see what I can see. Of the problems I have with the current Latest version, this is not high on my list - hence my request for advice on how to maintain a large number of storage nodes. I’m thinking we’ll look at a periodic cleanup process.

Thanks,
Jim

@george1421 Thanks very much. Trying this as soon as I can.

Jim

Server

• FOG Version: v1.5.0 RC-8 v16 Working Branch
• OS: CEntOS 7

Description

I’m having a problem with the GUI on the working branch (see version above). The GUI cannot change the Protected checkbox. I’m not seeing any responses to the bug report I posted in the Dev area, so I was hoping to get some SQL help in this area. I need to be able to upload some images that are set as Protected. I believe I need to change some value in a table for each of these to unprotected in order for a machine to be able to upload and replace the image stored in FOG.

Can anyone provide the SQL statements needed to do this, since I can’t do it thru the GUI at present?

All help is much appreciated.

Jim

Update - was able to add Host from PXE boot menu.

Jim

• Additional Possible Issue - Cannot manually add a Host to the system with Create New Host.

Entered all required values, hit Add. Nothing happens. Apache error_log has the following line or similar for each click of the Add button:
[Mon Aug 28 11:16:50.210637 2017] [:error] [pid 12409] [client 10.179.100.156:56393] PHP Fatal error: Call to a member function isValid() on null in /var/www/html/fog/lib/pages/hostmanagementpage.class.php on line 637, referer: http://fogserver/fog/management/index.php?node=host&sub=add

There appears to be no Apache error_log entry for the other 2 possible GUI issues.

Jim

Server

• FOG Version: v1.5.0 RC-8 v16 Working Branch Updated 10am EDT
• OS: CEntOS 7

Description

Was running a previous version of v1.5.0 RC-8 and found some GUI problems. Upgraded to latest, rebooted, problems persisted:

• Cannot Delete Snapins from a Host. Check boxes, hit delete, GUI posts the host was updated but refreshing page (F5 or exiting browser and reconnecting or by logging out and back in) shows all Snapins remain associated with the Host. Adding new Snapins works.

• Display of Protection State in the full Images list displays correctly - those images that are protected show the lock closed, those that aren’t protected show the lock open. In the general page for the image, the Protected checkbox always shows as unchecked, regardless of the actual state. Checking the box and updating causes Image Updated! message but refreshing the page (as above) shows the box to be unchecked. The All Images list show the Protection State exactly as it was prior to going into the General Tab and making the change. I tried changing the Protected box on Images that were protected and images that weren’t and no change could be seen on either the All Images List or in the General tab for the image.

These may be a database problem on my system or they’re GUI problems with FOG, but I don’t see other database consistency issues. I’ve made no changes to my system and have been busy testing Snapins and building images with no problems. I don’t know when I last tried to remove a Snapin association to a Host or to protect or unprotect an image but it could have been 2 weeks ago or more.

Please advise.

Jim

@wayne-workman Thanks…

@wayne-workman

Do I need to post this again under Bug Reports? I don’t see a way to move this to a different section of the forum.

Jim

Server

• FOG Version: 1.5.0-RC-8 v12
• OS: CentOS

Description

I’m working on a deployment that involves images and snapins at 10 sites, each site with it’s own FOG Storage Node. We’ve been working on testing for some time and have created and deleted many snapins, renamed snapins and changed the name for the file that’s uploaded to FOG for snapins.

I’ve known for a while that FOG replication only replicates the file associated with a Snapin, thereby orphaning files if I should rename a file. In previous versions of FOG, deleting a snapin resulted in the deleting of the file associated with the snapin. In 1.5.0, it appears this doesn’t occur, and that nothing is deleted from the snapin folder when a snapin is deleted.

So, what’s the recommended process for cleaning up the snapins folder on the default node and storage nodes?

It’s a pain to manually cross-reference every snapin to verify you can delete a specific file, AND it’s a pain to have to delete these files on all storage nodes (of which I’ll have 9).

Any assistance would be appreciated.

Thanks,

Jim

@tom-elliott

Great, fine and wonderful.

My intention was to make you aware of the issues we’ll encounter as an installation ages or when there are lots of PCs relying on 1 FOG server. It’s clear you get it, so I’ll just wait to see what happens. My history with deployments goes back 22 years (arghhh). My first deployment was 8000 PC in 1994 (NT 3.51). I’ve used various software packages and deployment methods. I’ve needed as many as 4000 PC to be deployed at a single site, and I’ve lived with these systems for years at a time. I’ve focused on hardware independent images from the beginning and typically create two base images for a given OS; one with MS Office and one without. Both have various tools installed (7zip, Acrobat, that sort of thing). I’ve leveraged snapins to arrive at the PC each user actually needs.

I’m guessing from your comments I’m atypical.

To address issues with reports, can you point me to any doc that tells me anything about how to create reports to be imported?

Thanks for your explanations.

Jim