RC10 Broken Items on upgrade
-
@Tom-Elliott mysql> update hosts set hostPubKey=‘’, hostSecToken=‘’, hostSecTime=‘0000-00-00 00:00:00’;
Query OK, 2590 rows affected (0.16 sec)
Rows matched: 6222 Changed: 2590 Warnings: 0 -
-
@adukes40 Now it’s a waiting game. I suspect you have some older new clients who were stuck in a loop trying to update their clients.
-
should I still continue on going up to RC-11?
-
@adukes40 just chill for 10 minutes. Then check top again, and the dashboard again.
-
@Wayne-Workman Ok, well I will go grab some lunch and then check on it.
-
Ok back from lunch. it seems to be hit or miss i fit loads, which is better than it was. Starts to look like the normal pages. Still having timeouts, and slow performance. Checkin while trying to image still takes time. and here is TOP
-
Something noticed. So I updated the servers to RC-10 this morning around 7:30am. Since then we have had three sites fully maxed out on bandwidth. a 10mb site, and 2 100mb sites. Complete 100% full pipe. I just disabled the virtual NIC on the master FOG server, and wala bandwidth dropped majorly.
What can cause this to happen? it also happened last month when I upgraded to RC-7. In that case it start replicated images for what ever reason. But since that time I turned off replication on the images so it would not happen next time. So from 7:30 this morning until 3:00 this afternoon, the outbound traffic of the master site has been maxed out while other sites incoming have been maxed or extremely high, and it was resolved by disabling the NIC.
-
@adukes40 can you check the replication logs?
-
@Tom-Elliott Trying to, but GUI is getting bogged down again. Where on the server itself can I find the logs for that?
-
@adukes40 /var /log forward slash fog
-
@Tom-Elliott in the interim please stop fog image replicator and snapin
-
@Tom-Elliott I stopped them, but I am not seeing anything in the replicator.log, seems pretty empty. And that was from 8:47 this morning. anything with newer timestamps doesn’t make sense or isn’t one of the sites in question.
-
@adukes40 pleSe stop the same services on all nodes?
-
@Tom-Elliott ok give me a few minutes
-
@Tom-Elliott so far I receive this on the nodes:
root@MSDBETS09:~# service FOGImageReplicator stop
- Stopping FOG Computer Imaging Solution: FOGImageReplicator start-stop-daemon: warning: failed to kill 1148: No such process
[ OK ]
root@MSDBETS09:~# service FOGSnapinReplicator stop - Stopping FOG Computer Imaging Solution: FOGSnapinReplicator start-stop-daemon: warning: failed to kill 1179: No such process
[ OK ]
- Stopping FOG Computer Imaging Solution: FOGImageReplicator start-stop-daemon: warning: failed to kill 1148: No such process
-
@adukes40 is the GUI becoming responsive?
-
@Tom-Elliott Stopping them doesnt seem to stop the traffic. All sites are back up to high usage, with master site 100% outgoing traffic pegged. This is after stopping those two services on all nodes
-
@adukes40 then can you get the latest ten lines of the Apache access logs?
-
10.104.12.83 - - [13/Sep/2016:15:45:49 -0400] “-” 408 0 “-” “-”
10.104.12.196 - - [13/Sep/2016:15:47:02 -0400] “-” 408 0 “-” “-”
10.104.12.13 - - [13/Sep/2016:15:49:13 -0400] “-” 408 0 “-” “-”
10.104.12.103 - - [13/Sep/2016:15:49:15 -0400] “-” 408 0 “-” “-”
10.104.12.59 - - [13/Sep/2016:15:50:04 -0400] “-” 408 0 “-” “-”
10.104.12.105 - - [13/Sep/2016:15:50:20 -0400] “-” 408 0 “-” “-”
10.105.13.139 - - [13/Sep/2016:15:51:16 -0400] “-” 408 0 “-” “-”
10.104.12.20 - - [13/Sep/2016:15:55:27 -0400] “-” 408 0 “-” “-”
10.102.12.74 - - [13/Sep/2016:15:57:08 -0400] “-” 408 0 “-” “-”
10.104.12.96 - - [13/Sep/2016:15:57:12 -0400] “-” 408 0 “-” “-”