Multi-Task Problem
-
@Fernando-Gietz
@Sebastian-RothWell the problem persists on the new server… Multicast.log shows no new tasks and zero multicast.log.udpcstxxxx files in the /opt/fog/log directory. I’m betting the database issues persist on. I’ll be in first thing tomorrow morning. (7AM MST)
Cheers!
-
@Sebastian-Roth
@Fernando-GietzHere is the latest…
[jgill@fogserver log]$ ls -lar total 80 -rw-r--r--. 1 root root 4056 Jun 12 09:22 servicemaster.log -rw-r--r--. 1 root root 1433 Jun 12 09:31 pinghost.log -rw-r--r--. 1 root root 3471 Jun 12 09:31 multicast.log -rw-r--r--. 1 root root 28408 Jun 12 09:22 groupmanager.log -rw-r--r--. 1 root root 7770 Jun 12 09:22 fogsnapinrep.log -rw-r--r--. 1 root root 4559 Jun 12 09:22 fogsnapinhash.log -rw-r--r--. 1 root root 1812 Jun 12 09:30 fogscheduler.log -rw-r--r--. 1 root root 8714 Jun 12 09:22 fogreplicator.log -rw-r--r--. 1 root root 4462 Jun 12 09:22 fogimagesize.log drwxr-xr-x. 5 root root 67 Jun 12 09:23 .. drwxr-xr-x. 2 root root 218 Jun 12 09:22 . [jgill@fogserver log]$06-12-19 9:22:41 am] ================================== === ==== ===== ==== === ========= == === == === === ======== ==== == ==== === === ======== ==== == ========= === ==== ==== == ========= === ======== ==== == === === === ======== ==== == ==== === === ========= == === == === === ========== ===== ==== ================================== ===== Free Opensource Ghost ====== ================================== ============ Credits ============= = https://fogproject.org/Credits = ================================== == Released under GPL Version 3 == ================================== [06-12-19 9:22:41 am] Interface Ready with IP Address: 127.0.0.1 [06-12-19 9:22:41 am] Interface Ready with IP Address: 127.0.1.1 [06-12-19 9:22:41 am] Interface Ready with IP Address: 172.16.1.19 [06-12-19 9:22:41 am] Interface Ready with IP Address: fogserver.bhs.tn [06-12-19 9:22:41 am] * Starting MulticastManager Service [06-12-19 9:22:41 am] * Checking for new items every 10 seconds [06-12-19 9:22:41 am] * Starting service loop [06-12-19 9:22:41 am] * No new tasks found [06-12-19 9:22:51 am] * No new tasks found [06-12-19 9:23:01 am] * No new tasks found [06-12-19 9:23:11 am] * No new tasks found [06-12-19 9:23:21 am] * No new tasks found [06-12-19 9:23:31 am] * No new tasks found [06-12-19 9:23:41 am] * No new tasks found [06-12-19 9:23:51 am] * No new tasks found [06-12-19 9:24:01 am] * No new tasks found [06-12-19 9:24:11 am] * No new tasks found [06-12-19 9:24:21 am] * No new tasks found [06-12-19 9:24:31 am] * No new tasks found [06-12-19 9:24:42 am] * No new tasks found [06-12-19 9:24:52 am] * No new tasks found [06-12-19 9:25:02 am] * No new tasks found [06-12-19 9:25:12 am] * No new tasks found [06-12-19 9:25:22 am] * No new tasks foundNothing is showing up in the logs at all.
And from the web interface…
-
And in the database appears?
See the multicastSessions and MulticastSessionAssocIf you go to the Active Tasks page, the multicast tasks appears?
-
MariaDB [fog]> SELECT * FROM multicastSessions; Empty set (0.00 sec) MariaDB [fog]> SELECT * FROM multicastAssoc; ERROR 1146 (42S02): Table 'fog.multicastAssoc' doesn't exist MariaDB [fog]> -
-
@Joe-Gill First make sure the multicast service is actually running:
systemctl status FOGMulticastManagerYou are on the right track here. Try this:
SELECT * FROM multicastSessionsAssoc; -
[jgill@fogserver log]$ systemctl status FOGMulticastManager ● FOGMulticastManager.service - FOGMulticastManager Loaded: loaded (/usr/lib/systemd/system/FOGMulticastManager.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2019-06-12 09:22:41 MDT; 1h 31min ago Main PID: 5610 (FOGMulticastMan) CGroup: /system.slice/FOGMulticastManager.service ├─5610 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager & └─5613 /usr/bin/php -q /opt/fog/service/FOGMulticastManager/FOGMulticastManager & Jun 12 09:22:41 fogserver.bhs.tn systemd[1]: Started FOGMulticastManager. [jgill@fogserver log]$ -
MariaDB [fog]> SELECT * FROM multicastSessionsAssoc; +-------+------+-----+ | msaID | msID | tID | +-------+------+-----+ | 57 | 0 | 87 | | 56 | 0 | 86 | | 55 | 0 | 85 | | 54 | 0 | 84 | | 53 | 0 | 83 | | 52 | 0 | 82 | | 51 | 0 | 81 | | 50 | 0 | 80 | | 49 | 0 | 79 | | 48 | 0 | 78 | | 47 | 0 | 77 | | 46 | 0 | 76 | | 45 | 0 | 75 | | 44 | 0 | 74 | | 43 | 0 | 73 | | 42 | 0 | 72 | | 41 | 0 | 71 | | 40 | 0 | 70 | | 39 | 0 | 69 | | 38 | 0 | 68 | | 37 | 0 | 67 | | 36 | 0 | 66 | | 35 | 0 | 65 | | 34 | 0 | 64 | | 33 | 0 | 63 | | 32 | 0 | 62 | | 31 | 0 | 61 | | 30 | 0 | 60 | +-------+------+-----+ 28 rows in set (0.00 sec) MariaDB [fog]> -
The problem is that the multicastSession is not created ( I am very clever boy XD) but we dont know the reason of this behaviour.
Any errors in the php log?
-
Which files would you like me to pull for you?
-
- /var/log/httpd/ssl_error_log
- /var/log/httpd/error_log
Try to see if when tou create a multicast tasks appears any error.
-
Will do. I’ll tail those files and see what happens when I create a new task. I’ll report back! Thanks!
-
OK. So now it’s saying:
“Invalid MultiCast Session” on the hosts. Nothing new in the log. I have the /etc/httpd/logs/error_log pulled tail -f and nothing new is showing up there.
-
@Joe-Gill Unfortunately don’t have enough time to do another debug session with you. But maybe I can give you enough hints so you can dig it up yourself.
Remember that I enabled mysql query logging yesterday?! First you login to the web UI, clear all the old tasks there and select the group you want to schedule a multicast task for. Prepare this as far as possible but don’t hit the button to create it yet but enable database query logging (and clear the log we created yesterday):
shell> echo -n "" >/var/log/mariadb/query.log shell> mysql -u root -p Password: ... mysql> SET GLOBAL general_log_file='/var/log/mariadb/query.log'; ... mysql> SET GLOBAL general_log=1;Now click create multicast task and then stop query logging again:
mysql> SET GLOBAL general_log=0;Now take a look at the query.log file. What you are looking for is probably something like
INSERT INTO multicastSessions .... Just use search function of the text editor to find that part. Now when you find this you just copy the whole SQL command and try to execute that in the mysql shell manually. Most probably you’ll get an error from that and we might find out why. -
Awesome! That gives me a place to start. Thanks! I’ll do some digging. I’ll be tied up here in a couple hours for the rest of the day. I’ll post what I find out before I head out though. Thanks again!
-
@Sebastian-Roth
@Fernando-GietzHere’s an error!
MariaDB [fog]> INSERT INTO `multicastSessions` (`msName`,`msBasePort`,`msLogPath`,`msImage`,`msClients`,`msSessClients`,`msInterface`,`msStartDateTime`,`msPercent`,`msState`,`msCompleteDateTime`,`msIsDD`,`msNFSGroupID`,`msAnon3`,`msAnon4`,`msAnon5`) VALUES ('Multi-Cast Task - Elem-Lab','62250','WIN10Base_SSD','40','','','eth0','2019-06-12 15:05:26','0','0','','1','3','','','') ON DUPLICATE KEY UPDATE `msName`=VALUES(`msName`),`msBasePort`=VALUES(`msBasePort`),`msLogPath`=VALUES(`msLogPath`),`msImage`=VALUES(`msImage`),`msClients`=VALUES(`msClients`),`msSessClients`=VALUES(`msSessClients`),`msInterface`=VALUES(`msInterface`),`msStartDateTime`=VALUES(`msStartDateTime`),`msPercent`=VALUES(`msPercent`),`msState`=VALUES(`msState`),`msCompleteDateTime`=VALUES(`msCompleteDateTime`),`msIsDD`=VALUES(`msIsDD`),`msNFSGroupID`=VALUES(`msNFSGroupID`),`msAnon3`=VALUES(`msAnon3`),`msAnon4`=VALUES(`msAnon4`),`msAnon5`=VALUES(`msAnon5`); ERROR 1054 (42S22): Unknown column 'msAnon3' in 'field list' MariaDB [fog]>Any ideas? I’ll be around all day today if you guys want to take a look at it.
Thanks!
-
Hi @Joe-Gill
And the msAnon3 column, does exist in the database?#desc multicastSessions; -
@Joe-Gill Sounds like your DB has more issues that what we have fixed last time. But this one sounds really strange! Please run
DESC multicastSessionsin the mysql command line and post output here. -
MariaDB [fog]> desc multicastSessions; +--------------------+---------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------------+---------------+------+-----+---------+----------------+ | msID | int(11) | NO | PRI | NULL | auto_increment | | msName | varchar(250) | NO | | NULL | | | msBasePort | int(11) | NO | | NULL | | | msLogPath | longtext | NO | | NULL | | | msImage | longtext | NO | | NULL | | | msClients | int(11) | NO | | NULL | | | msSessClients | int(11) | NO | | NULL | | | msInterface | varchar(250) | NO | | NULL | | | msStartDateTime | datetime | NO | | NULL | | | msPercent | int(11) | NO | | NULL | | | msState | int(11) | NO | | NULL | | | msCompleteDateTime | datetime | NO | | NULL | | | msIsDD | int(11) | NO | | NULL | | | msNFSGroupID | int(11) | NO | MUL | NULL | | | msShutdown | enum('0','1') | NO | | 0 | | | msMaxwait | int(11) | NO | | NULL | | | msAnon5 | varchar(250) | NO | | NULL | | +--------------------+---------------+------+-----+---------+----------------+ 17 rows in set (0.01 sec) MariaDB [fog]> -
Your database is … strange!!! XD
MariaDB [fog]> desc multicastSessions; +--------------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------------+--------------+------+-----+---------+----------------+ | msID | int(11) | NO | PRI | NULL | auto_increment | | msName | varchar(250) | NO | | NULL | | | msBasePort | int(11) | NO | | NULL | | | msLogPath | longtext | NO | | NULL | | | msImage | longtext | NO | | NULL | | | msClients | int(11) | NO | | NULL | | | msSessClients | int(11) | NO | | NULL | | | msInterface | varchar(250) | NO | | NULL | | | msStartDateTime | datetime | NO | | NULL | | | msPercent | int(11) | NO | | NULL | | | msState | int(11) | NO | | NULL | | | msCompleteDateTime | datetime | NO | | NULL | | | msIsDD | int(11) | NO | | NULL | | | msNFSGroupID | int(11) | NO | MUL | NULL | | | msAnon3 | varchar(250) | NO | | NULL | | | msAnon4 | varchar(250) | NO | | NULL | | | msAnon5 | varchar(250) | NO | | NULL | | +--------------------+--------------+------+-----+---------+----------------+ 17 rows in set (0.012 sec)
