Topics created by george1421

@Tom-Elliott Good thoughts about not focusing on bash scripts. My thoughts on this… At work, if you can point your peers to documentation saying “this project only supports this distribution” Generally your peers accept it. It’s been a topic that has come up before: Dropping installer support down to one or two distributions. CentOS and/or Debian. This probably deserves another forums topic. At this point, I think supporting one distribution is best. I don’t care which one it is, though Debian probably has the best shot at longevity. I fear CentOS will slowly become irrelevant to many as Red Hat focuses on supporting IBM (their parent company), giving less focus to everything else.

There is a nice article on how to setup NFSv4 with a single port open: https://peteris.rocks/blog/nfs4-single-port/

@Tom-Elliott said in Feature request for FOG 1.6.x - FOG Installer instll DNSMASQ: I’m on board for this as well, though wouldn’t mind some help in implementing. Sure, I will! There is no rush yet. I am working on making the move from 1.5.x to 1.6.x first and then we’ll look into all the feature requests opened lately.

Let me clarify, it turns off the limitation preventing the update to move all tables to innodb.

Fog 1.6 already does this.

@Wayne-Workman said in Feature request for FOG 1.6.x - Scheduled database maintenance: I should add I’m not talking about history. I’m referring to the issues that literally break fog. This stuff: Definitely good you phrase this more explicitely!! We should discuss those two things separately (maybe not in different topics though). While we tell people to sometimes try a cleanup of the DB I am not sure how much issues in the code are still causing this or if this is related to people coming older versions and have not done a cleanup in a long time. Don’t get me wrong, I am not saying the code is perfectly fine, I just don’t know as we don’t have enough evidence that this happens all the time. We would expect way more people to ask about this in the forums. On the other hand I do agree that running those cleanup (not the history ones!) once a week or even daily wouldn’t hurt: DELETE FROM `hosts` WHERE `hostID` = '0'; DELETE FROM `hostMAC` WHERE hmID = '0' OR `hmHostID` = '0'; DELETE FROM `groupMembers` WHERE `gmID` = '0' OR `gmHostID` = '0' OR `gmGroupID` = '0'; DELETE FROM `snapinGroupAssoc` WHERE `sgaID` = '0' OR `sgaSnapinID` = '0' OR `sgaStorageGroupID` = '0'; DELETE FROM `snapinAssoc` WHERE `saID` = '0' OR `saHostID` = '0' OR `saSnapinID` = '0'; DELETE FROM `hosts` WHERE `hostID` NOT IN (SELECT `hmHostID` FROM `hostMAC` WHERE `hmPrimary` = '1'); DELETE FROM `hosts` WHERE `hostID` NOT IN (SELECT `hmHostID` FROM `hostMAC`); DELETE FROM `hostMAC` WHERE `hmhostID` NOT IN (SELECT `hostID` FROM `hosts`); DELETE FROM `snapinAssoc` WHERE `saHostID` NOT IN (SELECT `hostID` FROM `hosts`); DELETE FROM `groupMembers` WHERE `gmHostID` NOT IN (SELECT `hostID` FROM `hosts`); You don’t want to run those on a regular basis I find because it can kill running tasks/sessions if those were setup to run over the weekend for example (state ID see here) DELETE FROM `tasks` WHERE `taskStateID` IN ("1","2","3"); DELETE FROM `snapinTasks` WHERE `stState` in ("1","2","3"); TRUNCATE TABLE multicastSessions; TRUNCATE TABLE multicastSessionsAssoc; DELETE FROM tasks WHERE taskTypeId=8; And history is another story altogether… TRUNCATE TABLE history; TRUNCATE TABLE userTracking;

The API and UI could be separated while still using the same port. The virtualhost config would look something like this: <VirtualHost *:80> ServerName dev.localhost DocumentRoot /home/projects/smk/cms ErrorLog /var/log/apache2/smk-cms-error.log </VirtualHost> <VirtualHost *:80> ServerName my-project.localhost DocumentRoot /home/projects/smk/deploy ErrorLog /var/log/apache2/smk-deploy-error.log </VirtualHost> Reference: https://stackoverflow.com/questions/6069892/different-virtualhosts-with-the-same-port If this separation were done, I would think the installer could ask you if you want to install the API (defaulting to yes) and ask if you want to install the UI (defaulting to yes). This would allow the admin to break stuff apart. I also think the database portion should be broken out, and the installer should ask if you want to install that or not (defaulting to yes). There should be installation arguments for these things too. Also, not wanting to delay 1.6 as noted by others. I would like to see this separation in a later release. As George has noted, if these things are separated it becomes easier to work on each one independently, meaning the learning curve to contribute is lower. Particularly with using popular frameworks. Just my 2 cents.

@george1421 We use the certificate store. Though this is something else I was hoping to change in the future because Mono on Linux and Mac OS X have a long history of issues with the certificate store and we might think about keeping the certs just in files on the disk. I have not had the time to think this through. Might be a dead road…

Testing systems Dell o7010 both fog server and client computer. Both systems have local ssd sata drives. The target computer is running a customized linux kernel 5.6.18 and a customized init but both as based on FOG 1.5.9. The customization was done to aid in debugging and bench-marking the systems. Testing script mkdir /mnt/locdsk mount /dev/sda1 /mnt/locdsk mkdir /images mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime "192.168.10.1:/images/dev" /images #Test 1 creation of local and remote file by target computer time dd if=/dev/zero of=/mnt/locdsk/L10gb.img count=1024 bs=10485760 time dd if=/dev/zero of=/images/R10gb.img count=1024 bs=10485760 #Test 2 cp files to and from server time cp /mnt/locdsk/L10gb.img /images time cp /mnt/locdsk/L10gb.img /images/L10gb-1.img time cp /images/R10gb.img /mnt/locdsk time cp /images/R10gb.img /mnt/locdsk/R10gb-1.img #Test 3 scp files to and from server time scp /mnt/locdsk/L10gb.img root@192.168.10.1:/images/L10gb-2.img time scp /mnt/locdsk/L10gb.img root@192.168.10.1:/images/L10gb-3.img time scp root@192.168.10.1:/images/dev/R10gb.img /mnt/locdsk/R10gb-2.img time scp root@192.168.10.1:/images/dev/R10gb.img /mnt/locdsk/R10gb-3.img #Test 4 ssh pipeline to and from server time cat /mnt/locdsk/L10gb.img | ssh root@192.168.10.1 "cat > /images/L10gb-4.img" time cat /mnt/locdsk/L10gb.img | ssh root@192.168.10.1 "cat > /images/L10gb-5.img" time ssh root@192.168.10.1 "cat /images/dev/R10gb.img" | cat > /mnt/locdsk/L10gb-6.img time ssh root@192.168.10.1 "cat /images/dev/R10gb.img" | cat > /mnt/locdsk/L10gb-7.img Testing results as captured. ## Building the test files both local and remote # time dd if=/dev/zero of=/mnt/locdsk/L10gb.img count=1024 bs=10485760 10737418240 bytes (11 GB, 10 GiB) copied, 20.2216 s, 531 MB/s **real 0m20.223s user 0m0.001s sys 0m6.460s # time dd if=/dev/zero of=/images/R10gb.img count=1024 bs=10485760 10737418240 bytes (11 GB, 10 GiB) copied, 93.3867 s, 115 MB/s **real 1m33.390s user 0m0.003s sys 0m5.369s ## Confirm that files exist and are properly sized # ls -la /mnt/locdsk/ total 10485785 drwxr-xr-x 3 root root 4096 Oct 9 08:25 . drwxr-xr-x 3 root root 1024 Oct 9 08:23 .. -rw-r--r-- 1 root root 10737418240 Oct 9 08:26 L10gb.img drwx------ 2 root root 16384 Jan 10 2013 lost+found # ls -la /images/ total 10519109 drwxrwxrwx 3 sshd root 63 Oct 9 2020 . drwxr-xr-x 19 root root 1024 Oct 9 08:23 .. -rwxrwxrwx 1 sshd root 0 Sep 28 13:36 .mntcheck -rw-r--r-- 1 root root 10737418240 Oct 9 2020 R10gb.img drwxrwxrwx 2 sshd root 26 Sep 28 13:36 postinitscripts ### Copy Local to Remote ### # time cp /mnt/locdsk/L10gb.img /images ** real 1m34.821s user 0m0.083s sys 0m7.314s # time cp /mnt/locdsk/L10gb.img /images/L10gb-1.img **real 1m34.759s user 0m0.046s sys 0m6.801s ### Copy Remote to Local ### # time cp /images/R10gb.img /mnt/locdsk **real 1m41.710s user 0m0.084s sys 0m11.327s # time cp /images/R10gb.img /mnt/locdsk/R10gb-1.img **real 1m41.520s user 0m0.095s sys 0m11.392s ### SCP Local to Remote ### # time scp /mnt/locdsk/L10gb.img root@192.168.10.1:/images/L10gb-2.img The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established. ECDSA key fingerprint is SHA256:OpIsFYWVDCr/ovMlmPPSl46jpT332P3+BHnchdxzTCI. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts. root@192.168.10.1's password: L10gb.img 100% 10GB 110.0MB/s 01:33 **real 1m40.007s user 0m44.460s sys 0m13.378s # time scp /mnt/locdsk/L10gb.img root@192.168.10.1:/images/L10gb-3.img root@192.168.10.1's password: L10gb.img 100% 10GB 109.5MB/s 01:33 **real 1m37.404s user 0m44.420s sys 0m13.068s ### SCP Remote to Local ### # time scp root@192.168.10.1:/images/dev/R10gb.img /mnt/locdsk/R10gb-2.img root@192.168.10.1's password: R10gb.img 100% 10GB 101.9MB/s 01:40 **real 1m44.166s user 0m43.986s sys 0m22.887s # time scp root@192.168.10.1:/images/dev/R10gb.img /mnt/locdsk/R10gb-3.img root@192.168.10.1's password: R10gb.img 100% 10GB 102.0MB/s 01:40 **real 1m44.620s user 0m43.437s sys 0m23.061s ### SSH Pipeline Local to Remote ### # time cat /mnt/locdsk/L10gb.img | ssh root@192.168.10.1 "cat > /images/L10gb-4.img" root@192.168.10.1's password: **real 1m35.562s user 0m42.701s sys 0m12.975s # time cat /mnt/locdsk/L10gb.img | ssh root@192.168.10.1 "cat > /images/L10gb-5.img" root@192.168.10.1's password: **real 1m35.749s user 0m43.478s sys 0m11.166s ### SSH Pipeline Remote to Local ### # time ssh root@192.168.10.1 "cat /images/dev/R10gb.img" | cat > /mnt/locdsk/L10gb-6.img root@192.168.10.1's password: **real 1m43.745s user 0m44.738s sys 0m20.828s # time ssh root@192.168.10.1 "cat /images/dev/R10gb.img" | cat > /mnt/locdsk/L10gb-7.img root@192.168.10.1's password: **real 1m43.564s user 0m43.976s sys 0m21.966s

@Wayne-Workman The replication services provided by FOG use their very own logic implemented in PHP to sync files via FTP protocol. I vote for getting rid of this! I would hope we can tunnel rsync through SSH to not have to open up another port in the firewall. Together with this I would want to get rid of FTP altogether by using SSH for the other places in the code where FTP is used at the moment (move images directory after upload and kernel after download).

Works like a charm! Thanks george1421!