Multiple FOG servers in one network
-
Is it possible to run multiple FOG servers within one network?
There is one DHCP server, so FOG servers would use dnsmasq.
I saw there is a UUID filtering through dhcp-boot parameter in ltsp.conf, could this be used or we have some other option( if any)?
The use case is simple, there are multiple teams with different machines, but connected to one network. -
@AndrewG78 said:
There are several identical broadcast responses.
Can’t explain that without having a full wireshark/tcpdump pcap file. Way too much information is missing to be able to get a glimpse on why this might happen
There is tftpd error - Error code 8: User aborted the transfer
It’s kind of a known thing. Before loading the boot file via TFTP the client requests the file size (via RRQ query command). The server answers the size query and for some weird reason the client sends back a “User aborted the transfer” and then sends a new request to actually download the file.
-
@Sebastian-Roth @george1421
Hi.
After quite a long time, I found free time to test filtering through the mac table configuration file.
It is working!!! Thx for the tips here.
However, I’d like to understand strange dnsmasq logs I noticed in the messages.- There are several identical broadcast responses.
- There is tftpd error - Error code 8: User aborted the transfer
Beside of above, it works
available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 PXE(ens160) <MAC> proxy tags: team1, ens160 next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 PXE(ens160) <MAC> proxy tags: team1, ens160 next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 PXE(ens160) <MAC> proxy tags: team1, ens160 next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 PXE(ens160) <MAC> proxy tags: team1, ens160 next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003016 PXE(ens160) <MAC> proxy tags: team1, ens160 bootfile name: ipxe.efi server name: <IP> next server: <IP> sent size: 1 option: 53 message-type 5 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id in.tftpd[27886]: Error code 8: User aborted the transfer in.tftpd[27887]: Client <machine_IP>finished ipxe.efi available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE PXE(ens160) <MAC> proxy tags: ipxe, team1, ens160 bootfile name: filenotneeded next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE PXE(ens160) <MAC> proxy tags: ipxe, team1, ens160 bootfile name: filenotneeded next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE PXE(ens160) <MAC> proxy tags: ipxe, team1, ens160 bootfile name: filenotneeded next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE PXE(ens160) <MAC> proxy tags: ipxe, team1, ens160 bootfile name: filenotneeded next server: <IP> broadcast response sent size: 1 option: 53 message-type 2 sent size: 4 option: 54 server-identifier <IP> sent size: 9 option: 60 vendor-class 50:58:45:43:6c:69:65:6e:74 sent size: 17 option: 97 client-machine-id available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE available DHCP subnet: <IP>/255.255.255.0 vendor class: PXEClient:Arch:00007:UNDI:003010 user class: iPXE localhost in.tftpd[27997]: Client <machine_IP>finished default.ipxe -
@AndrewG78 said in Multiple FOG servers in one network:
My only concern is, how to tell dnsmasq to read this particular file and how data should be formatted inside this MAC conf file?
May you shed some light on it ?File needs to look like this and if you name it
/etc/dnsmasq.d/amac_table.confyour dnsmasq service will automatically consider those definitions:dhcp-mac=set:team1,F0:DE:F1:EB:02:E0 dhcp-mac=set:team2,F0:DE:F1:EB:02:E1 ... -
@george1421 @Sebastian-Roth
Thank you for your valuable input. My idea is to separate not only the hosts but also snapins/images/literally everything. What I want to achieve is to have dedicated Fog server per team. The process of MAC address separation must be fully automatic.
I will develop the script that will:- Query database for hosts in the 60s loop
- Write MAC list to temp_mac file.
- If there is no amac_table.conf file, write temp_mac file as amac_table.conf
- If amac_table.conf exists, do diff on these files
- If files are same, do nothing
- If files differ, copy temp_mac over the amac_table.conf and restart dnsmasq service.
My only concern is, how to tell dnsmasq to read this particular file and how data should be formatted inside this MAC conf file?
May you shed some light on it ?
Obviously I will share my work here.
-
That way will work well, except host management will be a big PITA. I haven’t tried this but Sebastian’s well defined concept could be extended to make it a bit easier to manage with a bash/mysql script.
Since the dnsmasq configurations are additive I would suggest moving this section out to a new file in /etc/dnsmasq.d directory. If the default dnsmasq config file is called ltsp.conf, create a new one starting with a lower letter like amac_table.conf and add these into that file.
dhcp-mac=set:team1,F0:DE:F1:EB:02:E0 dhcp-mac=set:team2,F0:DE:F1:EB:02:E1Then in FOG add the computers that are part of team 1 to a group in FOG called team1, the computers that are part of team 2 into a FOG group called team2 and so on. Use loops so that you can expand groups and membership numbers.
Then finally create a bash script to build amac_table.conf from the FOG mysql database looping through the groups the last bit of the script will be to restart dnsmasq service.
The is only a suggestion, if you have a small number of groups/hosts then manually managing the groups in dnsmasq would be easier, but if you had more than a handful it would be advantageous to use scripting to manage the lists.
-
@Sebastian-Roth
HI. Thanks for this.
Looks very promising. Will tests this method soon. -
@AndrewG78 Having thought about this for a bit more I think this can be achieved without too much of trouble. I would suggest to not run dnsmasq service on all your FOG servers but have one FOG server designated as master proxyDHCP (dnsmasq). This way you don’t even need iptables to filter the packets. I played with the dnsmasq config a bit and came up with this:
# Don't function as a DNS server: port=0 # Log lots of extra information about DHCP transactions. log-dhcp # Set the root directory for files available via FTP. tftp-root=/tftpboot # Disable re-use of the DHCP servername and filename fields as extra # option space. That's to avoid confusing some old or broken DHCP clients. dhcp-no-override # make dnsmasq act as proxy server dhcp-range=192.168.2.7,proxy # PXE menu. The first part is the text displayed to the user. The second is the timeout, in seconds. pxe-prompt="Booting FOG Client", 1 dhcp-userclass=set:ipxe,iPXE dhcp-match=set:ipxe,175 # The known types are x86PC, PC98, IA64_EFI, Alpha, Arc_x86, # Intel_Lean_Client, IA32_EFI, BC_EFI, Xscale_EFI and X86-64_EFI # This option is first and will be the default if there is no input from the user. dhcp-mac=set:team1,F0:DE:F1:EB:02:E0 dhcp-mac=set:team2,F0:DE:F1:EB:02:E1 # Team 1 pxe-service=net:team1,net:!ipxe,x86PC, "Team 1", undionly.kpxe, 192.168.2.7 pxe-service=net:team1,net:!ipxe,IA64_EFI, "Team 1", ipxe.efi, 192.168.2.7 pxe-service=net:team1,net:!ipxe,IA32_EFI, "Team 1", i386-efi/ipxe.efi, 192.168.2.7 pxe-service=net:team1,net:!ipxe,BC_EFI, "Team 1", ipxe.efi, 192.168.2.7 pxe-service=net:team1,net:!ipxe,Xscale_EFI, "Team 1", ipxe.efi, 192.168.2.7 pxe-service=net:team1,net:!ipxe,X86-64_EFI, "Team 1", ipxe.efi, 192.168.2.7 dhcp-boot=net:team1,net:ipxe,filenotneeded,,192.168.2.7 # Team 2 pxe-service=net:team2,net:!ipxe,x86PC, "Team 2", undionly.kpxe, 192.168.2.4 pxe-service=net:team2,net:!ipxe,IA64_EFI, "Team 2", ipxe.efi, 192.168.2.4 pxe-service=net:team2,net:!ipxe,IA32_EFI, "Team 2", i386-efi/ipxe.efi, 192.168.2.4 pxe-service=net:team2,net:!ipxe,BC_EFI, "Team 2", ipxe.efi, 192.168.2.4 pxe-service=net:team2,net:!ipxe,Xscale_EFI, "Team 2", ipxe.efi, 192.168.2.4 pxe-service=net:team2,net:!ipxe,X86-64_EFI, "Team 2", ipxe.efi, 192.168.2.4 dhcp-boot=net:team2,net:ipxe,filenotneeded,,192.168.2.4You can have as many “team definitions” as you want and can assign clients via MAC address to any one team you want them to be in. The only thing you need to adapt is the IP addresses, search for
192.168.2in the conf file and adjust to your needs.Simply add new hosts to your dnsmasq config and they should perfectly register with the FOG server you teamed it up with.
This is a first proposal. Sure you could generate the
dhcp-mac=definitions from the database. It would also be possible to add more dnsmasq foo to direct unregistered clients to a special PXE menu where you could choose which team it belongs to and send it off to register on a particular FOG team server. Sure it need some modification of code to achieve that but I am sure it can be done.Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@AndrewG78 said in Multiple FOG servers in one network:
I have no access to DHCP server. Company’s policy.
Then request changes formally, in writing. If there’s an existing ticketing system, use that. State what you need, and why you need it. State the benefits. And be polite and respectful.
-
@Sebastian-Roth said in Multiple FOG servers in one network:
Why using dnsmasq anyway? What DHCP server do you have right now that cannot be modified to do PXE boot?
I have no access to DHCP server. Company’s policy.
-
@AndrewG78 said in Multiple FOG servers in one network:
dnsmasq will respond with dhcp offer as it does right now, but only to the machines included on MAC address list.
Simple query to FOG’s mysql will get registered hosts from DB and update the list automatically.
If this is not possible with dnsmasq I thought iptables could be used instead?
Obviously, we will lost the ability to register new machines from the FOG boot menu.Although it’s kind of a hack I kind of like the idea. Personally I would go the iptables route but just because I love it.
You could even get registration working I suppose. Let me think about the whole idea a bit more. Will get back to you soon.
Why using dnsmasq anyway? What DHCP server do you have right now that cannot be modified to do PXE boot?
-
@Wayne-Workman
@george1421
Would this be possible to use dnsmasq as a MAC address filter?
I imagine this kind of setup:- Existing dhcp server in the network
- Fog server 1 with dnsmasq
- Fog server X with dnsmasq
dnsmasq will respond with dhcp offer as it does right now, but only to the machines included on MAC address list.
Simple query to FOG’s mysql will get registered hosts from DB and update the list automatically.
If this is not possible with dnsmasq I thought iptables could be used instead?
Obviously, we will lost the ability to register new machines from the FOG boot menu.
-
@george1421 said in Multiple FOG servers in one network:
You will need a way to map the pxe booting computer to a specific fog server.
ISC-DHCP can also do it. You can define a next-server for a specific MAC address, though doing that at scale would royally suck.
-
@AndrewG78 OK the issue you are going to have is this. If all of the computers are on the same subnet then directing the pxe booting computer to the proper fog server is going to be a problem.
You will need a way to map the pxe booting computer to a specific fog server. Typically this is done by putting the target computers in their own subnet so you can define a boot server for that subnet. You might be able to do this with dnsmasq but you would need a unique identity for each system to define the boot server. The mac address might be usable, but then you will have an administrative issue when trying to map which computer goes to which fog server.
FOG does have a feature for the pc repair shops where you can image a computer using the iPXE menu only. In that when you pxe boot a computer, you can select deploy image from the ipxe menu. From there you can select any images on the fog server for deployment no tasking or gui access needed. To take that one step more, fog has a setting to only show the defined image for that computer on the deploy image screen. This might allow a user the ability to restore their computer directly from the iPXE menu. I’m not saying that is the proper case for your issue, but it might help.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
-
@george1421
Exactly, I just want to avoid the mess between users and their images/snapis/etc.
This is not repair shop deployment type. -
@AndrewG78 I’m still not totally sure of the use case here. Why do you feel you need to have a fog server for each team? Is that because you need to segregate what users access which server, where each server has its own applications that should not cross teams? (I am driving to an answer here, just stick with me).
Are you planning on using fog for “repair shop” type deployments (image it once and never see it again)?
-
We have only single subnet.
As Im not bale to restrict access to specific hosts nor to images/snapins, I wanted to create Jenkins server for each team. -
What is your use case for needing multiple fog servers?
Are the multiple subnets involved here?
