Multicast works when in the same ip segment, but doesn't work when it passes the pfsense firewall/router to an other ip segment.

[jow_jow_K]

Server

• FOG Version: 1.3.4
• OS: Ubuntu 16.04 LTS

Description

When I connect 2 clients on the same ip segment multicasting works. But when the multicast passes the pfsense firewall (all trafic in both directions is allowed) the multicast stays at “starting to restore image”.

[Tom Elliott]

@jow_jow_K multicast is typically a layer 2 thing. You would need to use a rendevous point to multicast if segments of networks are split up. The rendevous point would be a common point the server sends traffic to and clients request traffic from. This can be configured in the fog settings -> multicast settings

[george1421]

This is a little bit out of my wheel house. But for multicasting you need to have a multicast router to forward the mutlicast data packets across your network segments. This is a bit different than a normal router. It does a similar function but its different. The multicast router can either exist on your main router (as a service like mrouted) or on a different device dedicated to multicast routing.

Lucky for you that you are using a pfsense router. That has a built in igmp proxy service (a light weigh multicast router) that you can enable. If I remember correctly you have to define a multicast listening interface and then the multicast broadcast interfaces. I think the listening one is an upstream interface and the broadcast interfaces are the downstream interfaces. You will also need to allow this multicast address across your router.

Tom should be able to tell you what multicast address is used for FOG. (I don’t know off the top of my head).

[jow_jow_K]

I started testing this but now I get a different problem: I get “read image_hdr block_size error”. I looked it up and found this: https://forums.fogproject.org/topic/8743/can-not-deploy-using-multicast-read-image_hdr-block_size-error . I haven’t completely tried this but my firewall is on pass all to this network. Or do they mean software firewall?

I’ll test a bit more this evening.

Thanks for the help so far!

[jow_jow_K]

I set this in the pfsense igmp proxy:

And these setting in the fog setting tab:

The ip of the fog server is 10.1.0.35
And the ip of the pfsense in the server subnet is 10.1.0.254

10.1.2.254 and 10.1.3.254 are both subnets where the clients are located.

As soon as I change the multicast address in the fog setting to an ip it gives me “read image_hdr block_size error”.

Am I configuring something wrong?

[george1421]

@jow_jow_K To answer your question, I would think so. But also I’m questioning the default values for this too (as comparing it to my dev box where i haven’t touched the multicast settings). We may have to get some feedback from the @Senior-Developers as to the code defaults.

If it was me I would have the FOG_MULTICAST_ADDRESS set to the default multicast IP address (channel). Which if memory serves me right should be 224.0.0.1 with the FOG_MULTICAST_PORT_OVERRIDE set to some unique value. I would think that the defaults for FOG should be provided here by the application and not be blank or 0. I might also think the description for the field should have the default values listed in case someone mucks about changing the values. Knowing the defaults would be helpful.

So in regards to your settings. I would change the FOG_MULTICAST_ADDRESS to an IP address valid in the multicast range of 224.0.0.0 to 239.0.0.0. The FOG_MULTICAST_PORT_OVERRIDE should be blank or set to a random port of your choosing. I would go with blank for now.

Can we assume that fog nic eno2 is connected to the 10.1.0.x/24 subnet? Because fog will send out the multicast stream using this interface. You need pfsense to listen on its interface connected to the same subnet as fog is sending on. This is the upstream interface. pfSense will then rebroadcast the multicast stream on its downstream interfaces. Now with the switches on vlans for 10.1.2.x and 10.1.3.x if you don’t enable igmp snooping on those vlans the mutlicast stream will be sent to all network ports (akin to pim dense mode). If igmp snooping is enabled then the stream will only be sent to ports that request the stream (akin to pim sparse mode)

With that said, what is not clear to me is how the target computers ‘check in’ with the fog server. Do they do a direct unicast back to the fog server or do they ‘chat’ over the multicast channel. If they do, then pfsense may not allow a data stream back through the igmp proxy. This part I can only guess since I haven’t used the igmp proxy in pfsense before.

[jow_jow_K]

@george1421 do you tink there is any chance of multicast working if I add a storage node to every subnet?

[george1421]

@jow_jow_K I don’t have a solid answer for you. Let me dig into the multicasting code to see if I can glean some intelligence. Adding a storage node to each subnet will just add overhead without much help.

[Fernando Gietz]

I have the same problem with multicast wheen I send a task between different subnet.

I use RedHat 7.3 in my test server and with the 1.3.4 FOG version the multicasts worked fine, but I update it to 1.4.0 and surprise… The multicast doesn’t work.

I solved it changing the init.xz of 1.4.0 version with the init.xz of 1.3.4 version.

[shosta3]

@Fernando-Gietz

@Fernando-Gietz said in Multicast works when in the same ip segment, but doesn't work when it passes the pfsense firewall/router to an other ip segment.:

I have the same problem with multicast wheen I send a task between different subnet.

I use RedHat 7.3 in my test server and with the 1.3.4 FOG version the multicasts worked fine, but I update it to 1.4.0 and surprise… The multicast doesn’t work.

I solved it changing the init.xz of 1.4.0 version with the init.xz of 1.3.4 version.

It is worth mentioning this workaround only works if you use the partclone gzip compression method. Any image uploaded using partclone zstd will not be able to be deployed using the init from 1.3.4.

[george1421]

FWIW: There was a bug in the 1.4.0 inits that has been patched in the working release of FOG 1.4.1RC1. At this point 1.4.1RC1 hasn’t been released nor has a release date been set. Because it is an annoying bug that limits multicasting across subnets the fix has also been applied to the 1.4.0 (stable) code. So for those that are still having this issue on 1.4.0, rerun the 1.4.0 (stable) installer to get the patched inits (Note you must also purge some cached files before running the installer. From bin folder run rm ../binaries1.4.0.zip Then run the installer.) From what I understand no other changes have been made to 1.4.0 (stable).

[nx.vijay]

@george1421 said in Multicast works when in the same ip segment, but doesn't work when it passes the pfsense firewall/router to an other ip segment.:

Tom should be able to tell

I have setup my FOG server on Google Cloud. My Clients are different location of the world. I wanted to image them through my Fog server which is on GCP.

Can you please let me know what setup should be. I have pfSense firewall in my office.

[george1421]

@nx-vijay how many routers are between your fog server and your client computers? If its more than the one pfsense server you have in your office then you will need to enable igmp routing on all routers in question.

But then I would have to also question if you really need to multicast (send the same image to each computer at the same time)? Or would unicasting work just as well. Remember with multicasting the slowest computer (link) in the multicast group sets the speed for the entire group. So if you have one site behind a 1Mb DSL connection and one at HQ on 10GbE that computer behind the 1Mb will set the imaging speed for everyone.