Web GUI, Fog login problem
-
Perhaps we can be given the option to change the timeout length?
Is there a PHP edit we can do to change it? Maybe the FOG web GUI can pull this value from a file, instead of it being hard-coded into the PHP ?
-
[quote=“Wayne Workman, post: 43326, member: 28155”]Perhaps we can be given the option to change the timeout length?
Is there a PHP edit we can do to change it? Maybe the FOG web GUI can pull this value from a file, instead of it being hard-coded into the PHP ?[/quote]
You can change the socket timeout, but it is still going to eat performance with more users. Like I said before, it’s easy to cause a DoS by requesting that Ajax call repeatedly. This could also happen if you have three people on the website at the same time. Any time a user visits the login or views other pages that leverage fetchURL() like the configuration settings page, there is going to be a server freeze.
Considering that the only usage of fetchURL is for getting latest version or server count data, that could be handled outside of the web server. I’ll even write a service script to help with this.
-
[quote=“cspence, post: 43329, member: 28749”]You can change the socket timeout, but it is still going to eat performance with more users. Like I said before, it’s easy to cause a DoS by requesting that Ajax call repeatedly. This could also happen if you have three people on the website at the same time. Any time a user visits the login or views other pages that leverage fetchURL() like the configuration settings page, there is going to be a server freeze.
Considering that the only usage of fetchURL is for getting latest version or server count data, that could be handled outside of the web server. I’ll even write a service script to help with this.[/quote]
On a serious note, how likely is a DoS to happen from within your own network? Nobody is going to build a FOG server inside a de-militarized zone, with a public IP address…
FOG is for internal use.
-
[quote=“Wayne Workman, post: 43330, member: 28155”]On a serious note, how likely is a DoS to happen from within your own network? Nobody is going to build a FOG server inside a de-militarized zone, with a public IP address…
FOG is for internal use. ;-)[/quote]
Unless you’re running an air-gapped hacking proving ground for Information Assurance (Cyber Security) students at a University where you need to deploy fresh machine images in short amounts of time. Then it’s definitely a threat.
In that case, FOG is not for internal use. It’s for anyone who needs a copy of a machine image deployed.
-
Forgive my ignorance I suppose, but how, exactly, is fread a blocking function? I’ve explicitly set the file descriptor to be done in a non-blocking mode. This means, if there’s data to return, it returns it, if not it returns none at the time it’s being read.
-
No problem. I’ll just paste a bit of the documentation on that function here:
[COLOR=#0000ff][B]fread()[/B] reads up to length bytes from the file pointer referenced by handle. Reading stops as soon as one of the following conditions is met: [/COLOR]
[LIST]
[][COLOR=#0000ff] length bytes have been read [/COLOR]
[][COLOR=#0000ff] EOF (end of file) is reached [/COLOR]
[][COLOR=#0000ff] a packet becomes available or the [URL=‘http://php.net/manual/en/function.socket-set-timeout.php’][COLOR=#0000ff] socket timeout[/COLOR][/URL] occurs (for network streams) [/COLOR]
[][COLOR=#0000ff] if the stream is read buffered and it does not represent a plain file, at most one read of up to a number of bytes equal to the chunk size (usually 8192) is made; depending on the previously buffered data, the size of the returned data may be larger than the chunk size. [/COLOR]
[/LIST]
So let’s walk through the bullets:- If the file descriptor is waiting for input, it will definitely not reach the length.
- There is no EOF because the socket is still waiting for input.
- The socket will ultimately fail out because it can’t contact the server. In the meantime, the command still waits.
- I’m not exactly sure how this behaves with a network socket, but it is still waiting for buffered data to arrive.
One more thing to think about when trying to handle this process: If you were able to make fread() non-blocking (or multi-threaded), what affect does that have? Instant failure to grab any input. Additionally, since the function getting the data is not reading from readily available source (like a database entry), it has to wait or fail. This is why you need to have something like a service that does this outside of the server. The service can update the database or a file that the webserver could check. Instead of waiting for something that is not there at the instant it checks, it will always be there or not be there (EOF).
The fopen() function can be non-blocking because it can be redirected somewhere like a log or database entry. But be careful, this doesn’t mean you should redirect this to a database each time there is a call to fetchURL(). You will get data before it is overwritten by the latest fetchURL() call. In other words, stuff will wack out. If a lock is placed on that database entry while trying to write to it, you’ll have another source of I/O blocking. This is yet another reason to just set up a service that will only update periodically and not every time you visit pages.
If I made a mistake somewhere, feel free to point it out.
-
I’ll step through my knowledge, though I’m sure I am more likely wrong, but according to PHP’s own documentation:
[url]http://php.net/manual/en/function.stream-set-blocking.php[/url]
The first few lines:
[quote]mode
If mode is 0, the given stream will be switched to non-blocking mode, and if 1, it will be switched to blocking mode. This affects calls like fgets() and fread() that read from the stream. In non-blocking mode an fgets() call will always return right away while in blocking mode it will wait for data to become available on the stream.[/quote]The “mode” is to set the stream that’s being read directly into non-blocking.
Add to this that we’re using an AJAX call to even perform the fetchURL phase, and now a completely non-blocking stream that simply checks for data, this means it should be entirely nonblocking. For further test, I’ve even added the ajax call a specific async: true, call to ensure that if for whatever reason it can’t make it to the local ajax caller, it should return false. The ONLY think I believe could be potentially causing the “block” at this point would be the foreach loop at this point.
So maybe I’m thinking wrong, but when the only documentation I have to work off of states that something is non-blocking, I haven’t much choice but to entrust the official sources. I’m more on the line that the fread isn’t our blocking point there, but rather the for each loop itself.
I can see if removing the loop is better, but still don’t know EXACTLY what’s causing the block itself. It just seems odd that now that (from my perspective) we’re using ajax and an explicitly set non-blocking stream, that it’s blocking at all.[/quote]
-
It’s definitely the server-side though. You can lock up the server by just visiting the URL the Ajax call uses. After all, client-side scripts should not lock up the server.
-
Alright,
So I’m attempting more things. It is only using the stream to read the contents of the data at any given time. From my testing, the hwinfo (which also uses the same fetchURL command) doesn’t block the page anymore which was occurring with the fread. While potentially not perfect, I don’t know what else to try. The whole idea of the current version and number of sites is simply to give the normal fog user an idea of what is in use. Hopefully this new approach may work better.
Thank you,
-
Before you try to code around an issue we don’t quite understand (I would hate to see you waste time on the wrong chunk of code.), I have some more test results.
I tried commenting out the fetchURL() function contents, but it still froze up on the ajax call. So that was unexpected from what I know so far from analyzing the code. So I made sure that the apache server had been restarted and that you guys didn’t do something crazy with PHP (I had to make sure you weren’t running something like HipHop or some other compiled PHP project).
Next I installed Wireshark to see how it was potentially doing on the network. FOG was sending DNS requests like crazy to my internal DNS (which of course is also isolated in my network). The requests would fail out naturally.
By this point, I realized that this was a good time to follow my advice from earlier. I added 0.0.0.0 fogproject.org to my /etc/hosts. Instant page loads!
This leaves us with two+ questions:
[LIST=1]
[]Where in execution outside of fetchURL() does the server attempt to resolve fogproject.org when http(s)://foginstallation.tld/fog/management/index.php?node=client&sub=loginInfo?
[]Are those other queries being run non-blocking? (probably not)
[*]If everything is running non-blocking (if, remember), does that mean that the way PHP does DNS is still blocking? (Unlikely from what I would expect so far.)
[/LIST] -
There’s only two calls to fogproject.org during the login phase. They’re both, now, and even before, using the fetchURL method.
-
[quote=“Tom Elliott, post: 43343, member: 7271”]There’s only two calls to fogproject.org during the login phase. They’re both, now, and even before, using the fetchURL method.[/quote]
That’s even more interesting. Any ideas why if I comment out the contents of fetchURL() in FOGCore.class.php why it still attempts to visit URLs then? I have to be missing something…
-
I haven’t a clue, but I’m currently attempting a, seeming, more responsive approach from here:
[url]http://wezfurlong.org/blog/2005/may/guru-multiplexing/[/url]
-
[quote=“Wayne Workman, post: 43330, member: 28155”]On a serious note, how likely is a DoS to happen from within your own network? Nobody is going to build a FOG server inside a de-militarized zone, with a public IP address…
FOG is for internal use. ;-)[/quote]
and how would you expect your systems to communicate with your fog server when (nearly) all of your systems have public IP addresses?
-
[quote=“Junkhacker, post: 43495, member: 21583”]and how would you expect your systems to communicate with your fog server when (nearly) all of your systems have public IP addresses?[/quote]
Our district runs under just a hand-full of public IP addresses. The router handles NAT. Our clients have internally issued private IP addresses, and our FOG server has a static private IP address. They communicate fine.
I suppose you were saying that the systems you look after all have public IPs ? Now that’s interesting…
-
[quote=“Wayne Workman, post: 43506, member: 28155”]Our district runs under just a hand-full of public IP addresses. The router handles NAT. Our clients have internally issued private IP addresses, and our FOG server has a static private IP address. They communicate fine.
I suppose you were saying that the systems you look after all have public IPs ? Now that’s interesting…[/quote]
The University I work at runs all public IPs with minor exceptions.
-
The ideal of FOG is that it works regardless of environment.
It’s also the main reason for the work on the new client. First to make it damn near impenetrable (not wanting others to try but if they have methods to correct and further secure things it’d be much better I guess if we royally screwed it up). Second to make it more efficient (non cyclic based but rather event based) and third just get it more modern and updated. So far the work we’ve done has made sure the client will work at least back to XP but also work properly on 7, 8, 8.1, and hopefully future versions of software all at the same time to attempt a comminized programming experience to integrate with Linux and MAC from a single client point of view.
