SmaugFuss IMPORTANT Test

Posted by Matteo2303 on Tue 23 Jan 2007 09:33 PM — 35 posts, 110,840 views.

#0
Hello, is there someone that could be run the SmaugFuss in Windows (with cygwin+gdb) telling me the IP/PORT?

I need to test a possible crash with multiple connections that produce this error:

"accept_new: select: poll: Bad file descriptor"

Tnx
bye
USA #1
Could you maybe give a little more information on what exactly you are going to do? Multiple connections seem to work fine in most cases; have you found some kind of special problem?
#2
I want open 50-100 connections in 10 seconds, and close them all in one. Mud crashes... Can I try with you?
USA #3
That's odd, my MUD just got spammed from a connection (someone from Italy). At least 60 connections in a few seconds.
#4
Sorry Sorry Sorry, I call you in mud, but i press enter in my test program for mistake.

I think that there is a big vulnerability in all smaugfuss socks.

Sorry again Zeno.
I'm desolated.
USA #5
That's what I thought you were going to do, some kind of DoS attack. As a general rule it really isn't very polite to do that to somebody without being very clear about what you're going to do beforehand. :)

It wouldn't surprise me if the default SMAUG networking code has vulnerabilities like that. Is there any reason why you're looking for Cygwin MUDs in particular?
USA #6
Probably because Windows might have less of 'control' over sockets, IE the limit is not as large.
#7
>As a general rule it really isn't very polite to do that

Yes, sorry, a true mistake. (the mud crashed)

>Is there any reason why you're looking for Cygwin MUDs in particular?

Becose I work on cygwin and smaugfuss and the problem exist.

Bye
matteo
#8
>Probably because Windows might have less of 'control' over sockets, IE the limit is not as large.

Ok that's is possible, but this mean that all the people that run smaug under windows can crash with a simple DoS attack? :( I cant belive.
USA #9
Quote:
Yes, sorry, a true mistake. (the mud crashed)

Well, if you were referring to my MUD... no it hasn't crashed recently, at all.
#10
>Well, if you were referring to my MUD... no it hasn't crashed recently, at all.

So strange. I'm 99% secure that smaugfuss v1.4-->v1.7 under cygwin the mud crash with the "accept_new: select: poll: Bad file descriptor". The latest version (1.7) is more stronger, and 500connections are necessary sometimes. Are you sure tha your mud don't fall down after my DoS? Do you use cygwin?

Bye
USA #11
When this happens and the MUD crashes, do you get any kind of log message besides the perror report?


The man page for select will complain if you try to select on sockets that are closed.

The accept_new function adds all connections it thinks are open to the in_set before selecting on it.

Therefore, if some of those sockets are closed but the MUD does not completely correctly remove them in the right order with respect to select, the select call will fail with EBADF which is the error message you are seeing.

So, I'd like to know if any of the connections coming in are getting through and closing before the MUD panics, or if it's the first few that cause it to die.
#12
I dont speak well english :(
I can report my logs...

Smaugfuss 1.7 logs are different from smaugfuss 1.4. I report the two logs.

FUSS 1.7:

Wed Jan 24 00:55:17 2007 :: Sock.sinaddr: 127.0.0.1, port 4874.
Wed Jan 24 00:55:17 2007 :: Sock.sinaddr: 127.0.0.1, port 4875.
<LONG CUT>
Wed Jan 24 00:55:20 2007 :: Sock.sinaddr: 127.0.0.1, port 4887.
Wed Jan 24 00:55:21 2007 :: Sock.sinaddr: 127.0.0.1, port 4891.
Wed Jan 24 00:55:21 2007 :: Sock.sinaddr: 127.0.0.1, port 4902.
Wed Jan 24 00:55:21 2007 :: EOF encountered on read.
Wed Jan 24 00:55:21 2007 :: EOF encountered on read.
<LONG CUT>
Wed Jan 24 00:55:21 2007 :: EOF encountered on read.
Wed Jan 24 00:55:21 2007 :: EOF encountered on read.
Read_from_descriptor: Software caused connection abort
Wed Jan 24 00:55:21 2007 :: EOF encountered on read.
accept_new: select: poll: Bad file descriptor

Program exited with code 01.
(gdb) bt
No stack.

And under smaugfuss 1.4 (with some little changes)...

Wed Jan 24 01:01:40 2007 :: Broke all-time maximum player record: 66
Wed Jan 24 01:01:40 2007 :: Sock.sinaddr: 127.0.0.1, port 1432.
Wed Jan 24 01:01:40 2007 :: Broke all-time maximum player record: 67
Wed Jan 24 01:01:41 2007 :: Sock.sinaddr: 127.0.0.1, port 1434.
Wed Jan 24 01:01:41 2007 :: Broke all-time maximum player record: 68
Wed Jan 24 01:01:41 2007 :: [*****] BUG: Auth_check: exception found for (unknown)@127.0.0.1.
Wed Jan 24 01:01:45 2007 :: Sock.sinaddr: 127.0.0.1, port 1435.
Wed Jan 24 01:01:45 2007 :: Broke all-time maximum player record: 69
Wed Jan 24 01:01:45 2007 :: [*****] BUG: Auth_check: exception found for (unknown)@127.0.0.1.
Wed Jan 24 01:01:45 2007 :: [*****] BUG: Auth_check: exception found for (unknown)@127.0.0.1.
Wed Jan 24 01:01:51 2007 :: Sock.sinaddr: 127.0.0.1, port 1436.
Wed Jan 24 01:01:51 2007 :: Broke all-time maximum player record: 70
Wed Jan 24 01:01:51 2007 :: Sock.sinaddr: 127.0.0.1, port 1437.
Wed Jan 24 01:01:51 2007 :: Broke all-time maximum player record: 71
Wed Jan 24 01:01:52 2007 :: Sock.sinaddr: 127.0.0.1, port 1438.
Wed Jan 24 01:01:52 2007 :: Broke all-time maximum player record: 72
Wed Jan 24 01:01:52 2007 :: Sock.sinaddr: 127.0.0.1, port 1439.
Wed Jan 24 01:01:52 2007 :: Broke all-time maximum player record: 73
Wed Jan 24 01:01:52 2007 :: Sock.sinaddr: 127.0.0.1, port 1440.
Wed Jan 24 01:01:52 2007 :: Broke all-time maximum player record: 74
Wed Jan 24 01:01:52 2007 :: [*****] BUG: Auth_check: exception found for (unknown)@127.0.0.1.
Wed Jan 24 01:01:52 2007 :: [*****] BUG: Auth_check: exception found for (unknown)@127.0.0.1.
accept_new: select: poll: Bad file descriptor

Program exited with code 01.
(gdb) bt
No stack.
(gdb)


USA #13
Looks like an exit to me, and not a crash.
#14
Yes, is an exit:

if ( select( maxdesc+1, &in_set, &out_set, &exc_set, &null_time ) < 0 )
{
perror( "accept_new: select: poll" );
exit( 1 );
}

...but how I can prevent this? I think that's a bug. The descriptor cant become "-1"... this can be only if there is a problem imho.

Tnx
USA #15
The descriptor isn't becoming -1. The select call is returning a value smaller than 0 indicating that there was an error.

Without looking too deeply at the network code -- I don't have time now unfortunately -- I suspect that the issue has to do with improper cleanup of descriptors that were closed unexpectedly, which is what is giving you those messages ("EOF encountered on read").

Then, since the sockets are no longer in an acceptable state, when select comes around and accept_new adds all descriptors to the FD sets, select emits the error because some of the FDs given to it were invalid.
Amended on Tue 23 Jan 2007 11:30 PM by David Haley
#16
>The select call is returning a value smaller than 0

Yes, sorry, not descriptor. I know that the value is "-1" becaus I used printf( "%d\n\r", etc.. ) for see it.

In any case I think that this problem isn't only "my".
Tnx for your help. In the next days I'll investigate about this your words:

> select emits the error because some of the FDs given to it were invalid.

Bye!
USA #17
No, I agree that it doesn't appear to be a problem just with you. The SMAUG network code is pretty shaky -- networking stuff is hard to get right. (It's kind of like the mudprog discussion we've been having here.) So like I said, it doesn't really surprise me that it would perform poorly (i.e. die) under stress.
Australia Forum Administrator #18
Quote:

if ( select( maxdesc+1, &in_set, &out_set, &exc_set, &null_time ) < 0 )
{
perror( "accept_new: select: poll" );
exit( 1 );
}

...but how I can prevent this?


You could simply omit the exit line, and just ignore problems if select is overloaded by multiple frequent connections. This would stop the MUD from exiting, and may only happen occasionally.
USA #19
I suspect that might cause trouble. The problem is that one of the descriptors has been invalidated and is kept around in memory. If it gets left there, it might cause all kinds of trouble further down the road.

Still, it's worth a shot, I suppose... sometimes it's good to live dangerously. :-)
#20
>You could simply omit the exit line, and just ignore problems

No, make "//exit(1);" is a bad thing!
All existing descriptor "resets/restarts"; apparently is ok but it's a panic things: belive me, I had try :(
USA #21
What if instead of exit(1) you tried changing it to a continue statement? That in theory would skip it on that pass and come back for it later. Maybe the OS just needs a bit of time to clean up the mess.
#22
>What if instead of exit(1) you tried changing it to a continue statement?

Mmm, "continue"? Where?
"Select" istance isn't inside a loop.
#23
Ok, I had test using a GOTO istance (lol) and reproduce a continue effect. This freze the mud. Also, if I use a "return" instead of "exit(1)" when "select" istance is < 0, then all descriptors resets (player disconnected, etc...). This is not the right way.

Bye
USA #24
Right, ok. So I didn't notice this was happening in accept_new before. Scratch the continue idea.

With the situation you have you're probably out of luck without some major work on the networking code.
USA #25
i am not as advanced as some of you, but i wanted to put this out there, instead of exiting couldn't you just deny the connection, and let the client connecting know to wait a minute and retry.
USA #26
Unfortunately no; what's going on here is that somehow SMAUG's internal picture of the network state has gotten messed up and it thinks some connections are open that really aren't. Then it tries to poll on those connections and fails, which is what is causing select to generate an error.

What might be possible would be to loop over all known descriptors and test each one with select individually until you find the one that returns an error, and then completely dump it. You would only do this after select returned an error, of course. This is not an ideal solution -- the real fix would be to not get into this situation in the first place -- but it's better than nothing.
#27
>With the situation you have you're probably out of luck without some major work on the networking code.

Not "I have", but "we have". SmaugFuss under Cygwing Windows crashes in this situation. A DoS attacks generate a crash in a minute.
Australia Forum Administrator #28
Does this crash only happen under Cygwin? It wouldn't totally surprise me, that it is only unstable under Windows, and most people running production MUDs would be using Unix.

I once wrote a similar piece of code, to test multiple simultaneous connections to PennMUSH. It worked fine under Unix, but degraded rapidly under Windows. And that was using identical code.

I suspect this is an instability in the Windows Cygwin implementation of sockets, or even the underlying operating system support.
USA #29
I would tend to agree with Nick's assessment since I've stress tested my own code with someone deliberately attacking it with a miniature bot-net. The worst that happened was that it simply stopped accepting new connections until some of the existing ones dropped off. Tested on linux of course.

I don't really see much value in protecting against this in Windows for the very reason that Cygwin isn't suited for a production environment.
#30
I think so the problems is cygwin sockets, where I today notice another problem: with smaugfuss (1.4a?) can accept only 56/58 player connections!!!

Under linux is the same? [I had create a script that create a random player connecting to the mud and the limit is 56/58 connections].

Bye :(

ps: The smaug distro for cygwing, if the conditions are this, isn't good enough. Pardon me samson. I intended that nobody can run realy a mud under windows.
USA #31
Matteo, if this is a limitation of Cygwin sockets, then the problem has very little, perhaps nothing, to do with the SMAUG(fuss) code.

Under Linux, SMAUG(+fuss) can definitely accept more than 58 connections.

Like I said, this is very likely to be a problem of Cygwin -- not Windows, actually, but the Cygwin implementation under Windows. (The Cygwin folks say over and over again that their system is not meant to be for production software as several things aren't quite right yet.)
Australia Forum Administrator #32
Quote:

The smaug distro for cygwing, if the conditions are this, isn't good enough.


There is no real obligation on the part of the SMAUG distributors, when distributing free software, to put a lot of work into making it work on all operating systems (eg. Windows, Mac, etc.).

Running under Cygwin is fine for testing new code (eg. testing a new area) before putting it into production. But as I said before, if you plan on running a MUD for dozens of simultaneous users, you are better off using Linux.
#33
>There is no real obligation on the part of the SMAUG
>distributors, when distributing free software, to put a
>lot of work into making it work on all operating systems
>(eg. Windows, Mac, etc.).

Yes, i don't want missing of respect about cygwin project: it's a great things in any case! Probably with my bad english I don't show my opinion.

In any case... If someone in SmaugFuss has the problem that I illustrated in this post because cygwin-socket has some limitation, you could try to use INTERIX (Free M$ SFU35SEL file) instead of cygwin. But you can use this only with some Windows OS (XP Pro, NT, and ...).

I tested the smaugFuss in Interix opening more than 300 connections in one same without problems.

You need to add crypt and some other things, than run perfecly! (seems...)

Probably with interix works because in this case the smaug run like a native linux file.

Bye
Matteo
#34
PS: the mud under INTERIX speed up very very very much!
Cygwin seems very slow in compare.

Bye again