MCCP Implementation?

Posted by Starflight on Tue 08 Jun 2004 02:29 PM — 12 posts, 37,233 views.

Starflight Australia Tue 08 Jun 2004 02:29 PM #0

I'm trying to implement MCCP on a Circle-based mud and had some questions about how to go about it. I wouldn't say I have in-depth coding experience, but given a few examples I should be able to put it together.

I've done what I believe are the bare essentials but I'm not sure if the problem lies in the code, or the client itself. I'm running a version of the code on my own PC and connecting to it locally via zMud - I'll explain what I've done and hopefully someone can point me in the right direction from here.

Code: IAC WILL COMPRESS2
Client: IAC DO COMPRESS2
Code: IAC SB COMPRESS2 IAC SE

At this point, I get instant compression errors which leads me to believe my simple implementation is incorrect. Going through a bit of debugging, I believe the client is sending back an IAC DONT COMPRESS at this point which is generating the errors. The questions I have are: is my simple implementation correct, or do I need to delve into the whole zlib area? Can I get MCCP by connecting to localhost? Can anyone provide me with some good references or examples to help me along?

Nick Gammon Australia Forum Administrator Tue 08 Jun 2004 09:42 PM #1

Your negotiation looks OK, I would try it with MUSHclient to see if you get the same problem. It shouldn't send IAC DONT COMPRESS in response to IAC WILL COMPRESS2 however it is possible zMUD does that.

In any case, the client response shouldn't throw compression out, as compression is server to client, not client to server. I think the very next byte from the server after IAC SB COMPRESS2 IAC SE needs to be compressed text, make sure you haven't thrown in a newline or something before starting compression.

Yes, you should be able to test on localhost. I think someone else recently mentioned an MCCP snippet for servers on this forum, try searching for that. Also the Dawn Of Time codebase uses MCCP, you could download that and look for the relevant bits.

Starflight Australia Wed 09 Jun 2004 12:01 PM #2

I think the flaw is that I'm not actually compressing anything at all! I've done the basics, but I haven't altered the code which outputs text to actually compress anything.

I gather that all I really need on a basic level, is the deflate function - is that correct?

Darrik Vequir Wed 09 Jun 2004 01:55 PM #3

http://web.archive.org/web/20030219104302/www.randomly.org/projects/MCCP/codebase.html

This has a patch for Circle. I implemented the Smaug snippet, and it worked perfectly, except for one item... the snippet stopped compressing when it received DONT COMPRESS, even when it had received DO COMPRESS2... I had to add an if statement to make sure that it did not stop compressing on DONT COMPRESS if it had received DO COMPRESS2... if you need more info on that, let me know.

DV

Starflight Australia Thu 10 Jun 2004 12:03 PM #4

Many thanks for the help, got it all working. :)

Starflight Australia Sat 12 Jun 2004 05:41 AM #5

Ok, now having successfully managed to get MCCP up and running, I want to be able to toggle it on and off. I've managed to put together what I've seen from a few code snippets, however, I've struck a problem with compression still being active after I believe I've turned it off.

In short, what I've done to "switch it off" is:

deflate(compressed_stream, Z_FINISH)
deflateEnd(compressed_stream)

I then free memory assigned to the whole compression struct I'm using. The problem I've found is that the subsequent text generates a Compression Error - as far as I can tell, text is no longer being compressed and is not being sent in a compressed form to my output function.

What else do I need to do to be able to turn it off?

Greven Canada Sat 12 Jun 2004 07:10 AM #6

This is what I have for mine, this is called from another function, but...

bool compressEnd(DESCRIPTOR_DATA * d)
{
        unsigned char dummy[1];

        if (!d->out_compress)
                return TRUE;

/*    bug("Stopping compression for descriptor %d", d->descriptor); */

        d->out_compress->avail_in = 0;
        d->out_compress->next_in = dummy;

        if (deflate(d->out_compress, Z_FINISH) != Z_STREAM_END)
                return FALSE;

        if (!process_compressed(d)) /* try to send any residual data */
                return FALSE;

        deflateEnd(d->out_compress);
        DISPOSE(d->out_compress_buf);
        DISPOSE(d->out_compress);
		d->shellcompressing = d->compressing;
        d->compressing = 0;

Hope that helps

Starflight Australia Sat 12 Jun 2004 11:10 AM #7

I've implemented a similar function (which I've included below), the only difference being I don't call any "process_compressed" function to empty any remaining compressed text.


t->comp->stream->avail_in = 0;
t->comp->stream->next_in = dummy;

/* Empty the outbut buffer */
while (deflate(t->comp->stream, Z_FINISH) != Z_STREAM_END)
  deflate(t->comp->stream, Z_FINISH);

if ((error = deflateEnd(t->comp->stream))) {  /* Error! */
  sprintf(buf, "[MCCP] deflateEnd returned error %d for %s.\r\n", error, GET_NAME(t->character));
  log(buf);
}

free_mccp(t->comp);

The "free" function calls deflateEnd again (for the stream), free's (DISPOSEs) the buff_in (input buffer text) and buff_out (output buffer text).

zMud gives me a generic "MCCP Compression Error" and displays the offending text (which, coincidentally, happens to be the exact text I want displayed - just not compressed!)

MUSHClient gives me the error message "Could not decompress text from MUD: oversubscribed dynamic bit lengths tree". I believe this message will help debug my problem a little better if anyone could shed some light on what it means.

Nick Gammon Australia Forum Administrator Sun 13 Jun 2004 05:08 AM #8

You need to send the remaining compressed text, otherwise the other end (the client) doesn't know compression has ended. All you have done is tell the server. The compressed stream has to have the "compression terminated" sequence in it so the client knows to stop decompressing too.

Greven Canada Sun 13 Jun 2004 06:28 AM #9

Here is what I use for mine:

bool process_compressed(DESCRIPTOR_DATA * d)
{
        int       iStart = 0, nBlock, nWrite, len;

        if (!d->out_compress)
                return TRUE;

        // Try to write out some data..
        len = d->out_compress->next_out - d->out_compress_buf;

        if (len > 0)
        {
                // we have some data to write
                for (iStart = 0; iStart < len; iStart += nWrite)
                {
                        nBlock = UMIN(len - iStart, 4096);
                        if ((nWrite =
                             write(d->descriptor,
                                   d->out_compress_buf + iStart, nBlock)) < 0)
                        {
                                if (errno == EAGAIN || errno == ENOSR)
                                        break;

                                return FALSE;
                        }

                        if (!nWrite)
                                break;
                }

                if (iStart)
                {
                        // We wrote "iStart" bytes
                        if (iStart < len)
                                memmove(d->out_compress_buf,
                                        d->out_compress_buf + iStart,
                                        len - iStart);

                        d->out_compress->next_out =
                                d->out_compress_buf + len - iStart;
                }
        }

        return TRUE;
}

That will finish whatever is compressed still, and send it out. I see that you call deflateEnd, which is what will send the sequence to stop compressing.

On a side note, you can use something like this to show you in-mud stats, should your client not provide them, though MUSHclient does

	if ( ch->desc->out_compress && ch->desc->out_compress->total_in)
		ch_printf(ch, "Total size of input compressed:  &B[&w%d&B]&D\n\r", ch->desc->out_compress->total_in);
	if ( ch->desc->out_compress && ch->desc->out_compress->total_out)
		ch_printf(ch, "Total size of output compressed: &B[&w%d&B]&D\n\r", ch->desc->out_compress->total_out);
	if ( ch->desc->out_compress && ch->desc->out_compress->total_in && ch->desc->out_compress->total_out)
		ch_printf(ch, "Current compression ratio:       &B[&w%.2f%&B]&D&D\n\r", ch->desc->out_compress->total_out / 
			ch->desc->out_compress->total_in * (float)100));

Also, if you want to confirm which version you happen to be using:

ch_printf(ch, "Current ZLIB version used for MCCP: %d\n\r" MZLIB_VERSION);

Hope that helps some.

Starflight Australia Sun 13 Jun 2004 12:28 PM #10

Ok, I'm having a little trouble relating the SMAUG structure to the CircleMud one - I understand the theory behind it all, but not the pieces of the struct I should be trying to empty.

I'm using the structure defined below and the standard z_streamp from zlib.h:

struct compr {
  ubyte state; /* 0 - off. 1 - waiting for response. 2 - compress2 on */

  Bytef *buff_out;
  uInt total_out; /* size of input buffer */
  uInt size_out; /* size of data in output buffer */

  Bytef *buff_in;
  uInt total_in; /* size of input buffer */
  uInt size_in; /* size of data in input buffer */

  z_streamp stream;
};

I believe I need to set up a similar loop which repeatedly calls the write function (or write_to_descriptor in my case) until stream->next_out is empty.

Is that correct? Will the deflate call with Z_FINISH send the "compression terminated" sequence that is required?

Nick Gammon Australia Forum Administrator Sun 13 Jun 2004 10:16 PM #11

Judging by the code I have seen, yes it will.