'omit from output' triggers slightly delayed

Posted by Alisa on Mon 07 Oct 2019 07:14 AM — 39 posts, 120,051 views.

Alisa Mon 07 Oct 2019 07:14 AM #0

A game I play on recently made the questionable decision to relax their rules regarding spam frequency on public chat channels, so I am trying to set up a simple trigger to automatically omit the worst offenders from output, in effect causing them to cease to exist on my end.

Unfortunately, I seem to be doing something wrong. The trigger does work as intended but there seems to be some kind of slight delay involved with it. Omitted lines are indeed omitted, but only after a brief delay of around 1 or 2 seconds; long enough for me to glimpse the lines before they poof out of existence.

This causes my output screen to kind of erratically 'jerk' up and down when these lines are encountering the trigger, in that they briefly appear, cause the output window to scroll down one line to make room for them, then the line is removed and the output window jerks back upward when the line vanishes.

Being that this is somehow even more annoying than having to see the same obnoxious advertisements in chat every 5 minutes, I'm hoping someone here can take a look at this screencap of an example trigger and suggest what might be causing the delay here. I've seamlessly used this sort of trigger in the past so I'm not sure what I might have borked up to suddenly cause it to stop working as intended.

https://puu.sh/EpJko/48d2d111fd.png

(The <General> bit in the trigger is to include the displayed name of the chat channel so that I can only filter out lines beginning this way, rather than omitting any line even mentioning the offender's name.)

Nick Gammon Australia Forum Administrator Mon 07 Oct 2019 09:30 AM #1

First, we prefer if you copy the trigger as text so it can be pasted into a test without actually retyping:

For advice on how to copy aliases, timers or triggers from within MUSHclient, and paste them into a forum message, please see Copying XML.

It seems odd to check "Keep Evaluating" and then go to the trouble of running a script to stop evaluating. Perhaps you are trying to stop evaluation in all plugins? In any event, I would uncheck "Keep Evaluating". Example:


<triggers>
  <trigger
   enabled="y"
   ignore_case="y"
   match="^&lt;General&gt; Blah Blah Blah Spam Sure Is Annoying .+$"
   omit_from_output="y"
   regexp="y"
   sequence="1"
  >
  </trigger>
</triggers>

For advice on how to copy the above, and paste it into MUSHclient, please see Pasting XML.

I can't actually reproduce your problem. Using version 5.06 of the client I don't see the effect you are describing.

To download:

http://www.gammon.com.au/downloads/dlmushclient.htm

Please help us by advising the version of MUSHclient you are using. Use the Help menu -> About MUSHclient.

The general way that the client works is that a line from the MUD is displayed (byte by byte) in the output window, and then when a newline is received, triggers are evaluated. This conceivably might cause text to briefly appear and then disappear. On my PC, which is fairly fast, I don't notice that. The updating of the screen itself should occur periodically, not for every byte received. This might sound contradictory, but there are two things: the screen data as assembled, ready to be displayed, and the moment when it is actually displayed.

If you are using the latest client, and unchecking "Keep Evaluating" doesn't work, then a fall-back position would be to try to get the spam out of the input packet before it is processed at all. This might be fiddly to do if the spam spans multiple network packets, but if it doesn't it should be reliable.

A possible plugin to implement the packet approach would be this.

To save and install the Omit_Spam plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Omit_Spam.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Omit_Spam.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Omit_Spam"
   author="Nick Gammon"
   id="cd039821125303ed7c839fbf"
   language="Lua"
   purpose="Omits spam from packets"
   date_written="2019-10-07 20:17:00"
   requires="5.00"
   version="1.0"
   >

</plugin>


<!--  Script  -->


<script>
<![CDATA[

function OnPluginPacketReceived (s)

  return (string.gsub (s, "<General> Blah Blah Blah Spam Sure Is Annoying .-\r?\n", ""))

end  -- OnPluginPacketReceived

]]>
</script>
</muclient>

Note that this uses Lua regular expressions, not PCRE ones.

The Lua regular expression syntax is documented here:

http://www.gammon.com.au/scripts/doc.php?lua=string.find

In particular, a non-greedy wildcard is ".-" rather than what you had. This particular plugin scans the entire packet for the spam (which might occur multiple times) hence the test for a newline.

This test will fail if the spam spans multiple packets, but if you combine the plugin with the trigger it should be less annoying.

Nick Gammon Australia Forum Administrator Mon 07 Oct 2019 09:43 AM #2

A more sophisticated approach at the packet level would be to detect the spammer name, like this:


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Omit_Spam"
   author="Nick Gammon"
   id="cd039821125303ed7c839fbf"
   language="Lua"
   purpose="Omits spam from packets"
   date_written="2019-10-07 20:17:00"
   date_modified="2019-10-07 20:45:00"
   requires="5.00"
   version="1.1"
   >

</plugin>


<!--  Script  -->


<script>
<![CDATA[

idiots = {
   "Lorem",
   "ipsum",
   "dolor",
   "sit",
   "amet",
   "consectetur",
   "adipiscing",
   "elit",
   }

-- turn table into keyed by name table
  
idiots_lookup = {}

for k, v in ipairs (idiots) do
   idiots_lookup [v] = true
end -- for

function spammer_test (a)
  if idiots_lookup [a] then
    return ""
  end -- if
  
  -- show original line  
  return nil
end -- function spammer_test

function OnPluginPacketReceived (s)

  return (string.gsub (s, "<General> ([^ ]+) .-\r?\n", spammer_test))

end  -- OnPluginPacketReceived

]]>
</script>
</muclient>

Now you make a simple list of the idiots you want to ignore (this is case-sensitive). The function takes any input matching: "<General> somename somespam" and then checks the name against the list. If a spammer shows up the whole line is omitted, otherwise it is allowed through unchanged.

If there is some punctuation (eg. a colon after the name) then add that into the regular expression. This particular regexp assumes that the name is anything other than spaces, but you can tweak that.

Nick Gammon Australia Forum Administrator Mon 07 Oct 2019 10:19 AM #3

My solution above might fail if people put "<General> spammer spam" in a message that is not at the start of the line. I didn't want to test for a newline at the start, as packets probably don't start with a newline.

If this is an issue, we could add an extra newline at the start, check for "\n spam message \n" and then omit that first newline.

However let's not go to that complexity unless it turns out to be a problem.

Fiendish USA Global Moderator Mon 07 Oct 2019 02:48 PM #4

Quote:
Omitted lines are indeed omitted, but only after a brief delay of around 1 or 2 seconds

I wonder if the server is doing something strange, like waiting a second before sending newline in a separate packet, or sending newline at the start of each message rather than at the end. A snippet from the packet debug display might clarify. ( Edit menu -> Debug Packets )

Nick Gammon Australia Forum Administrator Wed 09 Oct 2019 09:09 PM #5

I agree with Fiendish that this is a plausible explanation.

Alisa Thu 10 Oct 2019 04:37 AM #6

Hello again! Just a quick note that I am not ignoring anyone or taking my help and running! Things have been a bit hectic this week and I haven't been able to MUSH as often as I would like.

I realized later that something which may or may not be useful for the troubleshooting is that the game uses Pueblo and, while I have the resulting clicky links disabled from displaying on my end, I think the game actually still transmits the code around them. Would that maybe have something to do with the weird jerky delay for the omitted lines?

Also! With the plugin helpfully provided, how does that search for the names? Does it search the entire line, or just the beginning? What happens if the user has a channel title showing prior to their name? Like a line might start out as '<General> Titlehere Namehere' before getting on with whatever they said or posed. And what happens if that title has punctuation included?

Sorry for all of the questions. I figured that there has to be something else influencing all of this if your own tests could not reproduce the problem, especially after I gave it a try on 5.05, 5.06, and the 5.07-prerelease with the same results across all three.

As for the packet debug, I ducked into the game for a moment to grab the debug for whatever line first popped up. I have no idea what all of this mess means but maybe it will be useful for your suspicions?

Incoming packet: 87 (2 bytes) at Thursday, October 10, 2019, 12:24:28 AM

ÿñ                 ff f1

Incoming packet: 88 (160 bytes) at Thursday, October 10, 2019, 12:24:33 AM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; .[38;5;   1b 5b 30 6d 26 67 74 3b 20 1b 5b 33 38 3b 35 3b
228mHoney.[0m .[   32 32 38 6d 48 6f 6e 65 79 1b 5b 30 6d 20 1b 5b
38;5;255mMaid.[0   33 38 3b 35 3b 32 35 35 6d 4d 61 69 64 1b 5b 30
m Ayako Kurota s   6d 20 41 79 61 6b 6f 20 4b 75 72 6f 74 61 20 73
ays, &quot;Don't   61 79 73 2c 20 26 71 75 6f 74 3b 44 6f 6e 27 74
 worry, Kamadeva   20 77 6f 72 72 79 2c 20 4b 61 6d 61 64 65 76 61
, White Ren is a   2c 20 57 68 69 74 65 20 52 65 6e 20 69 73 20 61
 totally differe   20 74 6f 74 61 6c 6c 79 20 64 69 66 66 65 72 65
nt person.&quot;   6e 74 20 70 65 72 73 6f 6e 2e 26 71 75 6f 74 3b

Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM

<                  3c

Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM

>                  3e

Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM

"                  22

Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM

"                  22

Incoming packet: 89 (5 bytes) at Thursday, October 10, 2019, 12:24:33 AM

<BR>.              3c 42 52 3e 0a

Nick Gammon Australia Forum Administrator Thu 10 Oct 2019 06:38 AM #7

Well, the plugin won't work with that because, amongst other things, the newline is in the 6th packet.

Is that what the MUD sends? It looks like rubbish.

For one thing, you have single byte packets which are inefficient. They could conceivably introduce delays.

There are colour codes there which would throw out my regular expression (that isn't their fault).


<General> Honey Maid Ayako Kurota says, "Don't worry, Kamadeva, White Ren is a totally different person."""

Is that what you see on your screen? A chat line ending in three double-quotes? Weird.

Packet 88 and packet 89 are just "<" and ">" which doesn't make any sense when using Pueblo.

Can you respond to this please?

Please help us by advising the version of MUSHclient you are using. Use the Help menu -> About MUSHclient.

Alisa Thu 10 Oct 2019 07:31 AM #8

As noted in my last post, I have been trying to fix this with 5.05, 5.06, and 5.07-pre to no avail. 5.07-pre is the one where that packet debug came from, just because it was the last version in the sequence I tried and thus the one I left installed.

I don't believe there are triple quotes at the end of the lines. I would have to check when I get home to be certain but that seems like an oddity I would have noticed a long time ago.

As for the code... yeah, the coder there is apparently absolutely terrible. I do not know enough about code to criticize but my coder friends and coders from other games pretty much despise this person, who supposedly steals code from other places, tacks on a few clumsy tweaks, then calls it their own product. They even obfuscate all kinds of things so much that players can't even examine themselves to see which attributes are set, rendering @decompile useless. I have no idea why but the coder people say it is to keep people from seeing the underhanded stuff this person does.

MUSH drama hoooooooooo!

Nick Gammon Australia Forum Administrator Thu 10 Oct 2019 10:16 AM #9

Actually, the packet numbers in what you posted look suspicious:


Incoming packet: 87 (2 bytes) at Thursday, October 10, 2019, 12:24:28 AM
Incoming packet: 88 (160 bytes) at Thursday, October 10, 2019, 12:24:33 AM
Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM
Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM
Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM
Incoming packet: 88 (1 bytes) at Thursday, October 10, 2019, 12:24:33 AM
Incoming packet: 89 (5 bytes) at Thursday, October 10, 2019, 12:24:33 AM

You have a lot of packet number 88 there. That doesn't look right. Either you edited what was in the packet debug, or the client has a bug with packet numbers.

Alisa Thu 10 Oct 2019 10:46 AM #10

Nothing was edited on this end. I wouldn't even know what to edit. I copied it straight from the window to a text file to ensure it copied okay, then from the text file to here.

If it helps, I can grab another packet sample later from 5.06. Maybe the prerelease of 5.07 is having a tantrum.

Alisa Thu 10 Oct 2019 12:12 PM #11

Here are three packet debugs from 5.06. The first two are from someone without a channel title, the third is from someone with a channel title that has color and a punctuation mark in it. I have no idea what could be causing the problem here so I figured giving examples of multiple situations might be helpful.

Incoming packet: 7 (49 bytes) at Thursday, October 10, 2019, 7:51:26 AM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; Grugork   1b 5b 30 6d 26 67 74 3b 20 47 72 75 67 6f 72 6b
 pets the puppie   20 70 65 74 73 20 74 68 65 20 70 75 70 70 69 65
s                  73

Incoming packet: 7 (1 bytes) at Thursday, October 10, 2019, 7:51:26 AM

<                  3c

Incoming packet: 7 (1 bytes) at Thursday, October 10, 2019, 7:51:26 AM

>                  3e

Incoming packet: 8 (5 bytes) at Thursday, October 10, 2019, 7:51:26 AM

<BR>.              3c 42 52 3e 0a

Incoming packet: 9 (71 bytes) at Thursday, October 10, 2019, 7:51:53 AM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; Grugork   1b 5b 30 6d 26 67 74 3b 20 47 72 75 67 6f 72 6b
 pulls Belladonn   20 70 75 6c 6c 73 20 42 65 6c 6c 61 64 6f 6e 6e
a down from the    61 20 64 6f 77 6e 20 66 72 6f 6d 20 74 68 65 20
ceiling            63 65 69 6c 69 6e 67

Incoming packet: 9 (1 bytes) at Thursday, October 10, 2019, 7:51:53 AM

<                  3c

Incoming packet: 9 (1 bytes) at Thursday, October 10, 2019, 7:51:53 AM

>                  3e

Incoming packet: 10 (5 bytes) at Thursday, October 10, 2019, 7:51:53 AM

<BR>.              3c 42 52 3e 0a

Incoming packet: 24 (88 bytes) at Thursday, October 10, 2019, 7:55:32 AM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; Go for    1b 5b 30 6d 26 67 74 3b 20 47 6f 20 66 6f 72 20
a .[38;5;80mswim   61 20 1b 5b 33 38 3b 35 3b 38 30 6d 73 77 69 6d
.[0m? Misty lean   1b 5b 30 6d 3f 20 4d 69 73 74 79 20 6c 65 61 6e
s on Nessa. Boop   73 20 6f 6e 20 4e 65 73 73 61 2e 20 42 6f 6f 70
 tanuki.           20 74 61 6e 75 6b 69 2e

Incoming packet: 24 (1 bytes) at Thursday, October 10, 2019, 7:55:32 AM

<                  3c

Incoming packet: 24 (1 bytes) at Thursday, October 10, 2019, 7:55:32 AM

>                  3e

Incoming packet: 25 (5 bytes) at Thursday, October 10, 2019, 7:55:32 AM

<BR>.              3c 42 52 3e 0a

Alisa Thu 10 Oct 2019 12:13 PM #12

And to be extra thorough, here is another packet debug from someone who should have been omitted and triggered that odd 1-second delay before the line poofed. Maybe there is something in here that isn't in the others?

Incoming packet: 20 (130 bytes) at Thursday, October 10, 2019, 7:54:52 AM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; .[38;5;   1b 5b 30 6d 26 67 74 3b 20 1b 5b 33 38 3b 35 3b
57mDo Not Trust.   35 37 6d 44 6f 20 4e 6f 74 20 54 72 75 73 74 1b
[0m Riven nods a   5b 30 6d 20 52 69 76 65 6e 20 6e 6f 64 73 20 61
nd maybe sneaks    6e 64 20 6d 61 79 62 65 20 73 6e 65 61 6b 73 20
a quick feel on    61 20 71 75 69 63 6b 20 66 65 65 6c 20 6f 6e 20
Ruby~ &quot;Good   52 75 62 79 7e 20 26 71 75 6f 74 3b 47 6f 6f 64
 to know.&quot;    20 74 6f 20 6b 6e 6f 77 2e 26 71 75 6f 74 3b 20
^^                 5e 5e

Incoming packet: 20 (1 bytes) at Thursday, October 10, 2019, 7:54:52 AM

<                  3c

Incoming packet: 20 (1 bytes) at Thursday, October 10, 2019, 7:54:52 AM

>                  3e

Incoming packet: 20 (1 bytes) at Thursday, October 10, 2019, 7:54:52 AM

"                  22

Incoming packet: 20 (1 bytes) at Thursday, October 10, 2019, 7:54:52 AM

"                  22

Incoming packet: 21 (5 bytes) at Thursday, October 10, 2019, 7:54:52 AM

<BR>.              3c 42 52 3e 0a

Nick Gammon Australia Forum Administrator Thu 10 Oct 2019 08:39 PM #13

The multiple packets with the same number seem to be recurring. It looks like this is an artefact of the MCCP compression. For some (odd) reason the message is being compressed multiple times in succession.

Quote:

As for the code... yeah, the coder there is apparently absolutely terrible.

That's certainly a possible reason.

Compressing a single-byte piece of data is likely to increase the amount of data sent, not decrease it.

Try turning off compression, that might cause the packets to not be split up like that.

Configuration -> Appearance -> Output -> Disable compression.

Then collect some more packets for us. Do those multiple quotes appear in the output on your screen or not? It would help to copy/paste (here) what you see in the output window as well as the packet debug (for the same chat messages).

Try also disabling Pueblo.

Configuration -> Appearance -> MXP / Pueblo -> Use MXP/Pueblo: No - Never.

Nick Gammon Australia Forum Administrator Thu 10 Oct 2019 08:43 PM #14

Alisa said:

Also! With the plugin helpfully provided, how does that search for the names? Does it search the entire line, or just the beginning? What happens if the user has a channel title showing prior to their name? Like a line might start out as '<General> Titlehere Namehere' before getting on with whatever they said or posed. And what happens if that title has punctuation included?

It just looked for the first word (something delimited by spaces) after the channel name. With all those colour codes you had it wouldn't have matched anything, and it looks like the names have spaces in them, so work would be needed to make that work properly.

Alisa Thu 10 Oct 2019 09:33 PM #15

Oh, right. The quotes thing. They don't appear on the screen at all despite what shows in the packet debug. Only the normal quotation marks in the appropriate places!

As for the rest, disabling compression alone made no difference and the packets kept getting split up.

Disabling Pueblo as well seemed to make a difference:

Incoming packet: 343 (206 bytes) at Thursday, October 10, 2019, 5:22:43 PM

&lt;.[35mGeneral   26 6c 74 3b 1b 5b 33 35 6d 47 65 6e 65 72 61 6c
.[0m&gt; .[38;5;   1b 5b 30 6d 26 67 74 3b 20 1b 5b 33 38 3b 35 3b
220mB.[38;5;124m   32 32 30 6d 42 1b 5b 33 38 3b 35 3b 31 32 34 6d
urst.[0m .[38;5;   75 72 73 74 1b 5b 30 6d 20 1b 5b 33 38 3b 35 3b
220mA.[38;5;124m   32 32 30 6d 41 1b 5b 33 38 3b 35 3b 31 32 34 6d
ngel.[0m Jo says   6e 67 65 6c 1b 5b 30 6d 20 4a 6f 20 73 61 79 73
, &quot;Meg is n   2c 20 26 71 75 6f 74 3b 4d 65 67 20 69 73 20 6e
ormal human bada   6f 72 6d 61 6c 20 68 75 6d 61 6e 20 62 61 64 61
ss, Jo is 'raise   73 73 2c 20 4a 6f 20 69 73 20 27 72 61 69 73 65
d in a lab to be   64 20 69 6e 20 61 20 6c 61 62 20 74 6f 20 62 65
 perfectly engin   20 70 65 72 66 65 63 74 6c 79 20 65 6e 67 69 6e
eered to murder'   65 65 72 65 64 20 74 6f 20 6d 75 72 64 65 72 27
 badass.&quot;     20 62 61 64 61 73 73 2e 26 71 75 6f 74 3b

Incoming packet: 344 (5 bytes) at Thursday, October 10, 2019, 5:22:44 PM

<BR>.              3c 42 52 3e 0a

Enabling compression again did not change these improved results, but the moment I enabled Pueblo again (and relogged, since it did not kick back in simply by toggling) it was back to the multiple packets and strange unseen quotes.

Nick Gammon Australia Forum Administrator Fri 11 Oct 2019 06:09 AM #16

What I am seeing above is the first real evidence that the MUD is introducing the delay.

Packet 343 comes in at 5:22:43 PM and packet 344 (which has the newline in it) comes in at 5:22:44 PM.

So there is a second difference. However because of rounding it may be a partial second, but there is a second for you.

What might be needed here is a packet pre-processor that waits until a whole line has arrived and then lets the packet through for evaluation. That would eliminate the difference between the bulk of the text and the newline.

However if such a thing was there all the time, and you log on with prompts, then you won't see the prompt (because they don't have a newline).

Do you log on with prompting? What I mean is:


Enter your name: (name)
Enter your password: (password)

Some MUSHes don't use prompting, you just type an entire line, eg.


connect (name) (password)

If you do use prompting, is there some string that could tell a plugin that you have logged in? eg. "Welcome to XYZ MUD!". This would need to be something that appears after the login sequence.

Nick Gammon Australia Forum Administrator Fri 11 Oct 2019 06:41 AM #17

This plugin should buffer the packets. Then your triggers should work.

To save and install the Buffer_Lines plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Buffer_Lines.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Buffer_Lines.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Buffer_Lines"
   author="Nick Gammon"
   id="8eb1cd324d58655b0ff057db"
   language="Lua"
   purpose="Buffer packets so they have a newline in them"
   date_written="2019-10-11 17:27:00"
   requires="4.80"
   version="1.0"
   >
</plugin>

<!--  Trigger to show you have connected  -->

<triggers>
  <trigger
   enabled="y"
   match="You have connected!"
   script="we_connnected"
   sequence="100"
  >
  </trigger>
</triggers>



<!--  Script  -->

<script>
<![CDATA[

local previousPacket = { }
local lines = { }
local buffer_packets = false

-- add to lines table
function addToLines (s)
  lines [#lines + 1] = s
  return "" -- omit from the existing line
end  -- end of function addToLines
            
-- here when we log in 
function we_connnected (name, line, wildcards)
   buffer_packets = true
end -- we_connnected
       
function OnPluginPacketReceived (packet)
  
  -- do nothing if we haven't logged in
  if not buffer_packets then
   return nil
  end -- not reached connection message
 
 
  -- add packet to previous ones
  previousPacket [#previousPacket + 1] = packet

  -- combine into one long string
  local allPackets = table.concat (previousPacket)   

  lines = { }  -- no lines yet
  
  -- break up combined packet into lines with LF in them
  local LeftOver = string.gsub (allPackets, "([^\n]*)\n", addToLines)
   
  -- if no linefeed in line return nothing
  if #lines == 0 then
    return ""
  end -- if
  
 -- this last part did not have a LF in it
  previousPacket = { LeftOver }
  
  -- return existing lines with the LF added back
  return table.concat (lines, "\n") .. "\n"
end  -- OnPluginPacketReceived

function OnPluginConnect ()
  previousPacket = {}
  buffer_packets = false
end -- OnPluginConnect

]]>
</script>

</muclient>

What this is doing is keeping a buffer, and adding new packets to it. When the buffer has newlines in it they are passed on to the client. This guarantees that the packets will end with a newline, and thus be processed quickly. Packets (such as you showed) with no newline are retained until a newline arrives.

To allow for the initial login prompts it waits for a line from the MUD "You have connected!" which you can amend to be something you see after you have logged in. Before that arrives the buffering is disabled, so you can see the name/password prompts.

Fiendish USA Global Moderator Fri 11 Oct 2019 06:10 PM #18

Quote:

Incoming packet: 344 (5 bytes) at Thursday, October 10, 2019, 5:22:44 PM

<BR>.              3c 42 52 3e 0a

Oof. Looks like I was right. What a terrible server design.

Nick Gammon Australia Forum Administrator Fri 11 Oct 2019 09:13 PM #19

I've rewritten the plugin to make it less confusing.

To save and install the Buffer_Lines plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Buffer_Lines.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Buffer_Lines.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Buffer_Lines"
   author="Nick Gammon"
   id="8eb1cd324d58655b0ff057db"
   language="Lua"
   purpose="Buffer packets so they have a newline in them"
   date_written="2019-10-11 17:27:00"
   date_modified="2019-10-12 08:06:00"
   requires="4.80"
   version="1.1"
   >
</plugin>

<!--  Trigger to show you have connected  -->

<triggers>
  <trigger
   enabled="y"
   match="You have connected!"
   script="we_connnected"
   sequence="100"
  >
  </trigger>
</triggers>



<!--  Script  -->

<script>
<![CDATA[

local previousPacket = ""
local buffer_packets = false

-- here when we log in
function we_connnected (name, line, wildcards)
   if not buffer_packets then
     ColourNote ("green", "", "Packet buffering now active")
   end -- if
   buffer_packets = true
end -- we_connnected

function OnPluginPacketReceived (packet)

  -- do nothing if we haven't logged in
  if not buffer_packets then
   return nil  -- this makes the client process the unchanged packet
  end -- not reached connection message

  -- add packet to previous ones
  previousPacket = previousPacket .. packet

  -- find last newline (the search for a newline is "greedy")
  sendThis, keepThis = string.match (previousPacket, "^(.*\n)([^\n]*)")

  -- if no linefeed in line return nothing
  if not sendThis then
    return ""
  end -- if

  -- save stuff past the newline for next time
  previousPacket = keepThis

  -- send up to the last newline
  return sendThis

end  -- OnPluginPacketReceived

function OnPluginConnect ()
  previousPacket = ""
  buffer_packets = false
end -- OnPluginConnect

]]>
</script>

</muclient>

Basically it works like this:

Take an incoming packet and add to the left-over packet from earlier
Search the combined packet (old ones plus the new one) for the last newline
If one found, send everything up to that newline to the client for processing
If none found, send nothing to the client
In any event, retain the excess bit at the end (which could be nothing if the newline was at the end)

There is still a test for the message "You have connected!" which indicates that you have moved past the login sequence of username/password. Edit that string to be something your MUD sends.

Alisa Fri 11 Oct 2019 09:28 PM #20

You have nooooo idea how much I can't wait to get back home so that I can test this out. Even if it doesn't solve the problem, you folks have done a tremendous amount of work to help me with this and I super appreciate it!

As for the connect message, it is actually something people say in chat on a fairly regular basis as a bit of a peculiar catchphrase. Will anything bad happen if someone says the line again after the plugin has already initialized for that session?

If so, I could always use the line it spits out stating when and where my last connection was from. I don't belieeeeeeve I've ever heard anyone use "Last connect was from" in normal conversation, although I suppose it could come up at some point if someone gets paranoid and quotes that line in chat when seeking help.

Nick Gammon Australia Forum Administrator Fri 11 Oct 2019 09:39 PM #21

Nothing bad will happen except you will see that message about "Packet buffering now active" again. You could always remove that, or ignore it.

Or you could change that function to be:


-- here when we log in
function we_connnected (name, line, wildcards)
   if not buffer_packets then
     ColourNote ("green", "", "Packet buffering now active")
   end -- if
   buffer_packets = true
end -- we_connnected

That way it only shows the message once per session.

Nick Gammon Australia Forum Administrator Fri 11 Oct 2019 09:41 PM #22

I amended the later version (1.1) to have that test, so you only see that message once.

Alisa Fri 11 Oct 2019 10:18 PM #23

Using "Last connect was from" did not seem to be starting the plugin for whatever reason. I switched it out for the oft-repeated line I was concerned about and updated the plugin to reflect the change you made, with a green "Packet buffering now active" line greeting me at my next connect.

It actually seems to have somehow caused chaos, probably related to < and > marks in displayed text. Exit names on the game have their shortcuts displayed in those (such as "Out <O>") and of course their usage in the channel names, ie <General>.

If I sit in place and execute the 'look' command a few times in a row, the very first one will display properly but then things go nuts. The <> around the exit shortcut are stripped out, as are those around the channel name, and are inserted in very strange places instead. Quotation marks are randomly appearing prepended to new channel lines as well.

https://puu.sh/ErtKa/6900b39830.png is an example screenshot so that you can see exactly how it looks on my end. As you can see, "Local Exits" has a bunch of >>> jammed into it (probably from the prior 'look' results), the <> are missing from the exit shortcut and channel names, and there are strange double quotes showing up before some of the channel lines but not all.

As for that random > before the first channel line after the look command (where the exit name is displayed), that happens each time I see an exit shortcut. The first line sent after that will have that random > stuck in there but then they stop until I see <> somewhere again.

Alisa Fri 11 Oct 2019 10:32 PM #24

It seems to also be stripping quotation marks from things! Both channel chatter and local room say. The double quotation marks prepended to lines seem to happen when the previous line had quotation marks stripped out of it, as they do not appear when the previous line was an emote without any quotation marks involved.

Alisa Fri 11 Oct 2019 10:43 PM #25

And one more tidbit! Apologies for this information trickle, I'm trying to provide anything which seems to be following a pattern.

Anywho, back to the quotation mark thing! It strips them out if they are present in a line, instead prepending the next line with each mark thusly removed. Interestingly, if I do something like @chan/recall that would result in a whole bunch of lines all at once, it will strip the quotation marks from the entire thing and then slap all of the missing ones to the start of the next new packet.

Or at least I think so. I tested it three times, with results that showed 6, 10, and then 8 quotation marks removed from each test, and their following packets each having 6, 10, and then 8 quotation marks prepended to them. Not the world's greatest sample size but it does seem to indicate a pattern to the seeming madness.

Nick Gammon Australia Forum Administrator Sat 12 Oct 2019 05:50 AM #26

Right. What I would like you to do is disable the plugin (there is a button on the plugin list for that) and then capture, say, 20 packets of the sort that are causing trouble. Chat, etc.

Post them here. Using that I can reproduce exactly what you are receiving and debug the problem.

Nick Gammon Australia Forum Administrator Sat 12 Oct 2019 05:51 AM #27

Which version did you use? The first one (1.0) or the second one (1.1).

Nick Gammon Australia Forum Administrator Sat 12 Oct 2019 09:14 AM #28

After extensive investigation (I could reproduce the problem) I have deduced that the MXP/Pueblo processing is interfering with the packet manipulation.

With the plugin installed (second version, 1.1) and with Pueblo off the artefacts of the weirdness at the start of the line goes away.

To briefly explain, when the MXP/Pueblo processing finds an entity (eg. < ) then it pretends it got "<" in a packet. This is then remembered in the code in the plugin, so that things like " in your packet are added to the "just received" packet stuff, and then appear at the start of the next line.

This looks like a bug in the interaction between the OnPluginPacketReceived and the MXP processing, however it has been around for a long time.

I suggest installing the second plugin, and disabling Pueblo.

Nick Gammon Australia Forum Administrator Sat 12 Oct 2019 09:18 AM #29

Quote:

... it will strip the quotation marks from the entire thing and then slap all of the missing ones to the start of the next new packet.

Pretty much what I would expect from the code. This is a bit kludgy but you are the first person to discover it (or perhaps, *I* am the first person to discover it, after writing the plugin).

To put that into context, MXP was implemented on 17th June 2001, and this is the first time this issue has arisen. :)

Nick Gammon Australia Forum Administrator Sat 12 Oct 2019 08:59 PM #30

I have altered the way the client handles MXP entities, so this problem should now go away, even if you have MXP or Pueblo enabled.

This change is in the 5.07 pre-release version which can be obtained by following the instructions here:

http://www.gammon.com.au/forum/?id=13903

Commit details are here:

https://github.com/nickgammon/mushclient/commit/6fd17d9

Alisa Sun 13 Oct 2019 01:29 AM #31

I am heading home now and will take a look shortly after I arrive! I'll post back here once I get settled in for the night.

Nick Gammon Australia Forum Administrator Sun 13 Oct 2019 04:39 AM #32

Just to be clear, the new version fixes the bug with Pueblo, but you still need the plugin if you want to combine multiple packets into one, to avoid the timing issue.

Alisa Sun 13 Oct 2019 04:46 AM #33

Good timing! I just settled into my office for the night and grabbed the latest 5.07-pre. At least I hope it is the latest! The upload showed that it was made almost three years ago but it was still somehow the most recent and specifically mentioned the MXP change, so I will assume the date is wonky.

As you said, I still noticed the odd timing on the omitted lines on the new version without the plugin. I've since added the plugin and reset everything, so now we play the waiting game.

On the plus side, yesterday's chaos with the < and > seem to be fixed! Channel names display fine, as do exit shortcuts. Now I will wait 20 or so minutes to see if any of the jerkiness still occurs and then make another post here.

Fiendish USA Global Moderator Sun 13 Oct 2019 05:27 AM #34

Quote:
The upload showed that it was made almost three years ago but it was still somehow the most recent

Yeah that's just a quirk of reusing the same release tag over and over again.

Alisa Sun 13 Oct 2019 05:51 AM #35

As an update, I am still watching. I think there is something amiss, as on two occasions during my dinner (while watching an idle MUSHclient window on the second monitor) a peculiar thing happened.

The displayed text hopped up a line as if something were appearing at the bottom but only a blank line appeared there, and when this happened the black 'MORE' flag appeared at the bottom of the client as if I were scrolled up and/or paused and not viewing the most recent text. Trying to scroll down at this time doesn't seem to do anything and the 'MORE' flag remains until something else finally comes through the pipe to be displayed so that I have something I can scroll down to and be 'caught up'.

I was delaying a bit before reporting this in the hopes of catching debug of the packets immediately prior to the quirk (it wasn't on at the time) but it hasn't happened since I enabled packet debug. Still watching though!

Nick Gammon Australia Forum Administrator Sun 13 Oct 2019 07:40 AM #36

What I am interested to know is: does the plugin largely (or almost completely) solve the original problem?

What you are describing sounds a bit odd - and possibly the code for displaying "MORE" is not totally working in conjunction with omitted lines. Possibly "MORE" is displayed and then with the line omitted it isn't taken away.

But without staring at the screen, looking for discrepancies, is the result satisfactory when just MUDding in general?

Nick Gammon Australia Forum Administrator Sun 13 Oct 2019 07:48 AM #37

There are complexities to making a client work in a way that people expect. As early as some time in 1996 (I think) the client changed from waiting for a newline to arrive before displaying text. This was to allow for the prompts sent out by Diku-style MUDs:

http://www.gammon.com.au/scripts/showrelnote.php?version=0.70&productid=0

Then in May 1997 the idea of "omit from output" was added:

http://www.gammon.com.au/scripts/showrelnote.php?version=2.00&productid=0

However the two things combined could conceivably introduce flicker if the entire message wasn't in a single packet.

You can see that this stuff has been around for over 20 years. :)

Alisa Wed 16 Oct 2019 12:48 PM #38

Nick Gammon said:

What I am interested to know is: does the plugin largely (or almost completely) solve the original problem?

What you are describing sounds a bit odd - and possibly the code for displaying "MORE" is not totally working in conjunction with omitted lines. Possibly "MORE" is displayed and then with the line omitted it isn't taken away.

But without staring at the screen, looking for discrepancies, is the result satisfactory when just MUDding in general?

Hello again! I haven't had much time to MUSH these past several days but it does seem like the original issue has been resolved by the combination of the 5.07-prerelease and the plugin you wrote.

I still want to try examining packet debug for the other issue when I have more time though, as I think it might cause trouble in some situations. Whenever that blank line suddenly appears at the bottom of the output screen, it will go away and correct itself on its own if you leave it be for long enough.

On the other hand, if you were to try scrolling down or otherwise correct the output display, you end up with that 'MORE' scenario I described before. Once that happens, your output won't scroll down even if you have it set to automatically unpause on activity. It will freeze like that and not show any new output until A) there is something new to show and B) you manually scroll down to force this new output to display.

So it's far from a dealbreaker but it could cause trouble for those who might be waiting for new activity while watching TV or otherwise doing a thing to pass the time -- you could have 3000 lines of new activity, yet you won't see a thing until you manually punt the screen downward. Even if you're actively watching for activity, you will still have to tap page down from time to time until a new line arrives and the resulting scrolling down temporarily corrects the odd pausing behavior.

To be clear, this rarely seems to happen, maybe once or twice an hour. With such rarity, I should be able to reliably get some packet debug for whatever packets immediately preceded the issue, which will hopefully be enough to isolate the cause. Just a matter of my getting some good ol' lazy time soon! I'll post back here again once that happens and I have some mystical mumbo jumbo code sorcery to share.

Beyond that, thank you muchly for solving the original problem! It is soooooooo much nicer now, as I am usually doing some work on the primary monitor while MUSHclient sits open on the second monitor. No matter how hard I tried to train myself to ignore that second screen constantly jumping up and down, I never managed to do it and my focus was constantly being disturbed as a result, not to mention my motion sickness! Hopefully this means I can now get more work done without having to minimize MUSHclient again.