Gammon Forum : SMAUG : Lua : Implementing Lua: Part 2?

Entire forum ➜ SMAUG ➜ Lua ➜ Implementing Lua: Part 2?

Implementing Lua: Part 2?

It is now over 60 days since the last post. This thread is closed. Refresh page

Posted by

Kasji (35 posts) Bio

Date

Sat 03 Sep 2016 05:10 AM (UTC)

Message

Greetings.

It has been a fair bit of time since I was last active. Quite the hiatus, but I'm hoping to jump back on the Lua bandwagon.

Previously, in my implementation of Lua, I had succeeded in making an interface similar to Aardwolf's, using metatables.

However, a very sly bug arose that I was unable to solve at the time (and actually still haven't solved.) With the use of coroutines, I had a system which could process a list of yielded Lua progs, which would be added to the list using hooks/triggers similar to the stock mobprog system (speech triggers, etc).

This system actually was running well until the bug appeared. It had a tendency to appear when I triggered a massive number of scripts in a short span of time. The mud would crash, but the crash actually was occurring inside of Lua according to the backtrace.

I'm still pretty much clueless as to why this was happening, however, I am hoping that Lua 5.3 might have fixed the issue, if it really was a Lua issue.

I am a bit curious if anyone out there has implemented Lua in a similar fashion. Running Lua scripts as a replacement to mobprogs, from C?

Top

Posted by

Kasji (35 posts) Bio

Date

Reply #1 on Sat 03 Sep 2016 08:24 AM (UTC)

Message

Just had a lucky break in discovering the reason for this crash inside lua_resume().

The reason is.... garbage collection!

What I've done now is, when a thread is created via lua_newthread(), I turn off garbage collection. When I'm done with the thread, I turn it back on, and no more crashes! I'm going to release this implementation soon, but for now, it is bedtime.

Top

Posted by

Kasji (35 posts) Bio

Date

Reply #2 on Sat 03 Sep 2016 02:12 PM (UTC)

Amended on Sat 03 Sep 2016 02:14 PM (UTC) by Kasji

Message

I've done a bit more work today, and it turns out that the garbage collector, although responsible for the crash, isn't the root problem. It turns out that if you load/run code (luaL_dofile, etc) in your main lua state, but run it from threads (specifically from multiple instances), it can lead to this situation. You have to run luaL_dofile or whatever on the thread state rather than the global state.

Looking at this now seems like common sense, so it's a bit unfortunate to have overlooked it. But I suppose better late than never?

As promised, the code:
http://www.smaugmuds.org/files/Lua-replace-mprogs-486/

Also, I've included Nick's objectmap.h code. I take no credit for this wonderful piece.

Top

Posted by

Fiendish USA (2,534 posts) Bio Global Moderator

Date

Reply #3 on Sat 03 Sep 2016 02:34 PM (UTC)

Amended on Sat 03 Sep 2016 02:35 PM (UTC) by Fiendish

Message

Welcome back!

https://github.com/fiendish/aardwolfclientpackage

Top

Posted by

Nick Gammon Australia (23,133 posts) Bio Forum Administrator

Date

Reply #4 on Sat 03 Sep 2016 10:41 PM (UTC)

Message

Glad you sorted it. It sounds like a subtle problem. Perhaps it is fixed in more recent versions of Lua. I don't remember reading about whether or not you should use doFile inside a coroutine.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

Top

Posted by

Nick Gammon Australia (23,133 posts) Bio Forum Administrator

Date

Reply #5 on Sat 03 Sep 2016 10:54 PM (UTC)

Message

Can you make a minimal test that reproduces the issue? I could submit it to the Lua developers if the bug is still there.

Is the collectgarbage done in the coroutine or outside?

I can't quite envisage the sequence you set up to cause this, but if you can make a 10-line program that reliably crashes, that would be interesting.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

Top

Posted by

Kasji (35 posts) Bio

Date

Reply #6 on Sun 04 Sep 2016 04:38 AM (UTC)

Amended on Sun 04 Sep 2016 04:43 AM (UTC) by Kasji

Message

Well, I might try to later, but I'd prefer to focus on mud related stuff. Here's the steps that cause the problem:

0) lua_command(ch, command, argument)

1) Check if file exists: LUA_COMMANDS_DIR "do_%s.lua"

2) Prepare Lua environment (load libs, etc using global Lua state)

3) luaL_dofile(global state, file) (doesn't run the command yet, because I decided commands should be inside a function in Lua)

3) Create thread state

4) Prepare ch metatable and push on stack, push argument or nil on stack.

5) Call do_%s function with lua_getglobal(thread state, funcname) and lua_resume(thread state, NULL, nargs)

6) Script yields and gets resumed later.

7) When at least one script finishes, eventually the garbage collector comes to collect on the loaded script from luaL_dofile that was done on the global state.

8) Any existing (unfinished) scripts of the same file crash when resumed.

So, basically, I was mixing global state and thread states, when everything should have been done on the thread state (in its own memory space). With the above, if you run a script that yields (it has to yield and be resumed for the crash to happen) it will crash when the garbage collector collects. You actually don't have to wait for this to happen either. If you call lua_gc(any state, LUA_GCCOLLECT, 0) after the first instance of the script finishes, any other instances that get resumed will crash.

So, to simplify this into a test case, I'd say...

1) luaL_dofile on the global state.
2) spawn 2 threads
3) lua_resume both threads
4) yield both threads
5) resume and finish 1 thread
6) let the GC collect
7) resume the other thread

I imagine this could even be simplified further. Just take 1 thread, start it, yield it, GC collect, and try resuming.

[EDIT] I do all Lua stuff from C. Scripts are just for mprogs or mud commands basically.

Top

Posted by Kasji (35 posts) Bio

Date

Reply #7 on Fri 09 Sep 2016 06:13 AM (UTC)

Amended on Fri 09 Sep 2016 06:20 AM (UTC) by Kasji

Message

This is a little something I came up with just now. Might be handy to someone.

#include <typeid>
#include <typeindex>

template<class T>
void lua_pushKeyValuePair(lua_State * L, const char * key, T& value)
{
   std::type_index typeT;

   if (!L || !key)
      return;

   typeT = typeid(value);

   switch (typeT)
   {
      case typeid(bool):
         lua_pushboolean(L, value);
         break;
      case typeid(short):
      case typeid(int):
      case typeid(long):
      case typeid(long long):
         lua_pushinteger(L, value);
         break;
      case typeid(float):
      case typeid(double):
      case typeid(long double):
         lua_pushnumber(L, value);
         break;
      case typeid(char *):
         if (!((const char*)value))
            lua_pushnil(L);
         else
            lua_pushlstring(L, value, strlen((const char*)value));
         break;
      case typeid(lua_CFunction):
         if (!value)
            lua_pushnil(L);
         else
            lua_pushcfunction(L, value);
         break;
      case typeid(void *):
         if (!((void*)value))
            lua_pushnil(L);
         else
            lua_pushlightuserdata(L, value);
         break;
      default:
         // Possibly use lua_pushlightuserdata?
         luaL_error(L, "lua_pushKeyValuePair: Unsupported type: %s", typeT.name);
         return;
   }

   lua_setfield(L, key);

   return;
}

Think it'll actually work? Gonna do some testing...

[EDIT] This requires: -std=c++11 or -std=gnu++11

Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

22,732 views.

It is now over 60 days since the last post. This thread is closed. Refresh page

Go to topic: Search the forum

top