Lua debug DLL

Posted by Nick Gammon on Sun 03 Oct 2010 10:38 PM — 1 posts, 8,828 views.

Australia Forum Administrator #0
I have spent most of yesterday debugging some obscure code problems in MUSHclient. I might have saved some time if I had a debug Lua DLL, so I have now made one.

This is available for download from here:

http://www.gammon.com.au/files/mushclient/lua5.1d.zip

Inside is a file lua5.1d.dll - you would need to copy that to where you are testing MUSHclient and rename it as lua5.1.dll (in my case the Windebug directory).

The difference is that LUA_USE_APICHECK is defined, which enables the checking of various parameters during Lua calls. These will assert if out of range.


#define LUA_USE_APICHECK
#if defined(LUA_USE_APICHECK)
#include <assert.h>
#define luai_apicheck(L,o)      { (void)L; assert(o); }
#else
#define luai_apicheck(L,o)      { (void)L; }
#endif


The reason for this is the time spent to debug this issue ...

I was testing doing function callbacks, and to make sure I had a valid function on the Lua stack was doing this:


  if (lua_isfunction (L, -1))
    {
    ... call the function ...
    }


Unfortunately this was randomly failing and doing strange things, such as nothing, or calling some unexpected other function.

This function effectively calls lua_type:


#define lua_isfunction(L,n)	(lua_type(L, (n)) == LUA_TFUNCTION)


lua_type is documented to return LUA_TNONE for a non-valid index (that is, an index to an "empty" stack position).

However there is an interesting trap. Consider this:


  lua_settop (L, 0);     // no Lua stack

  x = lua_type (L, 1);   // returns -1 (LUA_TNONE)
  y = lua_type (L, -1);  // returns random data


It turns out that lua_type (and indeed other functions) expect an "acceptable index" which is:


(index < 0 && abs(index) <= top) ||
(index > 0 && index <= stackspace)


Thus an index of 1 is acceptable (but invalid) however an index of -1 is not acceptable - in the situation where the stack is empty.

The debug build of the lua5.1.dll now asserts - so that in this situation I would have got an assertion to alert me to the fact that the index of -1 was unacceptable if the stack was empty.

The reason the stack was empty in the first place is another story, but that caused the issue I describe.

Since the test for an index of -1 returned random data, sometimes it returned 6 (which indicates a function) and sometimes other things.

So if you are playing with the source, and mucking around with Lua, the debug DLL may save some heartache.