Setting a table within a table using serialized variables

Posted by Charneus on Tue 03 Jun 2014 03:53 PM — 9 posts, 34,606 views.

Charneus Tue 03 Jun 2014 03:53 PM #0

I'm pretty sure my subject was quite vague, but due to the limits, I tried to explain what the situation is.

I'm running a tracking script to keep track of my gains throughout levels. It's another script I've been converting over from CMUD, and really is my pride and joy. Since I wanted to save all the information over time (and not really partial to CMUD's own zscript variables), I saved the tables to a .tbl file, which I could load up at any given time. This is akin to the serialize.save function you have.

Here's where my issue lies:

The whole of the database is called: trackerdb. Within trackerdb, there are top-level keys based on player name (useful for keeping track of individual alts progress). So, for instance, I would have a table of trackerdb["Charneus"], which has all of Charneus's stats, and trackerdb["Calamer"], which has all of Calamer's stats.

I can't seem to replicate this in MUSH, however, using serialize and setfenv. Due to the length of code, I'm going to just provide snippets so you can get the general idea.

What the serialized variable contains:

ReportChan = "gt"
  Quests = {}
    Quests.LastQuest = {}
      Quests.LastQuest.Pracs = 0
      Quests.LastQuest.Qp = 0
      Quests.LastQuest.Gold = 0
      Quests.LastQuest.Trivia = 0
      Quests.LastQuest.Trains = 0
    Quests.Total = {}
      Quests.Total.Pracs = 0
      Quests.Total.Trains = 0
      Quests.Total.Qp = 0
      Quests.Total.Completed = 0
      Quests.Total.Gold = 0
      Quests.Total.Failed = 0
      Quests.Total.BonusQp = 0
      Quests.Total.Trivia = 0
    Quests.Report = ""

What the trackerdb should serialize as after a new player has been initiated:


trackerdb = {}
  trackerdb.Charneus = {}
    trackerdb.Charneus.ReportChan = "gt"
      trackerdb.Charneus.Quests = {}
        trackerdb.Charneus.Quests.LastQuest = {}
        trackerdb.Charneus.Quests.LastQuest.Pracs = 0
        trackerdb.Charneus.Quests.LastQuest.Qp = 0
        trackerdb.Charneus.Quests.LastQuest.Gold = 0
        trackerdb.Charneus.Quests.LastQuest.Trivia = 0
        trackerdb.Charneus.Quests.LastQuest.Trains = 0
      trackerdb.Charneus.Quests.Total = {}
        trackerdb.Charneus.Quests.Total.Pracs = 0
        trackerdb.Charneus.Quests.Total.Trains = 0
        trackerdb.Charneus.Quests.Total.Qp = 0
        trackerdb.Charneus.Quests.Total.Completed = 0
        trackerdb.Charneus.Quests.Total.Gold = 0
        trackerdb.Charneus.Quests.Total.Failed = 0
        trackerdb.Charneus.Quests.Total.BonusQp = 0
        trackerdb.Charneus.Quests.Total.Trivia = 0
      trackerdb.Charneus.Quests.Report = ""

Unfortunately, all I wind up getting is a super mess. The code I'm attempting to use is:


function init_tracker(pname, reset)
  -- trackertemplate = {} -- original code
  trackerdb[pname] = {} -- second try
  setfenv (assert( loadstring( GetVariable("trackerdbtemplate"))), trackerdb[pname]) () -- trackerdb[pname] was trackertemplate on original code
  --trackerdb[pname]=trackertemplate -- original code
  local ttable = trackerdb[pname]
  local resettime = socket.gettime()
  for k,_ in pairs(ttable["Timers"]) do
    ttable["Timers"][k]["Start"] = resettime
  end
  ttable["HasPupped"]=0
  if reset then print(pname,"has been reset!")
  else
    print(pname,"has been initialized!")
  end
  SetVariable("trackerdb", serialize.save("trackerdb"))
end

So, is there an easy way to pull this off? Thanks!

Daniel P USA Tue 03 Jun 2014 06:32 PM #1

I assume you read the post on Seralization:

Please see the forum thread: http://gammon.com.au/forum/?id=4960.

I never managed to understand it completely, but I'm sure that comparing with Nick's example can help with debugging what you have so far.

As an alternative, how about encoding/decoding as JSON? There's a library built in.


json = require("json")
strQuests = json.encode(Quests)
print("JSON string to store: " .. strQuests)
Quests = json.decode(strQuests)

Nick Gammon Australia Forum Administrator Tue 03 Jun 2014 07:58 PM #2

What you are doing looks complicated.

Using the link Daniel mentioned I do this to serialize:


trackerdb = { }

trackerdb.ReportChan = "gt"
trackerdb.Quests = {}

Quests = trackerdb.Quests   -- copy by reference

    Quests.LastQuest = {}
      Quests.LastQuest.Pracs = 0
      Quests.LastQuest.Qp = 0
      Quests.LastQuest.Gold = 0
      Quests.LastQuest.Trivia = 0
      Quests.LastQuest.Trains = 0
    Quests.Total = {}
      Quests.Total.Pracs = 0
      Quests.Total.Trains = 0
      Quests.Total.Qp = 0
      Quests.Total.Completed = 0
      Quests.Total.Gold = 0
      Quests.Total.Failed = 0
      Quests.Total.BonusQp = 0
      Quests.Total.Trivia = 0
    Quests.Report = ""

require "serialize"  -- needed to serialize table to string

SetVariable ("trackerdb", "trackerdb = " .. serialize.save_simple (trackerdb))

-- debugging
print (GetVariable ("trackerdb"))

Debugging output (the serialized variable) is:


trackerdb = {
  Quests = {
    Report = "",
    Total = {
      Completed = 0,
      Trivia = 0,
      Pracs = 0,
      Trains = 0,
      BonusQp = 0,
      Gold = 0,
      Failed = 0,
      Qp = 0,
      },
    LastQuest = {
      Pracs = 0,
      Gold = 0,
      Trains = 0,
      Trivia = 0,
      Qp = 0,
      },
    },
  ReportChan = "gt",
  }

You do the serializing in OnPluginSaveState (so it gets written to disk).

To get the table back next time (eg. in the function OnPluginInstall) you just do this:


trackerdb = nil -- clear earlier value

assert (loadstring (GetVariable ("trackerdb") or "")) ()

-- debugging
require "tprint"
tprint (trackerdb)

Debugging print:


"Quests":
  "Report"=""
  "Total":
    "Completed"=0
    "Trains"=0
    "Pracs"=0
    "Trivia"=0
    "BonusQp"=0
    "Gold"=0
    "Failed"=0
    "Qp"=0
  "LastQuest":
    "Pracs"=0
    "Gold"=0
    "Trains"=0
    "Trivia"=0
    "Qp"=0
"ReportChan"="gt"

Now I have the table back.

Nick Gammon Australia Forum Administrator Tue 03 Jun 2014 08:00 PM #3

After reloading, if you want to use the Quests table (rather than trackerdb.Quests) just make another copy:


Quests = trackerdb.Quests   -- copy by reference

Lua copies tables by reference, not by value, so this copy refers to the original table.

Nick Gammon Australia Forum Administrator Tue 03 Jun 2014 08:05 PM #4

Re-reading your post I see you have another level of nesting, but that is easy enough to handle:


trackerdb = { }

trackerdb.Charneus = { }


trackerdb.Charneus.ReportChan = "gt"
trackerdb.Charneus.Quests = {}

Quests = trackerdb.Charneus.Quests   -- copy by reference

    Quests.LastQuest = {}
      Quests.LastQuest.Pracs = 0
      Quests.LastQuest.Qp = 0
      Quests.LastQuest.Gold = 0
      Quests.LastQuest.Trivia = 0
      Quests.LastQuest.Trains = 0
    Quests.Total = {}
      Quests.Total.Pracs = 0
      Quests.Total.Trains = 0
      Quests.Total.Qp = 0
      Quests.Total.Completed = 0
      Quests.Total.Gold = 0
      Quests.Total.Failed = 0
      Quests.Total.BonusQp = 0
      Quests.Total.Trivia = 0
    Quests.Report = ""

require "serialize"  -- needed to serialize table to string

SetVariable ("trackerdb", "trackerdb = " .. serialize.save_simple (trackerdb))

-- debugging
print (GetVariable ("trackerdb"))

Output:


trackerdb = {
  Charneus = {
    Quests = {
      Report = "",
      Total = {
        Completed = 0,
        Trivia = 0,
        Pracs = 0,
        Trains = 0,
        BonusQp = 0,
        Gold = 0,
        Failed = 0,
        Qp = 0,
        },
      LastQuest = {
        Pracs = 0,
        Gold = 0,
        Trains = 0,
        Trivia = 0,
        Qp = 0,
        },
      },
    ReportChan = "gt",
    },
  }

Reloading is the same:


trackerdb = nil

assert (loadstring (GetVariable ("trackerdb") or "")) ()

require "tprint"

tprint (trackerdb)

Output:


"Charneus":
  "Quests":
    "Report"=""
    "Total":
      "Completed"=0
      "Trains"=0
      "Pracs"=0
      "Trivia"=0
      "BonusQp"=0
      "Gold"=0
      "Failed"=0
      "Qp"=0
    "LastQuest":
      "Pracs"=0
      "Gold"=0
      "Trains"=0
      "Trivia"=0
      "Qp"=0
  "ReportChan"="gt"

Charneus Tue 03 Jun 2014 10:24 PM #5

Well, I've changed things over to the serialize.save_simple format, and I still keep creating a mess. The trackerdb, once serialized with one character, should only be 434 lines. However, it seems to do something similar to the following:


trackerdb = {
  Charneus = {
  ...
  },
  trackerdb = {
    Charneus = {
    ...
    },
    trackerdb = {
      Charneus = {
      ...
      },
    },
  },
}

And I wind up with a variable that is literally 2,610 lines long.

Here's the code as it stands now:


function init_tracker(pname, reset)
  assert( loadstring (GetVariable("trackerdbtemplate") or "")) ()
  trackerdb[pname] = {}
  trackerdb[pname] = trackertemplate
  local resettime = socket.gettime()
  for k,_ in pairs(trackerdb[pname]["Timers"]) do
    trackerdb[pname]["Timers"][k]["Start"] = resettime
  end
  trackerdb[pname]["HasPupped"]=0
  if reset then print(pname,"has been reset!")
  else
    print(pname,"has been initialized!")
  end
  SetVariable("trackerdb", "trackerdb = " .. serialize.save_simple(trackerdb))
end

trackertemplate loads properly - I debugged it with tprint. I'm really at a loss of what to do. If you need to see the variables directly, let me know and I can pastebin them.

Ultimately, I want trackerdb to load in the following manner:


trackerdb = {
  Charneus = {
  ...
  },
  Calamer = {
  ...
  },
  SomeAlt = {
  ...
  }
}

I hope that's possible. :\ Thanks!

Nick Gammon Australia Forum Administrator Tue 03 Jun 2014 11:02 PM #6

An alternative to serializing variables like that is to use the SQLite3 database interface.

Here is an example. First some helper functions:


local MUSHclient_Database_Errors = {
  [-1] = "Database id not found",
  [-2] = "Database not open",
  [-3] = "Already have prepared statement",
  [-4] = "Do not have prepared statement",
  [-5] = "Do not have a valid row",
  [-6] = "Database already exists under a different disk name",
  [-7] = "Column number out of range",
  } -- end of MUSHclient_Database_Errors

-- check for errors on a DatabaseXXXXX call
function dbcheck (code)

 if code == sqlite3.OK or       -- no error
    code == sqlite3.ROW or      -- completed OK with another row of data
    code == sqlite3.DONE then   -- completed OK, no more rows
    return code
  end -- if code OK

  -- DatabaseError won't return the negative errors  
  local err = MUSHclient_Database_Errors [code] or DatabaseError(db)
  DatabaseExec (db, "ROLLBACK")  -- rollback any transaction to unlock the database
  error (err, 2)                 -- show error in caller's context

end -- dbcheck 

-- Quote the argument, replacing single quotes with two lots of single quotes.
-- If nil supplied, return NULL (not quoted).
function fixsql (s)
  if s then
    return "'" .. (string.gsub (s, "'", "''")) .. "'"
  else
    return "NULL"
  end -- if
end -- fixsql

Open the database (creating it if necessary):


-- open database on disk
databasename = GetInfo (66) .. "tracker.db"  -- in MUSHclient directory
db = "db"  -- just an internal identifier
DatabaseOpen (db, databasename, 6)  -- Open_ReadWrite + Open_Create (6)

Create a couple of tables if they don't exist:


-- create the table
dbcheck (DatabaseExec (db, [[
  CREATE TABLE IF NOT EXISTS players(
    PlayerID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
    Name TEXT UNIQUE,
    Class TEXT
  )
]]))

-- create the LastQuest table
dbcheck (DatabaseExec (db, [[
  CREATE TABLE IF NOT EXISTS LastQuest(
    PlayerID INTEGER NOT NULL PRIMARY KEY,
    Pracs INTEGER,
    Qp INTEGER,
    Gold INTEGER,
    Trivia INTEGER,
    Trains INTEGER
   )
]]))

Find the player in the database (Charneus in this case) and if not found, add him:


-- add a player (if necessary)

-- test data
playerName = "Charneus"
playerClass = "Mage"

-- see if player already exists
PlayerID = DatabaseGetField (db, 
          string.format ("SELECT PlayerID FROM players WHERE Name = %s", fixsql (playerName)))

-- not on database, add it
if PlayerID == nil then
  dbcheck (DatabaseExec (db, string.format ("INSERT INTO Players (Name, Class) VALUES (%s, %s)", 
          fixsql (playerName),
          fixsql (playerClass))))
  PlayerID = tonumber (DatabaseLastInsertRowid (db))
  -- add some quest stats
  dbcheck (DatabaseExec (db, (string.format (
    "INSERT INTO LastQuest (PlayerID, Pracs, Qp, Gold, Trivia, Trains) " ..
    "VALUES (%i, 0, 0, 0, 0, 0)", PlayerID ))))
end -- if

print ("PlayerID =", PlayerID)

We also added a record into LastQuest with zero for each field.

Now update the LastQuest table with the current info:


-- test data
Pracs = 22
Qp = 33
Gold = 44
Trivia = 55
Trains = 666

-- update quest stats
dbcheck (DatabaseExec (db, (string.format (
  "UPDATE LastQuest SET Pracs = %i, Qp = %i, Gold = %i, Trivia = %i, Trains = %i " ..
  "WHERE PlayerID = %i", 
  Pracs, Qp, Gold, Trivia, Trains,  -- new data
  PlayerID ))))  -- primary key

Find the current LastQuest information:


-- get data back

-- prepare a query
dbcheck (DatabasePrepare (db, string.format ("SELECT * from LastQuest WHERE PlayerID = %i", PlayerID)))

-- find the column names
names = DatabaseColumnNames (db)

-- execute to get the first row
rc = DatabaseStep (db)  -- read first row

-- now loop, displaying each row, and getting the next one
while rc == sqlite3.ROW do
  
  print ("")
  values = DatabaseColumnValues (db)
 
  for k, v in ipairs (names) do
    print (v, "=", values [k])
  end -- for

  rc = DatabaseStep (db)  -- read next row

end -- while loop

-- finished with the statement
DatabaseFinalize (db)

Output from above:


Pracs = 22
Qp = 33
Gold = 44
Trivia = 55
Trains = 666

When done, close the database:


DatabaseClose (db)  -- close it

This is more complex than serializing tables, but more flexible perhaps in the long run. Also once you write to the database the data is on disk, even if the program crashes.

The Aardwolf mapper, for example, uses SQLite3 for its mapper database, and that is nice and fast.

Charneus Tue 03 Jun 2014 11:43 PM #7

I may look into that option, Nick.

As it turns out, I solved my issue of the multiple nested tables. It was a simple case of forgetting to update my OnPluginInstall, where I was still using setfenv instead of assert to load the trackerdb. Seems to be working fine for now.

Thanks again for all the help! Truly is wonderful to receive proper support when I can't figure something out.

Nick Gammon Australia Forum Administrator Wed 04 Jun 2014 12:11 AM #8

Glad it worked.


  assert( loadstring (GetVariable("trackerdbtemplate") or "")) ()

Strictly speaking, "assert" doesn't load the table, that simply checks for errors. It is "loadstring" that does the heavy work. The variable you pass to that is simply executed as code, and as it has a table definition in it, that table gets created. The assert simply warns you if it fails.