Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to "verify" your details, making threats, or asking for money, are
spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the
password reset link.
Entire forum
➜ MUSHclient
➜ Plugins
➜ Learning Mapper Database getting too big
|
Learning Mapper Database getting too big
|
It is now over 60 days since the last post. This thread is closed.
Refresh page
| Posted by
| Faedara
(106 posts) Bio
|
| Date
| Fri 28 Oct 2022 05:52 AM (UTC) |
| Message
| I'm having trouble with the learning mapper causing bloat, plugin failing to load, and subsequently crashing the plugin and deleting the entire database. The MUD I'm playing on (AwakeMUD) is rather large, admittedly, and room descriptions can be very descriptive, leading to 2k+ rooms added to the database.
Is there some method I can use to avert this crash, and subsequent loss of the map? I've been combing the forums for weeks and not found a solution. |
The eternally clueless <3
Currently looking for a mentor/tutor to help me learn everything about Lua from the ground up. I can't understand 28*41 until I learn what causes 2+2. | | Top |
|
| Posted by
| Nick Gammon
Australia (23,070 posts) Bio
Forum Administrator |
| Date
| Reply #1 on Sat 29 Oct 2022 06:41 AM (UTC) |
| Message
| You shouldn't lose everything. At the very least I would suggest making a backup of the mapper data (the state file) which you will find in the plugins "state" directory.
Its name will have "99c74b2685e425d3b6ed6a7d" in it.
For example, under Wine, the directory is:
~/.wine/drive_c/Program Files (x86)/MUSHclient/worlds/plugins/state
My state file is:
04cbe372cc03045372494c73-99c74b2685e425d3b6ed6a7d-state.xml
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
My world ID The plugin ID
Yours will have a different hex number at the start, which is the unique ID of your world file.
If you copy this somewhere else then you have saved the "good" file from your last run. Of course, don't copy it if the file seems corrupted. Preferably make a script to copy each time under a different name with the date/time as part of the name. Then you can always revert back to a good copy of it.
If things go wrong, put the copy back under its original name.
For a longer term solution I'll look at storing the mapper database as an SQLite3 database rather than into a plugin state file. That should be much more robust. However that may take a few days to write and debug, so as a temporary solution I would be making backups. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Fiendish
USA (2,533 posts) Bio
Global Moderator |
| Date
| Reply #2 on Sat 29 Oct 2022 01:48 PM (UTC) Amended on Sat 29 Oct 2022 01:51 PM (UTC) by Fiendish
|
| Message
| | Side note: The state file maximum size limit isn't very large (looks like 5mb? https://github.com/nickgammon/mushclient/blob/6b79a6d88ea811f797b143623e839eed07fa9802/xml/xmlparse.h#L8). I had to switch from storing intermediate data in the state file to storing it in a separate file for one of my plugins because people were experiencing crashes during reload of my chat capture plugin when they had a large amount of text captured. https://github.com/fiendish/aardwolfclientpackage/commit/e2d79fcb3d064904619d86781a2b664b5d069044 |
https://github.com/fiendish/aardwolfclientpackage | | Top |
|
| Posted by
| Nick Gammon
Australia (23,070 posts) Bio
Forum Administrator |
| Date
| Reply #3 on Sat 29 Oct 2022 08:43 PM (UTC) |
| Message
| | Maybe we should make it larger as an interim solution? |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Fiendish
USA (2,533 posts) Bio
Global Moderator |
| Date
| Reply #4 on Sat 29 Oct 2022 11:20 PM (UTC) |
| Message
| | We could. What's a good size though in 2022? 100? 200? |
https://github.com/fiendish/aardwolfclientpackage | | Top |
|
| Posted by
| Fiendish
USA (2,533 posts) Bio
Global Moderator |
| Date
| Reply #5 on Sat 29 Oct 2022 11:24 PM (UTC) Amended on Sun 30 Oct 2022 12:41 AM (UTC) by Fiendish
|
| Message
| |
| Posted by
| Fiendish
USA (2,533 posts) Bio
Global Moderator |
| Date
| Reply #6 on Sun 30 Oct 2022 12:48 AM (UTC) |
| Message
| |
| Posted by
| Faedara
(106 posts) Bio
|
| Date
| Reply #7 on Sun 30 Oct 2022 04:42 AM (UTC) |
| Message
| Wow, ya'll are an incredible team. I was expecting something along the lines of, "You forgot to update how much allocation is applied by this setting", not "I updated the client within a day to solve your problem, and we're rewriting the plugin to prevent similar future problems."
Absolutely astounding~ <3 Thank you. |
The eternally clueless <3
Currently looking for a mentor/tutor to help me learn everything about Lua from the ground up. I can't understand 28*41 until I learn what causes 2+2. | | Top |
|
| Posted by
| Nick Gammon
Australia (23,070 posts) Bio
Forum Administrator |
| Date
| Reply #8 on Sun 30 Oct 2022 06:00 AM (UTC) |
| Message
| | Fiendish's change is to allow slightly more than 20 times as much when reading an XML file. 100 MB to be precise. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Nick Gammon
Australia (23,070 posts) Bio
Forum Administrator |
| Date
| Reply #9 on Sun 30 Oct 2022 06:02 AM (UTC) |
| Message
|
Fiendish said:
But it seems so wasteful. The plugin should probably just be updated to write to plain files instead of the state file.
I want to do it with a minimal amount of extra work changing the plugin (because I am lazy). Perhaps serializing the plugin data (to a disk file) rather than putting it into a state file would be the simplest solution. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Nick Gammon
Australia (23,070 posts) Bio
Forum Administrator |
| Date
| Reply #10 on Sun 30 Oct 2022 06:53 AM (UTC) |
| Message
| I don't think it is particularly wasteful. That line that you changed is just a sanity check to not load too large files. The memory isn't actually allocated until the file is read in.
And if it is serialized a different way it will still take the same amount of memory. Unless it is rewritten to use the SQL database, which would be quite a lot of work. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).
To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.
12,678 views.
It is now over 60 days since the last post. This thread is closed.
Refresh page
![[Go to top]](/images/gototop.gif)
top