MDB_MAP_FULL error writing table to storage

[richK]

We are running into a problem when writing a table to storage using storage.set_table(). The error is: “MDB_MAP_FULL: Environment mapsize limit reached”. We hit this limit when the table gets to be about 62,000 bytes.

The MDB_MAP_FULL is a LMDB error, and means the maximum size in bytes of the DB has been reached, but 62,000 seams a very small value.

Has anyone else run into this issue? Is there a way to increase the map size?

[Santiago_Gutierrez]

Hello Richard,

We are sorry to hear you’re running into this issue. As you mentioned the default LMDB map size is in fact 64KB which is in accordance with the table size at which you’re encountering this error. The map size can be adjusted from your environment setup when using the Lua wrapper for the LMDB.
According to the LMDB documentation an environment should be initialized as follows:

lmdb. Environment ( path , map_size=10485760 , subdir=True , readonly=False , metasync=True , sync=True , map_async=False , mode=493 , create=True , readahead=True , writemap=False , meminit=True , max_readers=126 , max_dbs=0 , max_spare_txns=1 , lock=True )

where path is the directory whithin the file system where the database is to be allocated and map_size the max size in bytes that the database is allowed to grow to. By default this parameter is set to low to encourage a crash so that you can figure out a good size to work with.

To offer you further assistance, can you please provide a code snippet of the script that’s causing this error?

Regards,

Santiago Gutierrez
Customer Care

[richK]

The code generating this error is basically:

local storage = require “storage”
storage.set_table(storage_key, datatable)

How would the plugin get access to the lmdb library? require “lmdb” doesn’t work. Nor does
the storage API doesn’t expose the lmdb environment or the path to the database.