Doing some research can save you a lot of work: while looking around for a way to use SQLite as a persistent backing store for a Python dictionary I found at least two decent implementations. This blog post are my research notes.
Requirements
- pure Python, to ensure cross platform portability
- no additional external dependencies, to facilitate easy packaging
- portable data back-end format
- thread safe
- well written
- well documented
- open source
The requirement to have a portable back-end format makes SQLite based implementations such a strong preference as SQLite interfaces are available in a number of other programming languages as well, notably C
Thread safety is a strong requirement because I want to use this solution in CherryPy. It is not enough to restrict access to an object with some form of locking because if some database activity takes place, for example database transactions, this activity itself must be multithreading proof. Some database engines in Python are thread safe and SQLite can be made to work in a multithreading environment as long as each thread has its own connection. Check this post to see how this may be accomplished.
Python's shelve module
Python's shelve module is Python specific and not thread safe.
Seb Sauvages's dbdict
Seb Sauvages's dbdict is an interesting starting point although not thread safe
Erez' FileDict
Erez' FileDict is a more complete implementation but not thread safe either.
Tokyo Cabinet
Tokyo Cabinet feels a bit too complex for my taste and is another package that is not thread safe
Preliminary conclusion
Finding a thread safe solution to providing a persistent database backing for Python dictionaries is not as easy as I hoped. Finding one that meets my exact requirements may take more time than writing something from scratch which is a bit of a disappointment.