Python metaclasses

A basic object relational design

In this article we will explore metaclasses as a tool to link up Python classes and database tables.

Based in part on the information in illustrated in this
Python article.

Metaclasses are a daunting concept at first, yet it is worthwhile to understand the idiom because in some circumstances the are vital in getting hard jobs done. In this short article we'll look into a rough draft of some code that creates database tables at the same time a class is created and alters the class definition in such a way that accessing instance variables will result in calls to some database engine.

The first thing to understand is that Python metaclasses are not magic. Everytime you define a new class you already refer to the built-in metaclass type. Metaclasses are subclasses of type and regular classes are instances of a metaclass. This may sound weird but remember that everything in Python is an object, even a class definition. And because every object is an instance of some class, this is just a logical extension of a general concept.

If you want your class definition to be instanced by a different metaclass than type you will have to use the metaclass parameter in your class definition:

class MyClass(metaclass=MyMetaClass):
 ...

Your metaclass must be a subclass from Python's built-in metaclass type and should at least provide a __new__() method:

view plainprint?
  1. class MyMetaClass(type):  
  2.  def __new__(metaclass, classname, bases, classdict):  
  3.   ... do stuff to classDict ...  
  4.   return type.__new__(metaclass, classname, bases, classdict)  

This __new__() method should return a new class, for example by passing its attribute to the __new__() method of type like we do in the example above. The real power is hidden in the arguments passed to the __new__() method: besides the name of the class we are building and the base classes it will be a subclass of, we also have access to the class dictionary.

The class dictionary holds all class attributes, including class variables and methods (both class methods and instance methods). The beauty is that we can check and/or alter the contents of this class dictionary before we actually create a class. This could be used for example to check whether the class has some mandatory method definitions or overrides specific methods in its bases, something that is used to implement abstract base classes in Python.

Another application of metaclasses is to bridge domains. In the example below there are two domains: the classes and instances of those classes in Python's runtime environment and the tables and records in a relational database stored on disk. If we would like to map those classes to database tables metaclasses provide us with some excellent tooling because they allow us to act upon the class definition before the class is actually made to exist.

This means that we when we define a class our metaclass can check whether there is already a suitable table defined in the database and that class variables are mapped to columns in this table. It is also possible to change class variables into properties in such a way that accessing these properties from instances will result in proper sql statements issues to a database engine.

This may sound a bit abstract, so just take a look at the code below. It will not really interact with a database but just print the sql it would have used. The comments should explain what is happening in a fairly detailed manner:

view plainprint?
  1. from functools import partial  
  2. from random import randint  
  3.   
  4. class Attribute:  
  5.  """ 
  6.  Attribute makes it possible to distinguish class 
  7.  variables that should be backed by a columns definition 
  8.  for a table. 
  9.  """  
  10.  def __init__(self,constraints=''):  
  11.   self.constraints=constraints  
  12.     
  13. class DBbackend(type):  
  14.  """ 
  15.  A metaclass that will create a database table that 
  16.  contains columns for each class variable that is an 
  17.  instance of Attribute. It also replaces these class 
  18.  variables with poperties that will retrieve or update 
  19.  the database value when the attribute is accessed. 
  20.  """  
  21.  def __new__(meta, classname, bases, classDict):  
  22.   attributes = {}  
  23.   # create a suitable table definition. Although we  
  24.   # do not do a complete implementation here, we have  
  25.   # sqlite in mind, so no explicit types.  
  26.   for attr in classDict:  
  27.    if issubclass(Attribute,classDict[attr].__class__):  
  28.     attributes[attr]=classDict[attr]  
  29.   sql = 'create table if not exists %s ( %s )' % ( classname,   
  30.    ",".join(['id integer primary key autoincrement']  
  31.      +[ name+" "+a.constraints for name,a in attributes.items()]))  
  32.      
  33.   print(sql)  
  34.     
  35.   # we alter the __init__() method to create a database record  
  36.   def create(self,**kw):  
  37.    self.id=randint(1,1000000) # just for illustration, normally handled by autoincrement in the db  
  38.    sql='insert into %s (%s) values (%s) [%s]' % (self.__class__.__name__,  
  39.     ",".join(kw.keys()),  
  40.     ",".join(['?']*len(kw.keys())),  
  41.     ",".join(kw.values()))  
  42.    print(sql)  
  43.      
  44.   classDict['__init__']=create  
  45.     
  46.   # functions that retrieve/update a column in a database table  
  47.   def get(self,name):  
  48.    print('select %s from %s where id = ? [%s]' % (name,  
  49.       classname,str(self.id)))  
  50.     
  51.   def set(self,value,name):  
  52.    print('update %s set %s=? [%s] where id = ? [%s]' % (classname,  
  53.       name,str(value),str(self.id)))  
  54.     
  55.   # change each class var that holds an Attribute object to a   
  56.   # property that get/sets the appropriate column  
  57.   for attr in attributes:  
  58.    fget = partial(get,name=attr)  
  59.    fset = partial(set,name=attr)  
  60.    classDict[attr]=property(fget,fset)  
  61.     
  62.   return type.__new__(meta, classname, bases, classDict)  
  63.   
  64. if __name__ == "__main__":  
  65.   
  66.  # example, create a class with three attributes/database columns  
  67.  class Car(metaclass=DBbackend):  
  68.   make = Attribute()  
  69.   model= Attribute()  
  70.   license=Attribute('unique')  
  71.   
  72.  # create an instance  
  73.  mycar = Car(make='Volvo', model='C30', license='1-abc-23')  
  74.   
  75.  # retrieve various attributes  
  76.  model = mycar.model  
  77.  make = mycar.make  
  78.  lic = mycar.license  
  79.   
  80.  # set an attribute  
  81.  mycar.model='S40'  
Labels: , ,
Subscribe to: Posts (Atom)