Javascript syntax highlighting on Blogger without external hosting

Using Alex Gorbatchev's Javascript syntax highlighter is very simple to use and useful for about any language, including Python but most recipes for integrating it in Blogger depend on access to an external location to host the Javascript files. In this article I show you how to integrate syntax highlighting into your Blogger page without the need to host anything externally.

Note: I only managed to get it to work for the old version (1.5.x) but as you can see on my blog that version is still eminently usable. As you will see there is no particular reason why the approach sketched below won't work for newer versions and I will look at it again but for now we'll make do with what works.

Step by step installation of the syntax highlighter in Blogger

1. Download the package from Google Code
2. Unpack the rar file
3. Add a HTML/Javascript gadget to your Blogger layout (near the end of your page, e.g. in the footer)
4. Copy the contents of the Javascript files you need to this gadget
5. Add a extra line of Javascript to actually convert code fragments
6. Copy the contents of the CSS stylesheet into Bloggers advanced CSS configuration

The rar file can be unpacked with most unzip tools, 7zip worked for me (on Windows). The minified Javascript files we need are in the Scripts folder. You need to copy the contents of at least the file shCore.js plus the contents of any file suitable for the languages you want to highlight in your blog. For this blog I copied shBrushPython.js to start.

The HTML/Javascript is available from Bloggers gadget menu when you click on a a 'add gadget' link. It allows you to insert arbitrary HTML and Javascript into your Blogger pages. If you add or edit the HTML/Javascript gadget it opens a simple text editor where you can insert the code. Make sure that you embed any code that you copy inside a pair of script tags. Add the contents of shCore.js first, followed by the contents of the other files. Finish with a line like dp.SyntaxHighlighter.HighlightAll('code');. The resulting contents need to look something like this:

<script>
... contents of shCore.js ...
... contents of shBrushPython.js ...
dp.SyntaxHighlighter.HighlightAll('code');
</script>

Note that the HTML/Javascript gadget needs to be near the end of the page because any code you want to highlight should already be loaded once the HighlightAll() method is called. You can of course arrange to have this method called in an onLoad event, in which case you can probably choose to place the gadget anywhere.

The final step is to open Blogger's template editing page. Select Advanced from the left most column and then select Add CSS from the column next to it. This will open up a text editor where you can insert the contents from the file SyntaxHighlighter.css (it can be found in the Styles directory). Click Apply to Blog and you're done.

Python and Javascript: using the Flot plugin, part 3

In a previous article I sketched the road map for implementing a small web application to present data from a small weather station with the help of PyWWS and the Flot plug-in. In this article we show how to provide data from the pywww/DataStore module as JSON encoded information that we can use in AJAX calls.

Producing JSON encoded weather data with CherryPy

Our web application is a single web page with some added Javascript that relies on a number of services that provide JSON encoded data to AJAX calls. These services are implemented as Python classes within a CherryPy application. The first class we define is called Basic and its initializer takes a single tz argument. This will be the timezone used to display data.

class Basic:

    def __init(self,tz=pytz.timezone('Europe/Amsterdam')):
        self.tz=tz
        
    @cherrypy.expose
    def default(self,name,_=None,start=None,end=None):
        start,end = self.verifystartend(start,end)
        if name in { 'temp_out','hum_out','wind_ave',
               'wind_gust','wind_dir','rain'}:
            return self.getdata(name,start,end)

The crucial method is default(). It is exposed to the CherryPy engine with the cherrypy.expose decorator. A exposed method with the name default will receive any request that cannot be mapped to a more specific name. The name attribute will hold the final part of the URL. The _ argument will hold a random string that we simply ignore: it is added to any AJAX call by jQuery to prevent the browser from caching results. The start and stop arguments are optional and may contain a date/time argument in YYYYMMDDHH format. If the end argument is absent, it defaults to now, if the start argument is absent it defaults to 24 hours before end. These defaults are calculated by the verifystartend() method (not shown).

The next check is to see whether name is one of the known types of data in the DataStore. If all is well we simply pass the name, start and end arguments to the getdata() method that we encounter in the next section. The getdata() method is not part of the Basic class but will be provided by a mixin class called Service. The idea is that Basic and its subclasses provide web application logic to be embedded in CherryPy, while Service provides an interface to produce JSON encoded data based on the data provided by PyWWS.

We define two subclasses of Basic. The first is called Hourly and should return weather data that are the hourly averages. Its initializer takes a weatherdata argument which should point to the directory where PyWWS stores its data and an optional tz argument to hold the timezone:

class Hourly(Service,Basic):

    def __init__(self, weatherdata,
           tz=pytz.timezone('Europe/Amsterdam')):
        super().__init__(tz)
        self.datastore = DataStore.hourly_store
        self.weatherdata=weatherdata

The most important bit is that __init__() creates a datastore instance variable and initializes it to the hourly_store class from the DataStore module. The datastore and weatherdata variables will be used by the getdata() method from the Service mixin.

The Raw class is very similar to the Hourly class only its datastore variable is initialized to the data_store class from the DataStore module which produces non-averaged weather data.

class Raw(Service,Basic):

    def __init__(self, weatherdata,
           tz=pytz.timezone('Europe/Amsterdam')):
        super().__init__(tz)
        self.datastore = DataStore.data_store
        self.weatherdata=weatherdata

The Service mixin class is where all the hard work is happening. The bulk of the work is done by its getdata() method. It first creates a suitable instance of a pywws DataStore and then converts the data it retrieves from this datastore with the dumps function from the json module.

            
    def getdata(self,what,start,end):
        ds=self.datastore(self.weatherdata)
        return dumps(            
                [( self.millisecondsfromnaiveutc(
          data['idx']), data[what]) 
                 for data in ds[
                     start.astimezone(
        Service.utc).replace(
                       tzinfo=None):
                     end.astimezone(
        Service.utc).replace(
                      tzinfo=None)]
    ]
               )

The single argument to the dumps() function is a list of tuples, each consisting of a timestamp in milliseconds and a floating point value as this is the format that the Flot library expects. This data is retrieved from the datastore with the slice notation: ds[a:b] will retrieve all the data between a and b (if a and b are datetime instances).

All this work was needed to be able to request URLs like http://localhost:8080/temp_out and receive a response like [[123456,14.1],[123654,14.2],[123789,14.4]] for example.

In the next installment of this series we will look into the HTML and Javascript code needed to make this web application really work.

Other parts of this series

• Intro
• Converting PyWWS/DataStore to Python 3

Python and Javascript: using the Flot plugin, part 2

In a previous article I sketched the road map for implementing a small web application to present data from a small weather station with the help of PyWWS and the Flot plugin. In this article we tackle the conversion of the pywww/DataStore module to Python 3.

Converting the pywws/DataStore module to Python 3

Because pywws is written for Python 2.x we need to convert it to Python 3 because I don't want to continue developing for Python 2.x now that many frameworks and libraries are converted to version 3. In principle the 2to3 tool should be able to do the bulk of the work but we cannot be sure before we try and test.

For our web application we will only need to use the pywws/DataStore module to get access to stored weather data and fortunately it has no dependencies on other pywws modules. Converting this module with the 2to3 tool and comparing the differences shows only 3 changes:

• the ConfigParser module is now called configparser
• map() returns a map object instead of a list
• open() takes an 'r' mode argument instead of 'rb' in those cases where the result of the open() function is expected to return unicode strings instead of bytes.

The second change is the most fundamental. A map object is a generator and not a plain list. This means is has to be explicitly converted to a list if an operation expects a plain list. For an expansion from a list to a number of arguments this is correctly done by the 2to3 utility. For example, the following line

  return datetime(*map(int, (date_string[0:4],
                                    date_string[17:19])))

is translated to

  return datetime(*list(map(int, (date_string[0:4],
                                    date_string[17:19])))

All in all this is a fairly painless conversion process (but remember that I converted just the Datastore module, although with 455 lines this is pretty big). The next article will show how we implement server side component of the weather application based on PyWWS and CherryPy.

Python and Javascript: using the Flot plugin

Good graphs are not only about data, they should look good as well to attract attention. The Flot plugin for jQuery produces excellent graphs, is simple to use and shows nicely how to marry Python and Javascript.

In my opinion when designing a web application, the client side (the stuff that happens in the browser) is often not receiving the attention it deserves. Sure enough we design for low latency using AJAX and mimic screen interactions of regular applications as close as possible but often is still feels like we're looking a old web page.

One of the areas that can benefit tremendously from a well chosen library is graphs. In the coming months I will try to describe a simple web application that displays data from a weather station. The server side is all python of course and will make use of the PyWWS library to acquire the data. On the client side will employ Flot, a jQuery pluging that can produce attractive graphs in simple manner. A sample of the first draft of the app is show below:

In later articles I will show how to implement both server and client side.