Things I Learned: I Wrote my First Datasette Plugin

 I needed to learn Jinja and/or brush up on my Django skills for work. I decided to go the Jinja route because that template engine is used in other tools at work, plus I could write a Datasette plugin while I learned my way around.

It's always bugged me a bit that map_qsos.py programatically writes out the kml files that I use to display our ham radio outings on Google Earth. Print statements are the wrong tool for the job, which—at the end of the day—is to create the same kml lines and points substituting in different station names and locations in a loop. The process fairly screams template engine. Another issue is that even though the data is available in a database after its initial entry, with the previous usage model, I had to collect the versions of qso_update.csv I wanted to map and then combine their calls into a single file. This file was then operated on by map_qsos.py. The whole process was cumbersome, and did not lead to me wanting to explore maps of the log unless there was a really good reason. Now, I can create maps of entire five week timespans in just a few seconds

 As of today, the Jinja template engine is called from a Datasette plugin (only local to the project so far) that hooks into the QSO database for Project TouCans. I had a few dust-ups along the way. Here's what I learned:

I had a collision between the render call for my plugin and the leaflette-freedraw plugin

I'm convinced this was due to my lack of understanding and not the other plugin that's used in the QSO log database to geographically search for QSOs. The proscribed way to render templates in Datasette is to use an await style of call. I've never used these before. The call caused a collision for me, resulting in the leaflet-freedraw issue. I worked around this by instantiating my own Jinja template engine and calling it independently of the instance built into Datasette. All in all, I got to see a bit more of the Jinja ecosystem, which is what I was trying to learn, so it was a win-win.

I learned how to pass Python mehods into Jinja templates

Despite the—I'm sure—justifed complaining about the extra steps required to call methods from Jinja templates, it seemed pretty easy. The mapping template needed a way to determine what color of line to draw based on the RST, (from QSOs), or SNR, (from RBN spots), number stored in the database. Frankly, I wish I had time to learn a more elegant way like defining a new method for SQLite, but I need to learn Jinja, not SQLite, so calling methods from the template was the way I went. You can take a look at the plugin code I wrote today to see how to do this.

I learned the Datasette hook plugin mechanism

It was easier than I thought it would be. Basically, I had to implement two decorated methods: 

prepare_connection

and

register_output_renderer

The second of those points at my render_kml method which in turn receives the rows returned from the latest database query. It passes those rows along to the map template I mentioned, and that's that. When the user clicks on the kml link shown below, rm_toucans.kml is downloaded to their machines with all the perinent details.

To do what it does, the plugin needs the location of both stations, the time the QSO was made, and the received RST or SNR value for the QSO. The register_output_renderer method also accepts a pointer to a function that checks that the query has the appropriate columns to get the job done. The beauty of the Datasette plugin framework is that if the query doesn't have the correct columns, the kml link is simply not displayed with the query results.

The picture above that shows the kml link also illustrates my example for proving out that the new plugin works. The 12, 10, and 8 year-old gang and I camped in a variety of locations in California, Nevada, Utah, and Colorado over a period of five weeks in May and June. It was glorious! I can't recommend it enough! 

As we traveled, I made QSOs; mostly POTA with a few SOTA thrown in for good measure. The query above retuns all the QSOs I made. By clicking on the kml link, I get a kml file that when loaded into Google Earth, shows the whole ham radio trip on one map in a few seconds.

Things I haven't learned yet

There were a few features that I'd like to have learned more about, but that I had to gloss over because my focus was a quick ramp on Jinja.

Plugin Custom Templates

I wasn't able to get my plugin to automatically pull in custom templates. In other words, my code was not automatically aware of the templates directory. There's a workaround which is passing in the path to the templates directory on the command line that should have worked, but for the moment, it hasn't for me

python3 -m datasette rm_toucans.db --metadata qso_loc.yml --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so --plugins-dir=plugins/ --template-dir plugins/templates

Instead, I had to place the complete path to the template I used in the source code itself which is clunky and since I use this code on several machines and eventually hope to publish it online, kind of a big deal.

Update: I've since modified the code to use a hard-coded path that is relative to the repository directory.

I didn't have time to get the cookie cutter for plugins to work

This looks snazzy. I think it will also solve the above issue. It's not working for me yet though. I was running fast, but if I remember correctly, I just had python package dependency issues I didn't have time to iron out. Go figure.