Copasetic Flow

 Project TouCans just got a major database upgrade! The QSO records that KO6BTY and I log from field activations can now be appended directly from our GitHub CSV files instead of regenerating the entire SQLite database. This new method—based on an approach I first saw in Simon Willison’s video—keeps local data editable while automatically merging in the latest contacts from the cloud.

When making our QSO skip maps one of the issues we encounter is that the lines from our station to the F2 layer don't always start right at the ground. This seems to be an artifact of CZML and the fact that the Earth is not a sphere, but rather an ellipsoid, and even then, there are all those hills and mountains. The initial QSOs can wind mapped as up coming from underground: or, they can wind up starting from above the surface of the Earth In any event, it's been necessary with each map to adjust the elevation height of our station. I've yet to find a parameter to set in a czml that makes the starting point always be on the ground. "clampToGround" works for polylines that are always on the ground, but not polylines whose next point has an altitude greater than zero as the ground station to F2 layer polylines do. I added a feature into the JavaScript mapping application this week that allows me to adjust the height of all of the QSO's starting points at...

 New ham cleans up log book on QRZ with the help of o3-mini. A few days back, after the completion of SKCC month, a ham sent out a message asking others to please post their QSOs ot LoTW and/or QRZ.com for award verification. For me, a light bulb burst on. It does matter that I log my QSOs publicly. It had frankly, never occurred to me before, but it made sense. o3-mini and I got to work. I already have a log book built on Datasette for my QSOs. I just needed to get the QSOs from there and ship them off to QRZ.com . I needed it to be automatic though, since there were more than 1,000 QSOs to transfer. I asked o3-mini to write a Python script to do this for me. I gave it the database schema for my QSO database. I also fed it the l ogging API instructions for QRZ.com.  I had something that worked well enough after about 40 minutes. After three hours, that included a 40 minute coffe break, I had a script that automatically calculated my tx grid square, (it changes a lot becuase...

I've been referred to a lot of indie web blogs of late, and it's paying off quite nicely for me Here's the latest example. Simon Willison--co-creator of Django and creator of Datasette--has a blog that led me to Maggie Appleton's site . Once there, I found a very nice, and very pretty primer on databases . Within the primer was a perspective I'd never seen before. There was an emphais, (certainly not the only emphasis, but an emphasis nonetheless), placed on columns, like so: from " A Shelfish Starter Guide to Databases " I, frankly, had never considered coluimns in any way except, as 'fields' that contributed to rows, and that could have conditions placed on them. A column as a whole entity unto itself? I'd never considered such a thing. A few days later though, while studying the netCDF format used by COSMIC2 missions among projects, I suddenly needed that column perspective, and I had it! netCDF files from COSMIC2 are very much arranged a...

We're on the East Coast! KO6BTY, myself, and the 9 year-old, (known on the internets as Tawnse), flew out to Boston from San Francisco yesterday. We walked out of the airport to the ferry terminal! That's so cool! You can walk to a ferry from the airport. But do you know what we did then? We didn't take a ferry, we took a water taxi to a different ferry terminal! So many cool things already! Here's the view from the water taxi. We landed at Rowes Wharf which seems to be one of the fanciest wharf's in Boston, so.. yeah. From there we caught the ferry southeast to Hingham  and from there made it to our camp site at Wompatuck State Park. We're now making our war around via public transit taking the T back in to Boston South Station to catch a train from South Station into Philadelphia. Meanwhile! KO6BTY and I have had Cesium maps built into our QSO log for a few months now—ever since the day Simon Wilison nonchalantly pointed out that Datasette queries are URLs i...

 Here's the query I'm using for Burr Trail ============================================ select   tx_lng,   tx_lat,   rx_lng,   rx_lat,   rm_rnb_history_pres.timestamp,   rm_rnb_history_pres.dB,   rm_rnb_history_pres.Spotter,   haversine(tx_lat, tx_lng, rx_lat, rx_lng) as total_path,   gis_partial_path_lat(tx_lat, tx_lng, rx_lat, rx_lng, 200) as el_lat,   gis_partial_path_lng(tx_lat, tx_lng, rx_lat, rx_lng, 200) as el_lng,   id,   strftime('%Y%m%d', timestamp) as date,   strftime('%H%M', timestamp) as time,   'US-4399' as park,   'KD0FNR' as call,   1770.829467773438 as elev_tx from   rm_rnb_history_pres where   dB > 100   and timestamp > '2024-05-27'   and timestamp < '2024-05-28' order by   rm_rnb_history_pres.timestamp desc ======================================================= At some point I'll add an ionosonde field using this map for each QSO. For the Boulder, CO...

 The F2 skip traces in Cesium have looked a bit odd: Whyyyyy????? It's because they're starting from sea level, not the altitude of the station.... I think... What follows are somewhat random development notes. Relative to Ground: True, Alt: 0 Relative to Ground: True, Alt: 3600 meters Relative to Ground: True, Alt: 1772.717 Almost!!! Relative to Ground: true, elevation: 1770.829467773438 I got the more accurate elevation from the elevation plugin using: {{ tx_lat }},{{ tx_lng }}|{{ rx_lat }},{{ rx_lng }} Not quite there yet.... To get this close, I had to run with  But the CZML schema says: So, it seems like the correct altitude in this case should be 0. (Start the line on the ground.) But, if I put in zero for altitude, I get the result shown above Found a reference to quaternions. Ahhhh, quaternions. References Ionosonde Data This was made possible with data collected by  Reinisch, B. W., and I. A. Galkin, Global ionospheric radio observatory (GIRO), Earth, Pla...

 KO6BTY and I attended office hours with Simon Willison yesterday. At the very end of our call, Simon showed us how we could open a czml directly from our localhost Datasette server in Cesium. After about an hour this morning, the video below shows the results. Here's what you're seeing. There's an SQL query in our Datasette instance that grabs only the QSOs from our Cibola National Forest POTA back in March . That query resulted in 9 QSOs shown in the table. Our additional CZML (and now Cesium viewer) plugin creates a CZML map that is loaded into a Cesium Ion viewer at the bottom of the page. I'm not releasing the code that loads up the viewer quite yet because I have to get the access token squirreled away and all that good stuff. Meanwhile the CZML plugin has been released for a few days now. By the way, we need a way to pull adif files from POTA activations into Datasette. Keep an eye out :)

 I mentioned that I'd made a pull request for datasette and Windows path handling, and then forgot to talk about what my fix involved. Basically, Datasette as it stands feels that colons have no business in file paths. Windows on the other hand? C:\ Consequently my pull request revolved around changing datasette/utils/__init__.py by adding the following code : The code checks for a Windows style path specification. If it finds it, it gently moves the rest of the code around that first offending colon.

 It's been a busy and fun week again! Most of my time was spenting finalizing what the elevation profile Datasette enrichment would look like. I finally found the last piece of the puzzle laying around in a pile of Jinja template use practice. One of the goals this week will be to release the elevation package. I made my first pull request for an open source project, datasette, in this case. (I make them at workr, but this is my first foray into the wider world.)  ChatGPT provided the final kick to get the elevation maps plotted really eaisly. Subsequently, I learned about the Pyplot package and how to add labels to charts. I got a start back towards writing about dadding. I spruced up some writing I had lying around about cleaning the kitchen with the gang. I found another older piece that I love while checking out old blog entries here: Meerktas and Ravens . Oh, and then this one about what unschooling with toddlers lools like. Goals Release the elevation package Relea...

 I made my first ever pull request on GitHub this morning! The gang and I still mostly develop code on Windows boxes. It has more to do with the machines we have available to us on any given day than anything else. (The just-sit-down-and-type machine in our house is also for playing video games, so... Windows.) This has led to issues when using our Datasette enabled QSO logger and mapping applications .  Datasette on Windows has an issue with finding the SpatiaLite DLL file. The gang and I worked through the issues and documented our really clunky, blunt force, fix . The fix, however, amounted to changing the source code of Datasette itself on a per installation basis. That meant that if I moved to a new Windows machine I had to find the fix and re-implement it. Even worse, that meant that if I wanted to try something that worked only in a newer version of Datasette, I also had to reimplement the fix. That... Was a Lot. So, I was totally psyched when I got a little bit of ti...

 This is a note for myself for later when I try to construct a better set of instructions for adding Spatialite to Windows for use with Datasette . You can instruct Windows to look for DLLs in a new directory simply by adding that directory to your PATH envrionment variable via  set PATH=C:\path to your dll;%PATH% via .

 This is a small thing, but it's going to give me a lot of utility with the enrichments I'm writing. More than once, I've run into the need for an input argument to the enrichments I'm creating. Take the elevation enrichment for example. I'd like the user to be able to specify how long their elevation profile path should be in meters. My slop and drop method for adding an argument to rows of table data would be to simply specify a constant in my SQL query ala  select tx_lat, tx_lng, rx_lat, rx_lng, 200 as optional_meters from qso_table That would result in an extra column containing the number 200 on every row. That also won't work with enrichments for the moment because the operate on filtered tables, not generic SQL queries. That was irksome. The answer though, is simple, elegant even. For each enrichment , the user specifies their input using a template. For the elevation enrichment, the template looks like this: but I could have just as easily typed and now...

 I'm mostly writing this so that the next time I run into this, I'll have a handy set of notes. Many months ago, I found out that Datasette on Windows doesn't like this :             for extension in self.sqlite_extensions:                 # "extension" is either a string path to the extension                  or a 2-item tuple that specifies which entrypoint to load.                 if isinstance(extension, tuple):                     path, entrypoint = extension                     conn.execute("SELECT load_extension(?, ?)", [path, entrypoint])                 else:                     conn.execute("SELECT load_extension(?)", [extension]) in...

 I started a project, ahem, yesterday to 'quickly' see if ChatGPT could read the entire United States general class amateur radio exam question pool into a Datasette instance using the datasette-extract plugin. As of this morning, I haven't been able to coax ChatGPT using the gpt-4-turbo model. I capture my rather raw notes below. The short version of this is that I was never able to get the AI to capture more than 19 questions at a time. I'm hopeful that the pool could be moved into a database table using iterative processes, but for now, I've run out of time for this quick project :)  Occasionally ChatGPT seemed to hallucinate out part of its process into the table Notes Follow I'm going to track how easy it is to get the general exam question pool into a database using the Datasette Plugin. I started this endeavor at 20:37 UTC. Get my already existent OpenAI API key ready to go 20:43: Done. As usual with OpenAI, the hardest part was finding login screens ...

 This project has been going on for a while, but remember, the point of the elevation profile project was to add the elevation profile between our ham radio—Project TouCans—and the receiving station at the other end of a QSO to the map of the QSO itself. This week, I finally got to start work on the maps themselves, and I've found a few things. First, an image added to a Placemark description without a  width="100%" tag will overflow the pane that Google Earth Web creates for the Placemark A user can scroll the chart horizontally, but theOh  information isn't all visible at once. Adding the width tag back in fixes the issue on Google Earth Web, but leaves Google Earth Pro with a very small chart I thought a good compromise between the two might be to house the image in its own div. For Google Earth Web we get but on Google Earth Pro we still get the tiny chart: It seems the only way to get this done will be to learn the new Google Earth kml 'Carousel' extensio...