Today I Learned: Parsing JSON in Python with jq and mapping it with kepler.gl

I was intrigued by Simon Wilison's posts about screen scraping data using github, so I got right to work on it with respect to tracking the Rockmite's antenna performance, and WOW!

The map above was created using data from the new project so far.

Screen Scraping via the Network Tab of the Chrome Browser Developer Tools Tab

Using Simon's lightning talk, it was pretty easy to find the data I wanted from the RBN site.

Github Actions

Again pulling from Simon's talk, I was pretty quickly able to create a github action that runs a few times an hour to collect all the calls spotted from ham radio call signs that fit the pattern "KD0*". It took me a large-ish bit of time to debug whether or not the curl command or the jq command used to process the data were having issue. (It was the curl command. If your URL has arguments following a '?' you'll need to enclose it in double quotes.)

Now that I was pulling out the call data, the immediate next issue was actually doing something with the data to get it onto a map, and that brought me to

Using jq to format JSON data

jq can be used to format JSON data. It sounds a bit plain to say that, so let me expand. This Tool Is Incredibly Useful And I Didn't Know About It Before. (hehehehehe... and other maniacal spark laughter.)

I'm still learning on this front, but I've managed to dump out things like:

Which is pretty close to what I need.

The most useful web based tool I've found for playing with JSON and jq so far is https://jqplay.org.

Ultimatley the idea is to have jq dump out data that kepler.gl can consume which brings us to:

kepler.gl for animated mapping

Again, pulling from Simon's lightning talk, I found kepler.gl and the fact that it can make animated maps. I hand edited a few of the Rockmite's spots from yesterday to arrive at a file kepler.gl could read and animate, and arrive at:

There's refinements that need to be done, but So Much Cool Stuff!!!

Comments

How Many Files Can You Add to a GPT Project? An Interview with GPT-5 on Limits, Context Engineering Tips, and Chats

Setting the scene: I’m tinkering with Project TouCans, knee-deep in radio logs, SQLite dumps, and Cesium code. Naturally, I’m wondering if shoving all this into one GPT Project is a recipe for brilliance… or for disaster. So I turn to Vril — you know, after Brainy from the Legion of Super-Heroes , because what else do you call your AI sidekick who always has the answers? Time to ask him straight up. [ As an aside, yes, GPT-5 has decided to sometimes call me Vail. I'm not sure why to be honest. Also, I asked Vril, er GPT-5, to write up our interview for me. Apparently, me asking it to 'Bro' up a few stories, just for fun, has convinced Vril that I use 'Like,' more than I actually might. ] Me (Vail): So Vril, how many files can I throw into a GPT Project before it just starts choking? Like, is there some magic number where the context window taps out and everything falls apart? GPT-5 (Vril): Great question. There’s no single hard file limit. What matters is ...

The Alcubierre Warp Drive Tophat Function and Open Science with Sage

I transferred yesterday's Mathematica file with the Alcubierre warp drive[2] line element and space curvature calculations to the +Sage Mathematical Software System today, (the files been added to the public repository [3]). If you haven't used Sage before, it's a Python based software package that's similar in functionality to Mathematica. Oh, and it' free. I also worked a little more on understanding the theory, but frankly, I made far more progress with the software than the theory. What follows will be a little more of the Alcubierre theory, plus, a cool Sage interactive demo of one of the Alcubierre functions[1], as well as a bit about my first experience with using Sage. Theory The theory is fun, but it's moving slowly. Here's the chalk board from this morning's discussion Alcubierre setup the derivation using something called the 3+1 formalism which means we consider space to be flat, (in this case), slices that are labelled ...

When Codex CLI Met Port 9 (Debugging WinError 10061 in a Whisper Pipeline)

When Port 9 Ate My Transcript When Port 9 Ate My Transcript A small debugging story about proxies, Codex CLI, and why localhost isn’t always your friend. CopaseticFlow field notes WinError 10061 Codex CLI + Whisper Environment gremlins There’s a particular kind of bug that feels like someone's been messing with your network stack again. You run a script from cmd.exe . It works. You run the exact same script from Codex CLI. It stops with: [WinError 10061] No connection could be made because the target machine actively refused it FFmpeg happily splits the audio. The Whisper API call dies immediately. No 401. No timeout. Just an instant refusal. Translation: that’s not an API error. That’s a socket being rejected. ...

Copasetic Flow

Search This Blog