Today I Learned: Parsing JSON in Python with jq and mapping it with kepler.gl

I was intrigued by Simon Wilison's posts about screen scraping data using github, so I got right to work on it with respect to tracking the Rockmite's antenna performance, and WOW!

The map above was created using data from the new project so far.

Screen Scraping via the Network Tab of the Chrome Browser Developer Tools Tab

Using Simon's lightning talk, it was pretty easy to find the data I wanted from the RBN site.

Github Actions

Again pulling from Simon's talk, I was pretty quickly able to create a github action that runs a few times an hour to collect all the calls spotted from ham radio call signs that fit the pattern "KD0*". It took me a large-ish bit of time to debug whether or not the curl command or the jq command used to process the data were having issue. (It was the curl command. If your URL has arguments following a '?' you'll need to enclose it in double quotes.)

Now that I was pulling out the call data, the immediate next issue was actually doing something with the data to get it onto a map, and that brought me to

Using jq to format JSON data

jq can be used to format JSON data. It sounds a bit plain to say that, so let me expand. This Tool Is Incredibly Useful And I Didn't Know About It Before. (hehehehehe... and other maniacal spark laughter.)

I'm still learning on this front, but I've managed to dump out things like:

Which is pretty close to what I need.

The most useful web based tool I've found for playing with JSON and jq so far is https://jqplay.org.

Ultimatley the idea is to have jq dump out data that kepler.gl can consume which brings us to:

kepler.gl for animated mapping

Again, pulling from Simon's lightning talk, I found kepler.gl and the fact that it can make animated maps. I hand edited a few of the Rockmite's spots from yesterday to arrive at a file kepler.gl could read and animate, and arrive at:

There's refinements that need to be done, but So Much Cool Stuff!!!

Comments

The Valentine's Day Magnetic Monopole

There's an assymetry to the form of the two Maxwell's equations shown in picture 1. While the divergence of the electric field is proportional to the electric charge density at a given point, the divergence of the magnetic field is equal to zero. This is typically explained in the following way. While we know that electrons, the fundamental electric charge carriers exist, evidence seems to indicate that magnetic monopoles, the particles that would carry magnetic 'charge', either don't exist, or, the energies required to create them are so high that they are exceedingly rare. That doesn't stop us from looking for them though! Keeping with the theme of Fairbank[1] and his academic progeny over the semester break, today's post is about the discovery of a magnetic monopole candidate event by one of the Fairbank's graduate students, Blas Cabrera[2]. Cabrera was utilizing a loop type of magnetic monopole detector. Its operation is in concept very sim

Cool Math Tricks: Deriving the Divergence, (Del or Nabla) into New (Cylindrical) Coordinate Systems

Now available as a Kindle ebook for 99 cents ! Get a spiffy ebook, and fund more physics The following is a pretty lengthy procedure, but converting the divergence, (nabla, del) operator between coordinate systems comes up pretty often. While there are tables for converting between common coordinate systems , there seem to be fewer explanations of the procedure for deriving the conversion, so here goes! What do we actually want? To convert the Cartesian nabla to the nabla for another coordinate system, say… cylindrical coordinates. What we’ll need: 1. The Cartesian Nabla: 2. A set of equations relating the Cartesian coordinates to cylindrical coordinates: 3. A set of equations relating the Cartesian basis vectors to the basis vectors of the new coordinate system: How to do it: Use the chain rule for differentiation to convert the derivatives with respect to the Cartesian variables to derivatives with respect to the cylindrical variables. The chain

More Cowbell! Record Production using Google Forms and Charts

First, the what : This article shows how to embed a new Google Form into any web page. To demonstrate ths, a chart and form that allow blog readers to control the recording levels of each instrument in Blue Oyster Cult's "(Don't Fear) The Reaper" is used. HTML code from the Google version of the form included on this page is shown and the parts that need to be modified are highlighted. Next, the why : Google recently released an e-mail form feature that allows users of Google Documents to create an e-mail a form that automatically places each user's input into an associated spreadsheet. As it turns out, with a little bit of work, the forms that are created by Google Docs can be embedded into any web page. Now, The Goods: Click on the instrument you want turned up, click the submit button and then refresh the page. Through the magic of Google Forms as soon as you click on submit and refresh this web page, the data chart will update immediately. Turn up the:

Copasetic Flow

Search This Blog