Following on from my other Node Red exploits I’ve put together a flow that creates an interactive map of contacts that is generated from the WSJT-X ADI log file.
The flow is fairly straight forward and self explanatory so I won’t go into detail here but, will make a copy of the flow available for download at the end of this article.
Node Red Flow for processing the WSJT-X ADI Log file
The flow is incredibly quick at generating the map with all the pins on it, one for each station worked. The pins are colour coded, blue for FT8 and green for FT4. If you want to add other modes then just create a new colour entry in the Dynamic Icon Colour function.
The resultant map is fully interactive with each pin being clickable showing the QSO information in a tiny popup.
Node Red map generated from the WSJT-X ADI Log file
You can download the flow and try it yourself using the link below.
Following on from my previous article on Enhancing Digital modes with Node Red I’ve now got to a point where I have realtime decode information from the WSJT-X digital application being plotted on a Node Redworld map not just for CQ calls but, for stations in conversation too.
The flow has become somewhat more complex than it was originally as more and more functionality has been added. I have deliberately split out the flow process into more nodes than are really necessary so that the flow is easier to understand. Anyone from a programming background like myself will soon realise that a lot of the nodes could actually be combined into one big node however, the overall flow process wouldn’t be so easy to understand for the Node Red newcomer and would possibly put people off from trying it out.
Current WSJT-X Node Red flow
Above is a screenshot of the flow as it currently stands. It’s pretty easy to understand what is happening in the flow due to the fact that the processes are broken out into small, easy to digest blocks.
From the top down we connect to WSJT-X via UDP port 2237 and listen for the data stream. As the data is received it’s passed directly into the WSJT-X-Decode node that converts the information into a Node Red compatible format. The data is then filtered with only the information required being passed onto the next node. There are two outputs from the filter node as we require two different streams of information, namely “CQ” and “TX1” data. All the rest of the data from WSJT-X is ignored as it’s not required at this time.
The “Get freq & SNR + Others” node builds a decode message payload with all the correct data, in the right format ready to be passed on along the flow. This node also sets a number of parameters required by the map node to be able to control the display of the data.
The next node along is “Set msg.payload”, this brings together all the necessary data into a single message payload that is then worked on by all the nodes further along the flow.
The next 3 nodes perform the simple task of moving some of the data into the objects defined by the world map node, if the data isn’t moved into these specific objects the map will not plot anything.
Now we get onto the slightly more difficult bit that might put off those who aren’t from a programming background. The next 7 nodes are all javascript functions which I have created to perform tasks that cannot be done via the standard Node Red pallet.
At this point it’s worth noting that I’m not a javascript programmer, I’ve used Python, Rust, Go, C and many other languages during my 40 plus year career but, javascript has never been one of them. I’m sure any seasoned javascript programmer will most likely raise an eyebrow at my attempt at javascript programming but, you need to remember that I’m doing this in my retirement and my enthusiasm for learning yet another programming language has wained somewhat!
So, getting back to the flow, each javascript function does just one task each of which is as follows:
Set rx_time – Sets the time the data was received/processed
Remove RR73 in tx1 – Remove decodes where RR73 is in TX1 instead of a valid callsign
Set Dynamic Icon Colour – Sets the icon colour depending on what type of call is decoded
Maidenhead to LON/LAT – Converts Maidenhead locator codes into LAT/LON Coordinates
Calc Distance – Calculates the distance between “My QTH” and the DX station
Calc Bearing – Calculates the bearing/beam heading to the DX Station from “My QTH”
Generate Search URLs – Generates the URLs for QRZ and my own online log lookups
Editing the Calc Distance function with debug info in the far right panel
Once all the functions have run the resultant data set is forwarded on to the WSJT-X Stations Map node where it is plotted real time on a world map.
To view the map point your web browser at your PC running Node Red as follows:
http://radiopc.your.domain:1880/worldmap/
Or if you haven’t got a DNS setup at home then just use the IP Address of the PC instead:
http://192.168.100.10:1880/worldmap/
Don’t forget that for all of this to work you must configure WSJT-X to send data via UDP on port 2237 otherwise the flow won’t be able to connect and listen for the decode data.
You may have noticed that there are 3 other nodes that I haven’t mentioned yet. The two green greyed out nodes are Debug nodes that can be enabled when required to help see what is going on in the flow. These debug nodes will display data in the debug panel on the right of the flow editor screen when they are enabled, they are extremely useful for debugging!
The third is the blue My QTH node, this contains data pertaining to my QTH that is plotted on the map using an orange icon. You can easily edit this node to point to your QTH instead.
WSJT-X Node Red map showing orange icon denoting my QTH
Once the flow is deployed you’ll be surprised how quickly the data starts to be plotted on the map. Stations calling “CQ” are shown by Green icons and stations that are in a QSO with another station are denoted by the Red icons.
Each icon is clickable and will present all the information collected by WSJT-X for each station viewed.
WSJT-X Node Red World Map showing FT8 stations realtime on the 12m Band
The popup also has two clickable entries, one will take you to the qrz.com page for the station being viewed and the other will search my logs to see if I have worked that station already and if so it will open a new tab showing the information.
Node Red Function Editor showing the Generate Search URLs function
You can edit the “Generate Search URLs” node so that it points to your online logs search engine so that you can view your own log data instead of mine.
Below is a close up of the popups that are displayed when each icon on the map is clicked. The popups show the information collected from WSJT-X for each station plotted on the map.
Left – Green “CQ” Popup and Right – Red “TX1” in QSO popup
If you fancy trying this out for yourself but, don’t fancy creating all the nodes in the flow manually then I have made an export of the flow available for download. All you have to do is download the file, unzip it and then import it to Node Red and you’ll have everything built ready to play with.
For a couple of weeks now I’ve been playing with Node Red to add functionality to my digital mode applications.
To get to know how it all works I initially used Node Red to create a series of dash boards for my servers and virtual machines to show realtime information on CPU temperature, CPU load, memory usage and storage etc.
Node Red Flow to collect information from a virtual machine (VM)
This worked very well and I was soon able to generate the information I needed in a palatable format. This was a great way to get to know Node Red flow building and introduced me to using gauge and graph nodes in flows.
The resultant Node Red Dashboard for one of my Virtual Machines
Once I had mastered creating dashboards for servers/virtual machines (VMs) I then started to investigate using Node Red to plot data from WSJT-X on a map.
I currently use the PSKReporter website to see stations that I hear on a map as WSJT-X sends the data to the site automatically however, this information is always 5mins or more old. For some time I’ve been wanting to see the information realtime as it is received and so I was hoping to be able to achieve this via Node Red.
Node Red has nodes available for a multitude of applications all easily installed via the Manage Palette menu in the flow editor.
I installed the WSJT-X Decode and World-Map nodes and set about building a flow to capture the data and plot it on a world map.
Building a Node Red Flow to decode WSJT-X data and plot it on a World Map
Putting the building blocks of the flow together is fairly straight forward and easily achieved using the excellent flow editor built into Node Red.
I configured WSJT-X to make the decode data available via UDP on port 2237 and then started the flow by creating a UDP node that connects to WSJT-X using the same port. The data immediately started flowing and I could see the information via a debug node.
I can’t stress enough how useful debug nodes are in Node Red. You can add debug nodes onto any output on any other node to capture the data as it flows. This gives you the ability to check what you’re getting is what you expected and also to see the format the data is in. The debug data is displayed in the debug panel on the right of the flow editor in realtime and gives you a great view of what is going on in your flow.
I decided to start with capturing the data for stations calling CQ as this was easily identifiable in the JSON object coming out from WSJT-X.
Passing the output from the WSJT-X-Decode node into a switch node I added a rule that filtered out data containing “type: “cq” and passed it onto the next switch node that created a payload consisting of the station callsign, maidenhead grid square and type so that it could be passed onto the next node for processing.
The next node in the flow is a function, this is where it gets a bit tricky. To be able to plot data on the map we need the Lat/Lon coordinates of the station making the CQ call. Since WSJT-X uses maidenhead locator data I needed to convert this to Lat/Lon coordinates before passing the data to the map node to be plotted.
Since Node Red is written in Java all the functions have to be written in javascript. The problem here is that I am not a javascript programmer and so this meant I’d need to learn yet another programming language. Unfortunately Node Red doesn’t allow functions to be written in C, Rust, Go or Python, all languages that I know well and after retiring from over 40 years in the UNIX/Linux/IT world my enthusiasm for learning yet another programming language has wained somewhat.
Being so close to having a working solution I pressed on and after much head scratching I finally put together some javascript that converts the maidenhead locator information in to good old fashioned Lat/Lon coordinates. I’m sure a seasoned Javascript developer wouldn’t be impressed with my code but, it works and does what I need and so I’m happy with it for the time being.
WSJT-X FT8 stations calling CQ on the 60m Band plotted on a Node Red World Map
Once I had the location information converted it was just a matter of passing the data to the world map node in the correct format for it to be plotted realtime.
As you can see on the screenshot of the map above, it worked extremely well with stations popping up as they were decoded by WSJT-X.
I now need to refine the data sent to the map so that it shows the frequency the station is calling on, the time they made the CQ call and the mode (FT8/FT4 etc) being used.. I would also like to add the distance from my QTH to the station calling CQ to round the information off however, this will mean writing another javascript function which, I’m not sure I want to dive into just yet.
I also need to add into the mix stations that aren’t calling CQ but, who’s callsign and grid square are passed on from WSJT-X. This will mean I will then be able to add to the map those stations that are actively working other stations and maybe I might even be able to show a line between the two stations that are in QSO.
This has been a fun but, steep learning curve however, it will certainly add some great functionality into my radio room and enhance my radio HAM addiction even further.
The Weak Signal Propagation Reporting Network (WSPR) known as “Whisper” in the HAM community is a QRP/QRPp beacon mode that is used by many HAMs around the world to see pretty much realtime propagation on the HF bands.
I first started using WSPR when I lived in France some years ago and it proved invaluable for assessing antenna performance and directivity. It’s not a new mode by any means and nowhere near as popular as it used to be as it’s really been superseded by FT4/8 these days that provides the same functionality but, with QSO capability too.
Having an old RaspberryPi hanging around and reading about the WSPR software that’s available for it now I decided to put the Raspi to good use and build a WSPR beacon for the 20m band that I could leave on 24/7.
Having the EFHW Vertical at the end of the garden means that I can connect it directly to the Raspi without the need for an ATU as it’s fully resonant. (It’s actually resonant on 20m and 10m)
I normally run both my RaspberryPi mini computers completely headless and then SSH in to them from my MacBook Pro and decided this was the best way to go with the WSPR beacon too since the WSPR software is command line based and doesn’t require a GUI.
First thing to do was to upgrade the OS from Debian Buster to Bullseye. It’s been a while since I used the Raspi but, it fired up perfectly and connected to the LAN without issue.
After a little time I had the O/S updated to Bullseye and the Raspi was ready for the software build.
The WSPR program comes in source code only so, this means you have to compile it yourself. This isn’t a big job as it comes complete with a makefile.
Using a terminal run the following commands to download and compile the WSPR source code.
So first thing to do is install git.
sudo apt-get install git
Once git is installed I downloaded the software from the git repository.
It only takes a few seconds to download the software which is stored in a new directory called “WsprryPi”.
Before the code can be compiled there’s a small issue with the includes in one of the source code files that needs to be resolved so that the code compiles without error.
cd WsprryPi
vi mailbox.c
Using your favourite command line editor, ‘vi‘ in my case I added the following line into the include statement at the top of the code.
#include <sys/sysmacros.h>
Once added the full include statement looked like this:
Once done, I saved the file ready for compilation.
Compiling the code is easy, just run the make command and sit back and watch all the compiler messages scroll across the screen.
make
Compiling the WSPR source code
Once compiled without errors, I just needed to install the binary.
make install
At this point the software is ready to go.
I quickly soldered up a lead to go from the RaspberryPi GPIO pins to the Coax cable that is connected to the EFHW vertical antenna in the garden and connected it all up ready to test.
RaspberryPi 2 WSPR Beacon connected to EFHW vertical for 20m/10m bands
Pins 7 and 9 on the Raspberry Pi’s GPIO pins are where the signal is output. Pin 9 is the Ground pin, and pin 7 is the Signal pin. Pin 7 goes to the inner of the coax and pin 9 to the ground side of the coax.
The purple cable is the ethernet cable connecting the Raspi to my local LAN so that I can access it remotely via SSH. I’ve powered the Raspi off of the USB port on the wifi access point in the radio shack which is always on.
Once it’s all connected it’s just a case of starting the WSPR program from the command line as user root.
wspr -s -r M0AWS JO02 10 20m > ./wspr.log &
I run the WSPR program as root user so that it has the correct privileges to access the devices to communicate with the GPIO headers, if you want too start it as your normal user then you’d need to use sudo to gain the root privileges.
The command line options I’ve used are as follows:
-s
Check NTP before every transmission to obtain the PPM error of the crystal
-r
Repeatedly, and in order, transmit on all the specified command line freqs.
M0AWS
My Callsign
JO02
My Locator Square
10
The power being used in dBm
> ./wspr.log &
Redirects all output to wspr.log in the current directory and then puts the program into the background so that it is left running when I log out.
Once the program is started you can monitor progress by using tail on the log file.
tail -f ./wspr.log
The output you will see will be something like this.
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:06:01.015
TX ended at: UTC 2022-07-17 16:07:51.638 (110.623 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:08:01.015
TX ended at: UTC 2022-07-17 16:09:51.639 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:10:01.015
TX ended at: UTC 2022-07-17 16:11:51.642 (110.627 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:12:01.015
TX ended at: UTC 2022-07-17 16:13:51.639 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:14:01.015
TX ended at: UTC 2022-07-17 16:15:51.639 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:16:01.014
TX ended at: UTC 2022-07-17 16:17:51.639 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
Obtained new ppm value: 4.09996
TX started at: UTC 2022-07-17 16:18:01.015
TX ended at: UTC 2022-07-17 16:19:51.640 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:20:01.014
TX ended at: UTC 2022-07-17 16:21:51.638 (110.624 s)
Desired center frequency for WSPR transmission: 14.097100 MHz
Waiting for next WSPR transmission window...
TX started at: UTC 2022-07-17 16:22:01.004
TX ended at: UTC 2022-07-17 16:23:51.628 (110.624 s)
You can pass multiple bands on the command line if you want to hop around bands.
It’s also recommended that you add a low pass filter between the Raspi and coax connection to help suppress any harmonics that may be generated. You can make one easily enough using just a capacitor or there are a number of prebuilt low pass filters specifically made for the GPIO hat on the Raspi online.
With only 10dBm (10mW) output from the RaspberryPi it’s surprising the distances that the signal travels. In no time at all I had reports from all over Europe and as the day progressed reports started coming in from Iceland, the USA and Russia.
Map showing stations that heard M0AWS on WSPR
I used http://wspr.aprsinfo.com WSPR monitoring website to watch progress as the day went on and after 24hrs had been heard by a number of stations over 3000 miles away.
You can also get a more detailed view of reports from the WSPRnet website where you can query the database and create a detailed list of all decodes over a set period of time.
Detailed list of WSPR decodes
Since my EFHW Vertical is resonant on both 20m and 10m I’ll now run it for the next 24hrs on both bands to see what results I get.
I’ve had a few messages of late asking how I generate my WSJT-X log web pages for my website. The answer is pretty simple, I use a little BASH script that I started writing some time back and have gradually improved over the last few months.
If you have a PC running Linux or a RaspberryPi running any of the normal Debian/Ubuntu/Redhat/Fedora based Linux distro’s that are available today then, this script should work just fine.
I originally wrote this program using Python but, a friend of mine bet me that I couldn’t write it in BASH and so, I took up the challenge and this is the result. It’s pretty simple and uses all the normal UNIX command line goodies like grep, sed and awk.
The script also uses WWL to calculate the distance between two Maidenhead locator grid squares and so it’s important to have it installed before running the script. (The script checks for it at runtime and will exit if it is not installed!). You can install WWL using your package manager on your Linux distro or from the command line directly. Details are in the READ-ME.txt file.
Screen grab from my M0AWS WSJT-X Log web page
It’s important that you READ the READ-ME.txt included in the zip file before trying to run any of the scripts included as it details the variables that you need to enter values for to make the script run on your PC/Server. (Example entries are in place as supplied).
The variables are mainly just paths to files on your system and whether you want distances calculated in miles or kilometres. Other than that there’s nothing else required for it to run.
I’ve also included details on how to run the script automatically from a crontab so that your webpage can be updated automatically every few minutes/hours etc.
The script also includes code to add a PNG map file into the generated webpage so that there is a graphical representation of all your QSOs. The map isn’t generated by the script (something I need to add in the coming few months) so you’ll need to generate it yourself and then add it to the website either via the crontab script included in the zip file or manually.
I use the QSOMap website to generate my map image files and then have them uploaded to my web server via the crontab script.
The script processes 100 entries per second on my web server (Headless virtual machine running Ubuntu Server 64bit Edition) and so should be pretty fast on most PCs. It will run somewhat slower on a RaspberryPi so be patient!
You can download the script and associated information using the button below.
searchwsjtxlog v0.4 showing search for partial callsign
Being a UNIX/Linux command line guy I’m not a fan of a lot of these GUI based logging programs that are full of functionality I’ll never use. I currently have RumLogNG for the MacBook Pro but, I really don’t like it. It does many things I don’t need and not the things I really need when on air.
So I decided to write a bunch of command line based programs that do exactly what I want with minimal fuss. The first of these programs was adi2html, a simple program that converts the WSJT-X ADI log file into HTML so I can easily put it on my website.
I’ve now written a new program using the same BASH shell technique to allow me to search the WSJT-X log file quickly and easily whilst on air.
searchwsjtxlog is a simple little shell script that searches for either a full or partial callsign and presents the results instantly. I use this script a lot when working FT4/8 as I can see if I’ve worked a station before and on what bands easily and quickly.
The script works on Linux and MacOS Big Sur (Not got Monterey to test but, should work fine). If you find a bug please let me know and I’ll fix it as soon as possible.
Note: Since MacOS uses such an old version of the BASH it cannot handle spaces in the path to the wsjtx_log.adi file. On MacOS you’ll need to create a softlink to the file and then put the path to the soft link into the script for it to work properly. CD into the directory where you have saved the searchwsjtxlog.sh file and then run the following command:
Following on from the adi2html v0.5 release I’ve now added a little more code to include a map of the contacts detailed in the wsjtx_log.adi file so that there is a visual representation of the data included in the webpage. (Map must be generated externally and JPEG/GIF/PNG uploaded to website to be included in the webpage).
I’ve started using RUMLogng for logging on my MacBook Pro as it seems to be the most common logbook to use on the Apple Mac platform. I must admit that I’m not sure if I’ll carry on using it going forward as I’m not overly enamoured by it but, for the time being it’s all I have.
To get my logs on the blog I’ve written a small BASH shell script that reads the exported ADIF file from RUMLogng and creates a log in HTML format for each different mode.
You can see the output of the BASH script by selecting a log from the Logs menu at the top of this page.
If you want to be able to add your log to your own website then I’ve made the script available for download below.
Currently I’ve only tested the script with the few different modes that I have log entries for but, it should work for all modes. If you find any issues please contact me with all the details and I’ll look into resolving any problems.
As always, if you use my code to generate webpages on your website please consider leaving the footer in place so I get some credit, thanks!
Following on from my original article where I detailed the adi2htmlBash script that reads in the WSJT-X log and creates an HTML version of the data so that it can be presented in a palatable format on websites (See my WSJT-X log here), I’ve now released v0.5 for public consumption.
In adi2html v0.5 I’ve added colour to the webpage that is generated to make it easier on the eyes and rewritten the function process_log() so that it uses the column name to identify each field instead of the row position.
By making this change it should cope just fine if the order of the fields is ever altered in the WSJT-X log during its development.
As before the code can be downloaded using the link below.
If you use my code please consider leaving the footer in place so I get some credit, thanks!
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.Ok
