[RELEASE] Australian Bureau of Meteorology Data - Radar Images Data File

It's on my dashboard on my nest hub that's sitting on my desk! Yeah I had to run out to put the car under cover :grin:. Biggest hail I've seen in a few years, just under 10mm, luckily for only a short period.

Anyway I'm getting the same as you - no rain or clouds, only the background map.

1 Like

Ok, at least we have something in common.... that is not destructive...

It's tempting to literally let the dust settle on their website update, but it feels like that may have already happened. My guess is the site layout I rely on has changed. I'll see what I can find out. Alternatively I could refer the issue to the guys in Colorado I borrowed the code from (?).... :slight_smile: Last I remember (2-3 years ago) I think there was also an FTP option, so I might have an alternative if the site has changed.

Awesome. thanks! Looks like the feedback on the new site is pretty negative. I only use the phone app which is pretty much unchanged.

Yeah, me too

Might take a bit more brain power than I can muster tonight... Hopefully I will have some time in the next few weeks.

1 Like

No worries - when you have time to spare!

1 Like

Lol I just spent the last half hour using AI to help solve this, this is what I got back eventually (after a few twists and turns) which actually works. As suspected BOM changed the website but fortunately left the legacy maps mostly intact.

Please have a look at this when you can although the changes look pretty benign to me.

/*

  • Australian BoM Weather Radar Images Data File Driver
  • Retrieves URLs for radar images from the Australian Bureau of Meteorology (BOM) and stores them in a local file
  • that is linked to a separate device to display the radar images

*/
metadata {
definition(name: 'BoM Radar Images Data File', namespace: 'simnet', author: 'sburke781') {
capability 'Actuator'

  attribute 'lastupdate', 'string'

    command 'refresh'
}

}

preferences {
input (name: 'idr', type: 'text', title: 'Observation ID number (e.g. IDR043)', required: true, defaultValue: '')
input (name: 'dataFileName', type: 'text', title: 'Data File Name (inc. file extension, e.g. BoMRadar.json)', required: true, defaultValue: 'BoMRadar.json')

    input (name: 'locations',    type: 'bool',   title:'Locations Image', description: 'Include Locations Image?', defaultValue: true,  required: true )
    input (name: 'range',        type: 'bool',   title:'Range Image', description: 'Include Range Image?',     defaultValue: false, required: true )
    input (name: 'topography',   type: 'bool',   title:'Topography Image', description: 'Include Topography?',      defaultValue: true,  required: true )
    
  input (name: 'AutoPolling',     type: 'bool',   title:'Automatic Polling', description: 'Enable / Disable automatic polling',          defaultValue: true, required: true )
    input (name: 'PollingInterval', type: 'string', title:'Polling Interval',  description: 'Number of minutes between automatic updates', defaultValue: 15,   required: true )

    input (name: 'DebugLogging', type: 'bool',   title:'Enable Debug Logging',                   defaultValue: false)
    input (name: 'WarnLogging',  type: 'bool',   title:'Enable Warning Logging',                 defaultValue: true )
    input (name: 'ErrorLogging', type: 'bool',   title:'Enable Error Logging',                   defaultValue: true )
    input (name: 'InfoLogging',  type: 'bool',   title:'Enable Description Text (Info) Logging', defaultValue: false)

}

// Standard device methods
void installed() {
debugLog('installed: BoM Radar Images device installed')
}

void updated() {
debugLog('updated: update process called')
refresh()
updateAutoPolling()
}

void refresh() {
debugLog('refresh: Refreshing radar images')
retrieveImageURLs()
debugLog('refresh: Refresh complete')
}

// Preference setting management methods
void setIdr(String pidr) {
idr = pidr
infoLog("IDR set to ${pidr}")
}

void setDataFileName(String pdataFileName) {
dataFileName = pdataFileName
infoLog("Data file name set to ${pdataFileName}")
}

// General driver-specific methods
void retrieveImageURLs() {
debugLog('retrieveImageURLs: updating radar image data')

// Use the OLD regional BoM site which still works!
def getParams = [
    uri: "https://reg.bom.gov.au/products/${idr}.loop.shtml",
    headers: ['User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'],
    contentType: 'text/plain',
    textParser: true,
    timeout: 30
]

try {
    asynchttpGet('retrieveImageURLsCallback', getParams) 
} catch (Exception e) {
    errorLog "retrieveImageURLs: call to update radar images failed: ${e}"
}
debugLog('retrieveImageURLs: process complete')

}

void retrieveImageURLsCallback(response, data) {
if (response.hasError()) {
errorLog("retrieveImageURLsCallback: HTTP error - ${response.getErrorMessage()}")
return
}

if (response.getStatus() != 200) {
    errorLog("retrieveImageURLsCallback: Unexpected status code - ${response.getStatus()}")
    return
}

String shtmlResponse = response.getData()

String[] lines = shtmlResponse.split('\\r\\n|\\n|\\r')
def images = lines.findAll { it.startsWith('theImageNames[') }

debugLog("retrieveImageURLsCallback: Found ${images.size()} radar image lines")

if (!images || images.size() == 0) {
    warnLog("retrieveImageURLsCallback: No radar images found")
    return
}

String staticImagesJson = '{\n'
int b = 1
int f = 0
staticImagesJson += "\"background0${b}\": \"https://reg.bom.gov.au/products/radar_transparencies/${idr}.background.png\"\n"
if (topography) {
    b++
    staticImagesJson += ",\"background0${b}\": \"https://reg.bom.gov.au/products/radar_transparencies/${idr}.topography.png\"\n"
}
if (locations) {
    f++
    staticImagesJson += ",\"foreground0${f}\": \"https://reg.bom.gov.au/products/radar_transparencies/${idr}.locations.png\"\n"
}
if (range) {
    f++
    staticImagesJson += ",\"foreground0${f}\": \"https://reg.bom.gov.au/products/radar_transparencies/${idr}.range.png\"\n"
}
staticImagesJson += '}'
debugLog("retrieveImageURLsCallback: background images JSON = ${staticImagesJson}")

String imagesJson = '{\n'
int i = 1
images.each { line ->
    // Extract the filename using regex
    def matcher = (line =~ /"([^"]+\.png)"/)
    if (matcher.find()) {
        String imagePath = matcher.group(1)
        // Remove leading /radar/ if present to avoid duplication
        if (imagePath.startsWith('/radar/')) {
            imagePath = imagePath.substring(7)  // Remove "/radar/"
        }
        String fullUrl = "https://reg.bom.gov.au/radar/${imagePath}"
        imagesJson += "\"image${i}\": \"${fullUrl}\""
        if (i != images.size()) { imagesJson += ',' }
        imagesJson += '\n'
        debugLog("retrieveImageURLsCallback: Added image ${i}: ${fullUrl}")
        i++
    }
}
imagesJson += '}'
debugLog("retrieveImageURLsCallback: radar images JSON = ${imagesJson}")

updateDataFile(staticImagesJson, imagesJson)
String lastUpdate = new Date().format('yyyy-MM-dd HH:mm:ss')
device.sendEvent(name: 'lastupdate', value: "${lastUpdate}")

}

void updateDataFile(String pbackgrounds, String pimages) {
def imageCycle = findChildDevice('radar','ImageCycle');
if (imageCycle == null) {
createChildDevice('Image Cycle', 'radar', 'BOM Radar', 'ImageCycle')
imageCycle = findChildDevice('radar','ImageCycle');
}
imageCycle.setFullImageList(pbackgrounds,pimages)
}

// Child Device methods

String deriveChildDNI(String childDeviceId, String childDeviceType) {
return "${device.deviceNetworkId}-id${childDeviceId}-type${childDeviceType}"
}

def findChildDevice(String childDeviceId, String childDeviceType) {
getChildDevices()?.find { it.deviceNetworkId == deriveChildDNI(childDeviceId, childDeviceType)}
}

void createChildDevice(String childDeviceDriver, String childDeviceId, String childDeviceName, String childDeviceType) {
debugLog("createChildDevice: Creating Child Device: ${childDeviceId}, ${childDeviceName}, ${childDeviceType}")

def childDevice = findChildDevice(childDeviceId, childDeviceType)

if (childDevice == null) {
    childDevice = addChildDevice('simnet', childDeviceDriver, deriveChildDNI(childDeviceId, childDeviceType), [label: "${device.displayName} - ${childDeviceName}"])
    infoLog("createChildDevice: New ${childDeviceDriver} created -  ${device.displayName} - ${childDeviceName}")

}
else {
debugLog("createChildDevice: child device ${childDevice.deviceNetworkId} already exists")
}
}

//Automatic Polling methods

void poll() {
refresh()
}

void updateAutoPolling() {
String sched
debugLog('updateAutoPolling: Update Automatic Polling called, about to unschedule polling')
unschedule('poll')
debugLog('updateAutoPolling: Unscheduling of automatic polling is complete')

if (AutoPolling == true) {
sched = "0 0/${PollingInterval} * ? * * *"

   debugLog("updateAutoPolling: Setting up scheduled refresh with settings: schedule(\"${sched}\",poll)")
   try {
       schedule("${sched}",'poll')
       infoLog("Refresh scheduled every ${PollingInterval} minutes")
   }
   catch (Exception e) {
       errorLog('updateAutoPolling: Error - ' + e)
   }

}
else { infoLog('Automatic polling disabled') }
}

void getSchedule() { }

//Logging methods
void debugLog(String debugMessage) {
if (DebugLogging == true) { log.debug(debugMessage) }
}

void errorLog(String errorMessage) {
if (ErrorLogging == true) { log.error(errorMessage) }
}

void infoLog(String infoMessage) {
if (InfoLogging == true) { log.info(infoMessage) }
}

void warnLog(String warnMessage) {
if (WarnLogging == true) { log.warn(warnMessage) }
}

1 Like

In my line of work (Data Engineer (technically a senior)) I should advocate for the use of AI... but I still somehow doubt it in certain circumatances.... But hey, if it works, it must be right :wink:

But seriously, great work, and thanks for sticking at it more than I had the motivation to.

Hopefully there isn't too much more hail for us down this way....

1 Like

Perhaps you could produce your own AI version of the driver.... :wink: Maybe even @cometfish 's BOM driver... :grin:

1 Like

hmmmmmmmmm :thinking:

1 Like

The changes do mostly centre around the change from HTTP to HTTPS and a www address to reg., e.g.:

uri: "http://www.bom.gov.au/products/${idr}.loop.shtml",

vs

uri: "https://reg.bom.gov.au/products/${idr}.loop.shtml",

There are some other points that I will also look at... But shouldn't be a big update to make. If I get some time at the weekend I'll try to sort it out.

Nice work to make use of a tool I haven't really embraced like I should (AI).

1 Like

Yeh I did ask it to also do some optimisations but thought it was good enough at this stage to get it working. It sent me initially around in circles to try and get the javascript output from the new site lollll.

I'm going to start using it a lot more (perplexity pro - recommended by my BIL - they are very keen to get this out there and they're doing deals with quite a few companies to give out free subs to account holders). Got a few projects on the go which I want to mash up code from different sources.

1 Like