[RELEASE] wx-ApiXU-driver

community_driver

#162

-- == wu-ApiXU-Driver Release v1.4.3 == -- <— click here :slight_smile:

-- == Luxuriant-Driver Release v1.5.3 == -- <— click here :slight_smile:


#163

I'm still getting lux at 5, and it never changes. Also my bretwixt is: fully night time, when it's 8:40am here.

Also, my "mytile" has issues with location. It's blank. I tried using the "overright default city name", but it doesnt do anything for me.


#164

betwixt = fully night time means that 5 lux is intentional. :slight_smile:

The problem therefore is to find out why the time it thinks it is is not correct.

Here's what happens... SunRiseSunSet.com is queried for all the time info. For our purposes we need Civil Twilight Start, Sunrise, Noon, Sunset, and Civil Twilight End and they arrive in universal time,

Slice-of-Day is one of 5, in order:

  • "between twilight and sunrise"
  • "between sunrise and noon"
  • "between noon and sunset"
  • "between sunset and twilight"
  • "Fully Night Time"

If it's not one of the first 4, then it's Fully Night Time.

What you're telling me is that 1) SunRiseSet isn't correct for Today.. If it's from yesterday, for example then the current time would be AFTER yesterday's Civil Twilight End.. and then Fully Night Time would be correct. Or 2) local time is wrong.

Therefore, look at the sunRiseSet stored values and verify that it's correct... adjusting for your Timezone is the critical part...

For me, I've gotten this from the most recent call to Sunrise-Sunset.com:

For me, -8UTC, I look at sunrise and sunset. In this timezone, those are always going to be one day apart. So it shows sunrise as the 21st, which it is, and sunset as one day more: 22nd. That quickly tells me I have a "today" acquisition of sunrise/sunset.

All of that may be tooooooo theoretical.. :slight_smile:

Clicking Save Preferences will always cause all of the Loops to be queried. Lux is calculated first, unless there's no SunRiseSet yet. So SunRiseSet is queried, which won't work unless Lat/Long is set. In parallel, APIXU is queried to get Lat/Long.

Please screencap the sunRiseSet stored value and then click Save Preferences. 4 seconds later the sunRiseSet values should be updated. That section of the Device Info page is not dynamic and the refresh done because you clicked Save Preferences has already completed before the 4 seconds. Therefore, 5 seconds after clicking Save Preferences, refresh the page and look again (screen cap) at sunRiseSet values. PM them to me if you'd rather.

There's a "last resort" option too :smiley:

In the Source, remove the // for line 177:
// command "pollSunRiseSet" // --- delete for Release
save it, then refresh the page and a button will "magically appear." Enable debugs, open a browser tab for logs and click the pollSunRiseSet button.

One other area for exploration is the Scheduled Jobs. There should be 4 of them with cron style schedules:

The schedule numbers for poll and updateLux will have /number where the number is the interval you've selected:

03%20PM

I suspect you don't have a pollSunRiseSet. Clicking Save Preferences will cure that. :slight_smile:


#165

@csteele

The Luxuriant-Driver seems to still be using a float instead of an Integer on the newest version:


#166

hmmm.. well that's another error :frowning:

Fixed, thanks.


#167

CStelle,

Despite me pressing refresh, save preferences ten times this morning (I counted in the past logs!), it didnt work. I just tried again, for giggles, and it worked. It is now afernoon here, so there maybe something wrong with the logic working out "between sunrise and noon". Or it could be coincidence!

Nonetheless, here is my data, as requested.

and here are my relevant current states:
image


#168

@csteele

Morning again, and it's "fully night time". There must be some wrong logic for us close to the date line.


#169

I tested the LOGIC between last time and now.. and I can't see anything. What I did conclude is that it's non-trivial for me to test your timezone.

Testing the logic actually didn't take long, it's pretty much just subtraction to compare between two numbers.

As previously mentioned, a website is queried for 4 date/time values and those values are converted to a number (Epoch).

"between twilight and sunrise"
"between sunrise and noon"
"between noon and sunset"
"between sunset and twilight"

Then your date/time is pulled from the Hub and also converted to a number (Epoch).

After that it's just compare.. is "NOW" greater than X and less than Y.

The most fragile part is the fact that "NOW" comes from the Platform while the sunrise/set comes from external. If the date/time of your Hub is wrong... but that can't be because you'd have more than just LUX driving you crazy. :smiley: If the Latitude / Longitude of your Hub is massively inaccurate, then the sunrise sunset times would be for another part of the planet.

Are you using 24 hr clock?

Look again at your Current States.. Looking at what you posted last, 12:50 is going to be very close to celestial noon.. (not clock noon, but that moment the sun is directly overhead...) thus "betwixt' should be showing something with "noon" in the phrase :slight_smile:


#170

My #1 hypothesis at the moment is that Sunrise-Sunset.org isn't handling + offsets properly.

I moved my Hub to Sydney's Apple store. :slight_smile: (-33.868937,151.206666) and logs seem to agree:

dev:4 2019-08-24 06:24:27.134 am debug millis: 2019-08-24T06:24:27+10:00, 1566591867000, 1566503967000, 1566505487000, 1566525470000, 1566545454000, 1566546974000
dev:42019-08-24 06:24:27.111 am infowx-ApiXU lux calc for: -33.868937,151.206666

6:24am
and Google says:

6:24 AM
Saturday, August 24, 2019 (GMT+10)
Time in Sydney NSW, Australia

However, sunRiseSet which seems ok, isn't on closer inspection. From the log line above,

debug millis: 2019-08-24T06:24:27+10:00, 1566591867000

And that seems correct:

1566591867 converts to Saturday August 24, 2019 06:24:27 (am) in time zone Australia/Sydney (AEST)
The offset (difference to Greenwich Time/GMT) is +10:00 or in seconds 36000.

Therefore it's a simple matter of finding out which pair of numbers THAT value sits between:

1566503967000, 1566505487000, 1566525470000, 1566545454000, 1566546974000

and just visually comparing the first 6 numbers of Local time: 156659xxxxxxx is beyond the final number: 156654xxxxxxx

Indeed, Full Night is correct from THAT comparison.

A more correct initial value of that set would be: 1566590067000 vs 1566503967000

05%20PM

Even their web interface seems off -- saying today is the 23rd but tomorrow, is the 25th.

22%20PM

I've sent a message off to sunrise-sunset.org.. we'll see if they reply.

My hunt continues... :slight_smile:


#171

@csteele wanted to let you know that I've been having some problems with the wx-ApiXU-driver.

I have one virtual device that uses this driver. The device is used by Supertiles, and nowhere else.

Not too long after a hub reboot - less than a day - I start to notice slow down, lagging, some rules not firing, etc. Rebooting fixes it, for a while. I deactivated the device using the wx-ApiXU-driver and paused the Super Tile it's used by, and that has also taken care of the problem. The hub has been humming along with all rules firing as expected.

I have other Super Tiles that are not paused, so I don't think it's an issue with that app.

I'm also using the Luxuriant-Driver with a single virtual device that's used by both RM4 rules and Super Tiles, and it doesn't seem to be having the same issues.

wx-ApiXU-Driver is version 1.4.3. I've been having problems for a while. I updated hoping it would help, but it hasn't.


#172

I can't diagnose the issue from the information you've offered but I'm disheartened to hear it's occurring. My use is rather similar, one device that is used on Hubitat Dashboards and nowhere else. A Luxuriant device that is in RM Rules and Dashboard. My devices though, are on my 3rd hub, along with all my other 'Internet facing Apps' -- Alexa, Chromecast, etc. -- it is the 'glue' hub for everything else. Zigbee motion sensor Events traverse this 3rd hub to be sent to the hub that needs to know and to thus turn on lights. Any slowdown of 'coordinator' hub would be very noticeable.

The whole reason I stuck my fingers into APIXU, and creating "wx-ApiXU-Driver" (and Luxuriant) was to reduce the chance of slow downs. I had always considered Bangali's APIXU-Weather driver was perfect except for the potential to slow the Hub due to DB overloading, and a single poll loop that did everything in one pass.

I've split the single poll loop into

  • poll (of ApiXU.)
  • pollSunRiseSet (reduced to once a day instead of every 5, 15 or 30 mins.)
  • updateLux (that needs values from ApiXU and Sunrise-Sunset, found in memory.)

In addition, I've reduced the potential for DB overload, by allowing each of the weather attributes to be individually enabled. After that, I replaced each website query with AsyncHTTP to break the query into request/response and to let the hub have 100% of the time between for it's self. (The other way has the driver waiting for the response, which may be very fast to a human, but is glacial to a quad core CPU.)

The two drivers do the same job.. polling ApiXU, Sunrise-Sunset and calculating Lux, but wx-ApiXU-Driver spreads those events across the intervals between polls. You can see the results in Scheduled Jobs:

poll (of APIXU) occurs on the 18th second every /30 (30 mins, per Preferences) and updateLux occurs on the 0th second, every /5 (5 mins, per Preferences.) pollSunRiseSet runs on the 23rd second of the 10th minute of each new day. [updateCheck, a variation of Cobra's version checking, runs once on Fridays.]

Those 4 Scheduled jobs are the ONLY events triggering the driver. Even if the Internet is down, all that happens is the hub sends the request and gets no response, times out and tells the Driver causing an error message: "wx-ApiXU weather api did not return data" -- it's a fraction of the work it has to do if the driver actually got a response :slight_smile:

I'd be happy to work with you to try and figure it out when you get to the point of enabling it again.


#173

Thanks for working with me on this.

The issue is to do with betwixt. It goes to fully night time at the appropriate time, but it misses the morning routines (which I presume would be twilight, then between sunrise and noon).

As soon as the first poll/refresh occurs after noon, the betwixt changes to between noon and sunset. For the 22nd the illumanated (see q below) updated during PM, but not for the past two afternoons. It's remained at 5 lux, as has illuminance for every day.

Also, what is the difference between illuminance and illuminated.


#174

Date/Time math is one of the top 5 hardest problems programs face, I believe. Fundamentally the problem is that the values returned are UTC and more importantly, the always moving ‘Line of Date’. The website returning the values believes it is still ‘yesterday’ for 10 hours after it’s become ‘tomorrow’ Right this minute, it shows the time in Sydney to be 11:54 on the 24th, and that’s correct. But in 7 min it will say it’s 12:01 on the 24th BUT correctly calculate that tomorrow is the 26th.

It’s a free site, and I’ve asked them to look at it but I’m not expecting a rapid response because it might be just one person, part time, running it. And they are unlikely to believe me, at first.

I can query for tomorrow’s date but only if I calculate where midnight is on the planet and which side of that ‘Line of Date’ the hub actually is. It’s not a mathematical Line but a a geopolitical one and thus goes right back to my first words of this message.

The website did exactly as it predicted. It’s 12:05 in Sydney now but it says it is August 24th still. Tomorrow is correctly shown to be the 26th. For the next 10 hours, there is no 25th :smile::smile::smile:


#175

illuminance is a number.. It can be used in RM for comparisons. illuminated is a string with ", lux" appended and is used for display, I presume. That's what I use it for... on a Dashboard.

02%20AM


#176

@csteele did not mean to sound like I'm complaining... much :wink:. I'm really happy with luxuriant-driver. And I was happy with wx-ApiXU-driver until it appeared that the slowdowns were somehow connected.

I only have one hub, and I'm not likely to add another one at this time.

I have the same Scheduled Jobs as you showed:

I've enabled the device again, but left the Super Tile paused. We'll see what happens over the next day or so. To be honest, I need the luxuriant-driver for the lux value as input into some of my rules. But I don't have a device to use as a dashboard -- most of my automations just happen, or I tell Alexa to do them -- so the weather tile was just for my own edification and enjoyment. It's not strictly necessary.

If I do have a slow down again, what info would be most helpful? I'm happy to gather up whatever you need.


#177

ok @mike I have the problem fixed in both Luxuriant and wx-Apixu... or as Bruce says: "found and fixed, next release."

Would you like to test? I've PM'd you the latest.

The exercise also allowed me to see how to halve the number of comparisons.

Current States

betwixt : between sunrise and noon
illuminance : 8856
illuminated : 8,856 lux

11:23 AM
Sunday, August 25, 2019 (GMT+10)
Time in Sydney NSW, Australia


#178

-- == wu-ApiXU-Driver Release v1.4.4 == -- <— click here :slight_smile:

-- == Luxuriant-Driver Release v1.5.4 == -- <— click here

This release incorporates the fix needed for Oz. Also increased the poll of sunrise-sunset to 3 times a day, again to help Oz and anyone that’s far away from the Prime Meridian.


#179

I'm a little confused why passing the date to the API doesn't solve your problem. This is the link that I use.

def todayDate = new Date().format('yyyy-MM-dd')
def requestParams = [ uri: "https://api.sunrise-sunset.org/json?lat=$location.latitude&lng=$location.longitude&date=$todayDate&formatted=0"

and that query is scheduled every day around 12:10 am so that the poll is done on the correct date. That should work no matter where you are located since you are passing the date you want it for to the API. For example, it is 11pm here in New York on the 26th. If I request it for the 27th, astronomical twilight ending, which would be after midnight UTC on the 28th, reports the correct time of 2019-08-28T01:36:41+00:00.

Similarly, if I request the parameters for the 28th right now for Sydney, it reports back the sunrise time of 2019-08-27T20:19:26+00:00 UTC, which is 8pm the night before UTC which translates to 6am AEST on the 28th in Sydney, correct? Or have I totally screwed up my time zones, which is all together possible.

When I visit sunrise-sunset.org it reports the correct current date/time that I can see. But I also don't see where it passes that info back from the API. It only passes back the scheduled times. From what I have been able to test, using the current date fixed all of the issues. If there's something I'm missing, please let me know.

Amen brother!!! Always a PITA. One of the reasons I have an online Epoch converter bookmarked. :slight_smile:


#180

@csteele, I re-enabled the device using wx-ApiXU-driver, but left the app using that device disabled. So far (3 days now), everything is working just fine. I have to think that the issue was the app, since I turned it back on before your latest update. I'm going to call it good for now.


#181

Thanks @csteele for working with me on this. It's been working flawlessly, and it even has the correct name in the dashboard, which it never did before this last update (stupid aussie weather providers).