In the above settings I don't have the "Don't turn off if already on" option checked because it has started causing problems. I have been using these settings with that option on for a year with no issues. Just recently (last few weeks) the Kitchen Main Lights is not turning off although the other 2 lights are turning off. If I then go to my dashboard on my computer and turn off the kitchen lights manually (which the dashboard thinks are off already) it does nothing until I turn them on and back off again. Works fine if I turn off that option hence why it is off atm.
This makes me suspect a device problem. I'd leave any apps, including Motion Lighting or Dashboard, out of the picture to narrow this down and just focus on the device page. Under "Devices" (on the left-side menu), click/tap into your Kitchen Main Lights to look at its device page. Pay attention to "Current States" (top right if you're on desktop, somewhere towards the top if you're on mobile). You should see "switch: on" change to "switch: off" and vice versa in real time (within milliseconds or seconds, depending on your device) as you send commands like "On" and "Off" to the device. These are the same commands Motion Lighting and Dashboard are sending, and the value under "Current States" is the same value these apps are reading, so if that ever shows unexpected values, an app would have problems, too.
My guess is that you'll find some oddities there if you play around enough (if not--and maybe even if so--feel free to add back in the Motion Lighting app and see if states show values as expected based on what the app is doing). How to fix that is another problem, but at least you've identified a caused and ruled out Motion Lighting or Dashboard themselves ... if this is correct.
I think it may have been a device problem. I turned it back on to try test your theory and now it won't do it anymore so will try if it ever starts again. Ghost in the machine.