I have the turn off if already partially off enabled as shown above however I just tried disabling that and it appeared to make no difference. I can work around it by using the pre-set per mode option it seems because then it can't set the preset lights because it doesn't have a preset-off for that mode which solves my problem 
Here is the log from when I turn that option off showing it set the lights to the preset, then wouldn't turn them off. Notice the last (top) line happens after the 30 second delay that seems to be wanting to turn them off but realizes the mode is wrong. It seems to me the same logic should be in the part of the code that activates the preset-off which doesn't seem to check that the lights are not in active mode right now.
2022-11-12 11:05:37.731 PM [info] Office Lights not Active
2022-11-12 11:05:34.029 PM [info] Grace period before off: 30 seconds
2022-11-12 11:05:33.975 PM [info] setLevel: Office Tripod Lamp 💡, level: 50
2022-11-12 11:05:33.852 PM [info] setLevel: Office Ceiling Lights, level: 10
2022-11-12 11:05:33.763 PM [info] Activating Preset Off
2022-11-12 11:05:33.756 PM [info] Turn Off Event: 'Office Lights (Active)'
2022-11-12 11:05:29.196 PM [info] Activating for Night
2022-11-12 11:05:29.189 PM [info] Activate from 'Office Lights'
If I change to using preset per mode, then the log is different, it doesn't manage to turn on any lights (which is good since there isn't a preset for night and this is my workaround)
2022-11-12 11:13:22.431 PM [info] Grace period before off: 30 seconds
2022-11-12 11:13:22.401 PM [info] Activating Preset Off for Night
2022-11-12 11:13:22.394 PM [info] Turn Off Event: 'Office Lights (Active)'
2022-11-12 11:13:19.583 PM [info] Activating for Night
2022-11-12 11:13:19.577 PM [info] Activate from 'Office Lights'
