You might also be able to do like @aaiyar and some others have done and move all those devices to a separate zigbee controller. The only trick is connecting that controller back with HE which can be done with Node-RED or possibly MQTT.