Yes, you are correct, these devices still get notifications while asleep. However, these devices, while asleep, will save power for quite a bit of time by dropping/minimizing the WiFi connection. This is, as @roberto mentioned, a designed power saving feature. The iPhones and iPads will still wake up the WiFi connection periodically, check to see if there is anything to do or receive, and then go back into this power savings mode.
The iPhone HTTP Presence Sensor driver polls the device via a HTTP call to see if it is on the WiFi network about every 1 minute. While aspleep, iOS devices will stop responding for a decent chunk of time, and then resume responding for a while, then stop responding, etc... This is the phone trying to save battery life by minimizing its use of its WiFi radio. It also explains why the HTTP Presensce Sensor method is only part of a solution for presence detection.
If there is a magic set of iOS settings that would keep the phone's WiFi active the whole time (without trashing battery life), many of us would be very interested in know what those settings are.
UPDATE: Once home, I ran a quick PING test against my iPhone 11 and my iPad Pro (WiFi only) and both devices failed to respond to ping requests about 80% of the time when their screens were off (i.e. asleep.). Both devices start responding to ping requests for a short period of time, and then they stop for a while, start for a while, stop for a while. It is very consistent and appears to be deliberate.