I believe this is the root cause. I'll link a thread I found on the smart things forum about bulbs in general. I think the issue is that as I added more bulbs, they went from being connected to the hub, to be connected to each other and then the hub, which is causing issues since they're poor repeaters. The hue hub apparently controls bulbs slightly different, and it seems to fix the issue, at least according to a few people in the other thread. I was able to get a hue hub for under $40 shipped, so I figured I'd try it before switching over to z-wave switches or changing to a different brand bulb.
Here's the thread in question:
And while trying to find that I found this, which seems to explain the problem exactly.
