Yeah, sorry, that was poorly worded. I had assigned the device to a couple of dashboards, but I had not actually created any tiles which used the device yet.
From logs I was able to see that something with the attribute "button" was throwing the >1024 character warning message, so I used this app to figure out which app or device was using an attribute with that name.
Then after that, it was easy to figure out which device was the problem and some creativity was needed to reduce the character count for the iframe that that "button" attribute was referencing.