Are you changing the Device Name? Or the Device Label? If the Label field is populated, it will override whatever is in the Name field. Might that explain the behavior you’re seeing? Also, be sure to click the Save button after making your changes on the device details page.
Also would be a good idea to know where you're looking. "Past Logs" will show the old name for some time, for example--until the earliest log entry (which determines what is displayed) is automatically purged after the space limit is reached. This depends on how log-gy your devices are. Just one example!