You should switch to using the Alexa Skill. First remove the Alexa App, then add the skill via the Alexa App on your phone, and set it up there. That will install the Alexa Skill app on the hub, and allow you to select the devices.
When removing the old app and installing the new will Alexa need to have all the devices re-configured/named and routines fixed? Or will all the existing devices in Alexa just get a new 'link' so to speak to HE through the new Alexa skill?
When you authorize the Alexa Skill, it will offer a selector for you to select those devices it is authorized to control. Just select the ones you want, and the rest is automatic -- nothing else to configure. You do it all in the Alexa app.
You can continue to use the Alexa App if you want. It's still there and it still works.
If all you need is to control switches and dimmers, and the app is working, you don't need to switch to the skill. But know that the skill has expanded capabilities, if you ever want to get more out of Alexa compatible devices, like locks, thermostats, garage doors, etc.