Yet when I hit the "Arm Away" there is no chime every 2 seconds, I'm guessing the Keypad doesn't see what device to chime. I've tried to add a rule that would point to this setting but that didn't work either.
The keypad chime, is just a signal device, it has no actual chime functionality and needs Rule Machine or Simple Rules to actually provide the audible chime functionality. This is a virtual switch device, which will toggle on/off when the chime function is running on whatever delay you set (2 seconds default) during a count down for selected commands.
You will need to setup a RM rule, to monitor this keypad chime virtual switch, and when it turns on/off, you can then trigger your actual chime/light device to operate as you want.
Let me know if there are additional issues when you are setting this up.