It is in priority notification, but I set it to alarms only.
I see. That's pretty odd. When the device is unlocked, are you hearing all notifications or just certain ones? If only certain ones, it might help narrow down the cause.
One thing you might try, that might not help but won't hurt, is to wipe the cache partition.
1. Ensure the device is powered off, and unplugged
2. Press and hold the Volume Down button and Power button (located on the right edge of the device) for approximately 3 seconds then release.
3. From the "Boot Mode Selection Menu" screen, select Recovery.
Note: Use the Volume buttons to scroll through the menu options and the Power button to select.
4. From the "No Command" screen (Android figure lying on his back), press and hold the Power button, tap Volume Up once and then release the Power button.
5. Select "wipe cache partition".
Note: Use the Volume buttons to highlight and the Power button to select.
6. "Reboot system now" will be highlighted, press the Power button to select it.
Note: Allow several minutes for the reboot process to complete.
A more extreme measure would be to factory reset the device, but you probably want to exhaust all other possibilities before going that far, as it's not a guarantee that it will fix the issue.