A couple of suggestions off the top of my head:

• Remove external SD card reboot without card.
• Try a Different Charger.
• Enable USB debugging mode (this should prevent the kies interface)
• Clear cache and dalvik cache from recovery mode.
• If you are rooted then install Titanium Backup make a complete backup and reinstall your ROM and restore your backup.
• Uninstall non-important apps and see if the problem goes away.

Some info from your side might help,
like: Rom name? Rooted? Kernel? Last thing that you were doing before the problem started.

 

Try this first:

With the SD card in the handset, connect to your PC as "USB Storage".
When the card appears in Explorer as a removable drive, right-click the icon and select "Properties" then "Tools", "Error-checking - check now".
Enable "Automatically fix file system errors" (if not by default) and let the scan run.
When complete, use the "Safely remove hardware..." option in Windows' System Tray before disconnecting the handset.

 

@Slug
He mentioned that his problem was persistent even without the external SD card, so I don't think that it would work. But Hey it's worth giving a shot, I would just recommend that in addition to scanning the external SD card also run a scan on the internal SD card.

@Stixeh
How well are you versed with flashing and/or rooting?

I am going under the assumption that you are on a fully stock rom, stock recovery.
The following is the way to clear cache via recovery.

Step 1. Boot into recovery mode.
1.a. To boot into recovery mode, first switch off your phone.
1.b. Hold volume up + menu + power button while booting and don't let go till you see a screen similar to the one in the attached image

Step 2. Use volume buttons to scroll to the option clear cache and use menu button to select
Step 3. Reboot device

Clearing the dalvik cache on stock recovery is more complex, and can be done only if you are rooted.

 

Another thought came to my mind, but mind it that this is untested on a stock recovery and may not work for you.

• Download the GS2blitzer script by hawker (from this post).
• Copy it to your internal sd card.
• Boot into recovery mode (you can boot into recovery mode using the method outlined in the above post.)
• Scroll down and select apply update from sdcard.
• Select the zip file that you downloaded.
• Select Yes.

Hopefully this should wipe the cache and dalvik cache without root.
But i caution you, this is and untested unverified method.