Having cleaned up your head, you might also like to de-clutter you WordPress options table. Sure, it won't alter the look or performance of your site—but it's what's on the inside that counts right? Unfortunately deactivating a plugin or deleting it from wp-content/plugins will most often not clear out the data it saved to wp-options.
How do I know what options are stored?
If you are currently logged in as site administrator you can see all the options stored in your wp-options table by visiting /wp-admin/options.php. Although you can change these options, you can't remove them:

How do I remove obsolete options?
Firstly, some well-beahaved plugins will actually do this for you. For example, Quoter offers to tidy up its own mess:

Unfortunately such graces do not appear to be commonplace.
Secondly, there is a plugin in beta as of this writing called Clean Options. Sadly when I ran this it didn't identify any of the options left behind by long-disused plugins.
Thirdly, you can use phpMyAdmin. Say you deleted Ultimate Tag Warrior from your plugins folder after migrating to the WordPress built-in tagging system. To remove UTW's leftovers from your options table, use the following SQL syntax:
DELETE FROM wp_options WHERE option_name LIKE 'utw_%';
The % is a wildcard, so it will remove all 24 UTW-related rows in wp-options.
After spring cleaning my options table went from 469 to 205 rows—evidence of my experiments with all sorts of plugins when I was a WordPress newbie.









Thanks for the heads up, Bruce!
I almost forgot about this sort of hidden option. Now lets see how much there is to clean on my side / site.
Get to it Henk: it's quite therapeutic, I assure you ;-)
Groovy. Hey, I like your new favicon/gravatar!
Gee, I've cleaned up to about +100 rows, mostly of themes I've experimented with over the last months.
Now all I need is to get rid of are these slow queries in MySQL - like this:
User@Host: wp_user[wp_user] @ localhost []
Thanks Rob. It started life as a road sign ("Beware of kiwis") in the beautiful Tongariro National Park. I cleaned it up, straightened it, added transparency etc in Photoshop. Scaled pretty well from the original 1706 x 1706 pixel version!
You turning this into a competition Henk? Mine's shorter than yours, huh? ;-)