May I suggest you try exposing yourself to chance and new discoveries by not buying any new music that costs over $3 an album unless you buy it directly from the band?
Find the clearance shelf at your local indie record store (Streetlight on Market's is in the upstairs back right corner & prices are $1 or $1.95). Check out yard sales & thrift stores.
Sign up for LaLa and turn the stuff that didn't work out into what you know you want.
Discardia 
And there’s lots of free entirely legal music online these days, to listen to, and to download. Browse around myspace.com, lots of bands show their stuff there. And remember mp3.com, from way back when in the Golden Age? They still exist, and still have free music.
LikeLike
Don’t forget the library- especially in larger towns- I’ve been going a little crazy and getting lots of great music. And it is FREE!
LikeLike
And don’t forget about magnatune.com — it’s basically music using the shareware model.
LikeLike
mixed-tape.com
it’s a promo site for mercedes benz. you can listen online and all the tracks are downloadable for free!
LikeLike
Can you also buy that from outside the US?
LikeLike
Note: This post is from 2006.
LikeLike