Welcome to the world of audiophiles, where good enough usually isn't!
I'm by no means an expert, but like to think I know a thing or two about it. First off, lets get the digital vs analog thing out of the way.
First, there was analog. Analog stores all the "data" about the sound that the receiving equipment could hear, continuously. Back when everything was analog, this meant that you had as close to the real thing as possible.
Fast forward a hundred years and you've got digital. Like Analog, digital stores what it can hear. Unlike analog, digital samples the sound within a certain frequency range, and at a certain refresh rate. It does so through a device called a DAC (digital/analog converter). The better quality the DAC, the better your digital recording, and the larger your file size. Digital means you lose quality. However, that doesn't necessarily matter, since humans can only hear so much anyway. Some people swear by Analog, some people say digital is better... I say use what you like and is convenient for you.
Out of the above basics the important thing to note is that, with digital, the quality of your audio source is important. If you want real high quality, you want audio with the most data available. This generally means FLAC, WAV, or some other lossless audio codec. Most folks use MP3s, which are compressed (more data loss). If you chose to go the MP3 route, get the highest bitrate files you can.
Now here's the real kicker... the above stuff doesn't really matter. If you like the sound of something and it's good enough for you, well, congrats, you've found your happy medium. I know folks that have spent over $100k on their sound system, and I honestly can't tell the difference between it and my ~$1k setup in terms of audio quality.
So, I'd recommend analog/vinyl if you want to store records. I do think they sound "warmer" and better, but they take up a lot of space and need regular care. I view these as my "premium" listening service. I have my commodity turn-table (direct drive) with a good quality needle (replaced regularly) hooked up to a good quality pre-amp, and then pumped in to my tube amp (analog also) which is then hooked up to a nice pair of studio reference monitors (self-amped, designed to accurately reproduce sound).
I'm looking to expand to a Sonos system for my normal listening needs around the house. Sonos is a bit expensive, but the audio quality and features are solid and they provide great flexibility of listening. Since most of my newer stuff is digital, this works well for me.
Hope the above helps a bit. Below are a few links you might find useful.
http://lifehacker.com/5883665/how-to-be-an-audiophile-on-the-cheap
http://www.sonos.com
http://www.pooraudiophile.com/
http://www.cheapaudiophile.com/