I grew up on a dairy farm. When we wanted milk to drink in the house, we would get a container and just dip it out of a 1,000 gallon tank from cows that we milked. Yes, with whole, non-homogenized milk, you have to shake it to get the cream to mix with the milk because the "cream rises to the top". (Ever wonder where they got that phrase?)
ANYWAY, all that cream is fat. Take a look at the nutrition label of your grass-fed milk and if you look closely, you will see how much fat is in just a cup (one serving) of your milk. THAT is why that milk tastes so good. It's not necessarily because the cows were fed grass (although different cow feeds can affect the flavor of the milk), it's because you have all that cream or fat in your milk. It's also why premium ice cream tastes so good, because of all the fat it has in it. While your tastes buds will like this milk, your arteries and heart won't.
I went to college in the great State of Nebraska, (Go Big Red!!!!!!). While there, I used to spend my Thanksgiving break, Christmas break and Easter break with a family who had a diary farm way out in the Sandhills. Whatever milk you drank came right out of the cow. I have no doubt it was high in fat, but we were working in sub-zero temps, (it gets extremely cold in the Sandhills), so those extra calories helped us stay warm in the bitter winter weather. The only thing that kinda bothered me was the milk barn wasn't the cleanest place and had a lot of cow patties on the floor. I'll never forget the first time they took me out there and were showing me the set up. While we were chatting about the system and operation, Paul, (my friends father) set a five gallon bucket on the floor amidst all the cow poo, took the lid off the tank, and proceeded to dip out some milk. When he did, little pieces of cow poo floated off into the fresh milk. He told me several times, "I've got all this stainless steel equipment and I still keep getting a grade "B" rating - I don't know why." I didn't say anything but I was thinking, "it could be that earthy additive you just put in the tank." They were good people, hard working, salt of the earth and always made me feel welcome and at home which was very nice for a Florida boy who couldn't afford to travel home for the holidays.
That is the thing about pastuerization, there is no real necessity to keep the the milk as clean as possible. My neighbor had a small dairy herd and he was pretty lax about cleanliness too.
We have a couple dairy goats that we milk and odd hair and some dirt/manure does go into the bucket. But we have a very fine mesh filter that gets it out and then it goes into the fridge about 5 minutes after it came out of the teat, so its rare the milk ever tastes funny. But we don't keep it around for weeks either, 4 or 5 days at most. And we do have to give it a shake after a day or two.
I'm with Stanely though, that if you are going to drink full fat milk, grass fed is better for you than grain fed. Just like with beef, pork, chicken, and eggs.