That's usually the only way I travel! Then again, I have severe agoraphobia and social anxiety....and just about everyone I've ever been in a car with for more than 30 mins considers me an insufferable a-hole (as a passenger OR driver). When I have time off from work and money to burn, it's not all that unusual for me to call home at 10pm and explain that I am 400+ miles away from and won't be back for a day or so.
It's nice having a break from compromises and other people's problems, faces, and voices - nobody that has to pee even though we just stopped, nobody who's tired and wants to go back to the hotel ( or is "tired" and wants to go back to the hotel), nobody instagraming, posting, liking, retweeting, or looking at dogs that we're never going to adopt - we talked about it a hundred times! We're not moving anytime soon and we're both gone 10+ hours a day! A dog won't work, stop showing me their cute faces.
As far as getting stares and such - for the most part it feels a lot more akward than it really is. I've been to bars and restaurants alone where I've paid anywhere from $5-$100+. The only people who make it you feel out of place are the waitstaff, but usually just because your alone and less likely to leave a large tip ( and if they do a poor job, then DON'T tip - you're on vacation, you're never gonna see those people again. Part of why I travel is to be a completely unrestrained, unabashed, class-a [censored]. Then again, I'm not a "good person")
All I can say is you do you and enjoy every minute of it. Especially if it's paid vacation time - you're getting paid to not be at work.