You have to run the numbers. Get an Excel sheet, or at the very least a piece of paper. Calculate what you will spend if you just keep paying as you are right now. Then find out what the closing cost on a refi will cost, then add in what you will now spend. That will tell you more than anything we can tell you.
Will say that this is predicated upon you staying there for the duration. If you move in a year or two, then it is unlikely that you would be ahead by refinancing.
FWIW, I bought mine in 2005 also. 6.125%. Refi'd back in 2010/11 into 3% & 15 years, for the same payment I shaved years off. I watch the rates going up now and think, man am I glad I did that.
Oh, and if you don't have PMI right now, you might pick that up. Bank might decide the house is worth less than you think. When I did a refi I wound up getting PMI--but my total out of pocket was still less, over the long haul.