Getting rust off and keeping it off are two very different things.
To get rust off, apply almost any type of oil (but a CLP type oil works well) to the rusted area. Then rub lightly with 000 or 0000 steel or bronze wool until all the rust is gone. To finish wipe all of the residue off and re-treat the metal.
To keep the rust off, you need a good rust preventative. Most general purpose lubricants, gun oils and CLP type products do a poor job of rust prevention. My favorite products are Corrosion-X, Eezox and fluid film in that order. Corrosion-X is awesome because it is an amazing lubricant as well as an excellent rust preventative. Eezox is a dry lubricant type product and fluid film is an easy to find (Lowes, Autozone, & Napa), lanolin based rust preventative, non stick and light lubricant.