Had to look up where Charlestown is. I have relatives in New Albany, so I have been in your area in the past.
I would contact your Purdue University turfgrass experts through your local county extension agent to get advice tied to your conditions and locale. Depending on what your current turfgrass species are, you "might" be able to use a selective post emergent herbicide that will take out the Johnsongrass without harming the other species. Certainty Turf Herbicide is such one product, BUT you have to know what you are doing and follow directions exactly.
Another post emergent herbicide is named Celsius - for warm season turfgrass species only.
EDIT: Outrider mentioned below is not labeled (not legal) for residential lawn use, unless your lawn is a "pasture", LOL. EDIT No. 2 : Outrider is the same active ingredient as Certainty....one labeled for farmers, the other for turfgrass!
Great minds think alike MParr!
Beautiful country in southern Indiana!