Yea...its been awhile...the neutral and ground are one when they enter the box..with a ground near box and one at the pole. If the ground between pole and box is poor..or should say neutral/ground....also the pole with trany is blocks away..
Our electrical grid setup is over 100 years old. Codes might be different elsewhere..
That neutral/ground wire from the pole, which is required to be grounded at the service disconnecting means, is considered a neutral between the house and the pole. Very little current actually flows in the ground between the pole and the house. If that neutral wire between house and pole has a bad or broken connection, it will cause wandering voltage on the 120V L1 and L2 conductors.
On the other hand, you could just completely disconnect the ground wire from the ground rod at the house (and the pole) and so long as that neutral wire is still intact, the voltage on the 120V L1 and L2 conductors will still be correct.