First thing you should do is control your test. Make sure you're testing against the same endpoint. Names can resolve to different IPs, based on proximity and IPs are not necessarily unique.
Once you have a known endpoint and experience large variations in latency between your two services, then I'd do a traceroute to dig into why. Likely, there will be a single hop in the path of the "slow" location that will be responsible for the increased latency you're seeing. My money is on your provider's first peering point to your destination and not necessarily within its own network.