tl;dr: Executive summary: It is not fundamentally true that the Earth goes around the Sun; it is just easier to calculate things that way.
We like to say that the critical event that started the scientific revolution is the discovery that the Earth goes around the Sun, and not the other way around. This was incredibly important; the hypothesis by Copernicus led to the immensely important data-taking by Tycho Brahe and the quantitative, theoretical explanation of it by Kepler. Galileo's discovery of moons of Jupiter bolstered the case in important ways, and Newton's quantitative description of it all in terms of the inverse-square law solidified it all into an edifice of great importance, that is just as important and valuable today as it was then. It is also a great example of how a scientific discovery requires both observational and theoretical backing to become confidently adopted by the community.
In the 20th Century, Einstein brought us General Relativity, with the eponymous generality granting us immense coordinate freedom. That is, there are (infinitely) many ways we can make decisions about what is stationary and what is moving, and what we choose as reference points. In some choices, calculations are harder. In other choices, calculations are easier. In yet others, certain symmetries become more obvious or more valuable for making predictions. That is, GR delivers to us lots of choices about how to think about what's moving and how.
So the crazy insane thing is this: In GR, there is no answer to the question of whether the Earth goes around the Sun or whether the Sun goes around the Earth. There is literally no observational answer to the question, and no theoretical answer. All observations can be incorporated to an analysis from either perspective. The question of which goes around which is not a question you can ask in the theory.
That said, it really is far, far easier to do calculations in the Copernican frame. Indeed, absolutely all calculations of Solar System dynamics are done in this frame with post-Newtonian code. The way I see it (with modern eyes) is that Copernicus's hypothesis was based on parsimony or simplicity and was adopted for that reason. Brahe and Kepler confirmed that the data are consistent with Copernicus's simple model (though with the eccentricities added). After Brahe and Kepler it was still possible to understand the observations in an Earth-centered (or even stranger) coordinate system, but was far, far easier to do calculations in the heliocentric frame.
Even today, now that GR is our model of gravity, we still calculate the Solar System with Newtonian codes (with adjustments to approximate GR corrections). And even today, now that we have this amazingly accurate model of the Solar System, we still often calculate the positions of celestial bodies by looking at paths on the celestial sphere, as did Ptolemy. How we calculate something is incredibly context-dependent, and doesn't always respect our most fundamental ideas. And the truth of Copernicus's hypothesis really just represents the pragmatism of the present-day mathematical tools. All these thoughts bolster my rejection of scientific realism and play into questions of social construction and so on. It also bolsters my view that Ockham's Razor should be thought of as a statement about calculation, not truth.
Sure the Earth goes around the Sun! But let's remember that this is a statement about calculation and pragmatism, not the fact of the matter.
