The difference reflects the history of each country. In England many years ago, when education was very limited, they opened schools to anyone who could afford the tuition, so that even commoners could send their children to school if they had the money. Therefore, the concept was that such schools were "public." In the US, which is a much younger country, education had already become more widespread by the time of independence, so schools were funded by the government, and those schools were called "public" because they admitted all children for no charge. Private schools are not open to everyone, and can admit only the students they choose to admit.
A "private school" in North America is called a "public school" in England, while a North American public school is called "state school" in England, as far as I know.