In theory, these could have been three different mountains in different parts of Norway, with slightly different spellings. But I searched my map app, and found only one mountain, Nystugguhøa in the Dovre mountains. There were no matches for the two other spellings. So you can safely assume that these are different spellings of the same mountain name, with Nystugguhøa as the official spelling.
Stug(g)u means cabin/cottage in the context of place names. This is dialect, as Winenous explains. This word is spelt "stue" in Bokmål and "stove" in Nynorsk, but there is no official spelling standard for dialects. I usually see the spelling "stugu" in surnames and farm names, but the word is pronounced with a short u (the first u) and I suppose that is why they used the double g.
"Høa" is the definite form of the noun, and "hø" the indefinite. Many mountain names use the definite form (e.g. Hallingskarvet), but some use the indefinite (e.g. Ustetind). I think "høin" is the dialect version of the plural definite form. According to the map, this mountain has two tops, so it can make sense to use a plural version of the name if you regard it as two mountains instead of one.
