Phonemic analysis comes down to the most parsimonious description of the meaningful distinctions made by a language.
<hue/hew/Hugh> and <you> are distinct in most if not all varieties of English. Influenced perhaps by the spelling, I'm sure most native speakers would perceive this as /hju/ v /ju/.
It seems no more useful to posit a new extra phoneme here than it would be to suggest that the two Ls in little should be counted as different phonemes: no one doubts that in most varieties of English, the realisations of /l/ are quite different, but as they pattern neatly within the overall phonology as one phoneme, this is the most obvious and, I think, universally adopted analysis.
The case is slightly different with /h/ v /ŋ/: while they are distributed distinctly, and one could use the same sign for each without causing confusion, most people would feel that the realisations are too far apart for them to be assigned as one phoneme, and there are no structural reasons in the language that suggest any relationship.
In the end there can be no perfect phonological analysis, it is simply the structure that seems to make most sense.