You should say:
I'm a fan of MJ's.
Most people would say:
I'm a fan of MJ.
However, in your original examples "I'm a fan of you" sounds weird. I think with pronouns the rule has to be followed strictly, but with names it's more flexible.
I'm an admirer of yours
He's a friend of mine.
I'm a fan of Michael Jackson('s)
To me, this last sentence sounds better without the saxon genitive.