While I agree with Pachomius, I would add that "fairly unpleasant character" could very easily be meant as tongue-in-cheek, an understatement in some contexts -- so that actually the meaning could be "extremely unpleasant character."
Example: Imagine a preview of the movie that shows the man impaling his enemy on a stake. The narrator might then say, "he's a fairly unpleasant character."
I agree with coiffe. Fairly unpleasant would not mean that he was fair--neither would he be lightly unpleasant. Fairly unpleasant is really, truly unpleasant--even to the most fair-minded people that would give him all excuses as to why he was grouchy--even to someone more than fair-minded--he would seem unpleasant.