"Build a friendship" sounds normal to me, sevengem. I have heard "build" used this way with the noun "relationship" fairly often before.
"Build up a friendship" isn't anything that I've ever heard or seen anybody use. However, it is easy to understand, and I might not even notice a speaker's use of "up" in that phrase in a conversation.
I can't see any important difference in meaning. "Build up" expresses the idea of 'making something larger' a little more than "build" does. But that's a very slight difference.