Generally, this means that something has been achieved, but only by a small margin.
"His arrow barely hit the target", for example, means that his arrow did hit it, but it almost did not (like hitting the very edge, for example).
"I barely even know him", in the same sense, means that you do know him - but you don't know all that much about him. Perhaps you know his name and nothing more, or perhaps the person turned out to be different than what you thought he was like.
"I barely know the answer" seems incorrect (this is very binary, either you know it or you do not) - but you could read this as "I know the correct answer, but I don't know why it's correct".