I would translate אל as towards just for the sake of clarity, which necessarily implies movement and closing the distance.
So with verbs of movement of one physical object towards another it's (i guess) always אל
With pronominal suffixes אליי, אליה, אליכם etc. it's used both literally and figuratively, that is when reference to a person implies the person's place.
The difference between the connotation may transpire from context where אל in physical sense seems to be more pertinent to continuous actions while in figurative sense - to completed (perfective) actions, e.g.
מספר שניות הלכתי אליו - for a few seconds i was walking towards him
אחרי עבודה הלכתי אליו - after work i went to his place
Well that's the usage purview. It's very difficult to give account of all nuances and possible permutations.
In some context they could be more or less synonymous but not in others.
Example: פניתי אליו - I turned to him (mainly figuratively, for help, with a request, much rarer physically) vs פניתי כלפיו - I turned towards him (physically, turned my body to face him)
The more one immerses into the language the more they start to feel the shades of meaning and appropriate usages.