Are you sure that 2 is incorrect? I thought it was correct!(1) is correct. (2) is incorrect, but would be correct in reverse:
Visiting a bunch of children was all I did.
In my opinion, neither is correct, at least in US English. As an editor, I would reject the second and would correct the first by removing the word "to".Are both grammatically correct?
1) All I did was to visit a bunch of children.
2) All I did was visiting a bunch of children.