No, they do not mean the same thing. However, they are similar.
Have you been to" is used with locations that one would visit. Cities, countries, restaurants, etc. You can replace "Have you been to" with "Have you (ever) visited".
"Have you gone to" is used when there is a task that needs to be accomplished at a location. "Have you gone to the store yet?" "Have you gone to college yet?" There is a purpose in mind, other than a visit.
I imagine the phrase you need would be: Have you been to the yard before? The word "before" would trigger that it is a visit.