Actually there is no connection. The original skunkworks factory was named and headed by a man name Kelly Johnson who worked for Lockheed Aircraft. Using his special team of engineers he developed the first American jet powered airplane during the second world war (WWII), and others like the U2 Spy plane and the famous SR71 supersonic spy plane. The name was stolen (borrowed) from a U.S. comic strip called "Lil Abner". Lil Abner was a cartoon character who lived in the hills and made moonshine whiskey in a secret whiskey still he called the skonk works, a colloquial pronunciation for SKUNK, (mofeta/sorrillo). Presumably the still that produced the moonshine stunk like a skunk, ergo the name skonk works.
Since Kelly Johnson's projects were all top secret, naming it after Lil Abner's still seemed appropriate.
The skunkworks name soon became a popular way to name a group of workers and engineers pursuing a task with special talents and needs
Por supuesto que equipo especial es la traducción formal. Pero la manera coloquial en español tiene que ser inventada a partir de alguna anécdota. Trabajé en una empresa de ingeniería donde había un grupito al que le asignaban trabajos de élite y de algún modo secretos. Les decíamos Los Chirimías porque algunos de ellos tocaban instrumentos musicales y cantaban.