Number in this sense just means something like "item". It is used especially for an article of clothing; although number has other similar meanings (where it refers to a thing or an action, or even a person), they are not necessarily related in sense development to this usage, I don't think.
I also don't think it originally meant anything other than "item" (in the fashion industry); it's just that it is almost always used in conversation to describe some garment which is attractive, "sexy", etc. It also often goes with the intensifier "little" (see Renaissance Man's link).
I think your sentence is fine, but it's pretty much the same as the dictionary examples, but with the colour changed.