This phrase owes it to Sufi wisdom. Here is the background below.
Based on Sufi wisdom, Hans Christian Andersen tells the tale in his "The Emperor's New Clothes", the story this expression derives from.
There existed an emperor who loved wearing fine clothes and spent all of his people's money on them. He had a different set for each hour and was, without doubt, the finest dressed man in the land.
One day, two swindlers claiming to be weavers entered the Emporer's city and proclaimed they were capable of making the finest, lightest, most magnificent cloth the world has ever seen. So extraordinary was this cloth, it was invisible to anyone who was incompetent or stupid.
Hearing of the weaver's amazing "talent", the foolish Emporer thought he could use such cloth to weed out undesirables in his city. He paid the swindlers an enormous sum & they set out to "create" the clothes; knowing they would only need go through the motions.
The Emperor sent several advisors to guage their progress and all the advisors reported the cloth magnificent, not wanting to appear unworthy for seeing nothing at all; the cloth didn't exist!
Finally the clothes were "finished", the swindlers already having counted the gold and jewels they had received. A procession was arranged to show off the Emporer's new clothes and the entire city gathered in the center to view them. Having been "dressed" by the swinglers, who remarked how wonderful he looked, and how light the cloth appeared on him, he appeared before his people.
The people, having heard of the weaver's abilities and the cloth's fictious properties, were amazed and offered thunderous applause to the now beaming Emperor. None of them were willing to admit that they hadn't seen a thing; for if anyone did, then he was either stupid or unfit for the job he held. Never before had the emperor's clothes been such a success.
While expressing admiration at their Emporer's new "invisible" clothes, a small boy cried out... "But the Emperor has no clothes!"