I have personally seen inspectors demand all work completed without a permit be torn out.
Sources of being tipped off included:
--Neighbors with long standing feuds.
--Other contractors angry they did not get the bid. Its not uncommon for a contractor who made a bid to snoop around in the following days or weeks.
--A zealous building inspector driving through a neighborhood and seeing a contractor making frequent trips in/out to a workvan suggesting work is being performed.
--A dumpster in the driveway is a big tip off.
In some locales, a permit can only be issued if a licensed contractor will be performing the work, even with trades such as roofing. Homeowners are not allowed to perform their own repairs and upgrades.
Ofcourse, the scope of the project plays a major part in your question.
With matters of electrical or with plumbing gas lines, I can appreciate the strict requirements since improper installation can result in high risk to anyone in the residence...or an entire building if not a single family home.
Have I always pulled permits? No...but I get along with my neighbors.