Sounds like normal behaviour to me.
RPM drop simply means you are putting a load on the alternator. If there is a problem, I would expect the battery or wiring to be the issue; the alternator is doing exactly what is should be doing if either of those are weak or poor connections, etc exist.
He doesn't mention the make/model/year (*always* include this information if you want help with a problem -- NO EXCEPTIONS) so we are forced to guess here, he also doesn't say if this idle behaviour is upon startup or after a drive. This too is important information if he wants a diagnosis.
But, going by the (very little) information we have, some alternators do not charge until they reach a trigger RPM which activates the voltage regulator, so 100% of your idle power will be battery power. This could easily be normal behaviour in that condition. Toyota is one example, you need to blip the throttle to about 1850 engine RPM before the alternator will operate.
Other alternators do activate the voltage regulator, but at low RPM the alternator will have limited ability to provide current. Just like a gas-powered generator, the alternator needs both certain design criteria plus RPM to generate current and maintain voltages.
Engine Management Computers on modern vehicles are load-sensing systems; in most cases they will increase fuel delivery (or reduce it) to maintain a steady idle RPM regardless of load. So it's possible you have a sensor problem (not reading engine RPM correctly). Most likely an issue with very new (infant failure) or very old (plain worn out) vehicles.
Remember the alternator is a direct-drive device; it operates from idle to redline, it's power delivery is *directly* related to RPM (the voltage regulator often "wastes" power at higher RPMs), and at redline it is turning at extremely high RPM. Most alternators have a reduction gear so that engine redline RPM (say, 6500) is higher than alternator maximum RPM (perhaps 4200). That means that at idle (let's say 750 engine RPM) the alternator is operating at maybe 500 RPM. In other words a low generating condition, so that most power must be provided by essentially the battery alone.
Window motors are high current devices; the demand is a very high load on any electrical system, alternator working as designed or not. The reason this is not considered a problem is that window switch actuation is a short term temporary condition, unlike, say, the defrost blower. So automotive engineers do not build in the charging system power to account for that load for any length of time (it's treated like your starter motor ... which will explode or burn wiring if you try to use it for a number of minutes) and because of that actuating them is a very good load test, and I would not be surprised to learn many vehicles struggle with such a load at engine idle alternator RPMs.
Certainly others have made good suggestions, but in essence they are all things that should be in good condition in any case, so if they need attention, it should be done immediately. But I doubt that once corrective work on the electrical system has been performed, there would not be a similar situation under the conditions the OP has described. In other words, it may be perfectly normal and as expected.