Why remove it not gonna make a different! Most engine have a quart of oil left in the lifter valley that you can't change!!!!
So, where do you draw the line? In my opinion, and I know this against what you guys are saying, if you are doing an oil change, isn't the purpose of the oil change to get as much of the dirty/used oil out as you can?
Sure, maybe the filter will be fine and it can handle another OCI or 2, but if you have a quart held up in the engine and another 1/4-1/2 liter left in your filter, that adds up to a lot of dirty oil left in the engine.
Personally, since I like knowing that I got as much used/dirty/spent oil out of my engines as possible, I have switched to TG's instead of Ultras.
Each to their own, but if I am crawling underneath my vehicles to access the drain plugs, it isn't really any extra work to remove and replace the filter while I am under there. My Ram isn't the easiest filter to access, but with a rag laid over the steering rack (for a just in case) and a cut down ice cream container to catch the oil, it takes me moments while my oil is still draining.