If the pan is easy to remove and replace, I'd drop it and change the filter and clean the magnet. If it's a pain to remove, I wouldn't bother. I have two vehicles where removing the pan is a big pain. For example, on my Escalade, the exhaust runs right under the transmission pan and you have to unbolt the Y-pipe from the exhaust manifolds so the pipe droops down enough to get the pan off. On my Sienna, some of the pan bolts are just above the subframe and you have to unbolt the transmission mount and jack the transmission up a few inches in order to get access to those bolts. I will not be removing the pan on those two vehicles again. But if all of the pan bolts are easy to access and there's nothing in the way of getting the pan off, then I'd drop the pan.