I'd give it a while. In my experience RTV, unless it's being used as a mild adhesive to hold a paper gasket on, should be allowed a good 8-12 hours before the part it's used on is placed in service. Even when you're just using it to hold a gasket it's a decent idea to let it set up for a minimum...