Sounds like fun!

I think The heavy-barreled Remington Model 700 a fine choice for pest control. The .204 Ruger uses a light, wind-susceptible bullet (32 grain) and launches it at high velocity, so it provides a flat trajectory. It'll ruin a varmint's day, no question.
The .22-250 or .223 will toss a heavier bullet, and there's something to be said for that as well. Consider where you usually do your varminting and consider the distance and prevailing winds.
I think the .204 Ruger cartridge would be fun to work with and handload. The Rem 700 has proven itself to me. Whatever your decision, varmints will not be safe in your vicinity!

i have a remington 700 sps tactical in 223 and i love it, ive also shot a savage 110 in 204. its also a great gun but i can out shoot it with my 700 in 223. to me the 204 it just a little light for what i like, but like Edward said it will ruin a varmints day.

The .204 is getting rave reviews from buddies who shoot them. The Rem 700 SPS is a great platform. I haven't gone to it personally as I like the versatility of the .22-250 or the .220 Swift. They can reach nearly the same speed with a cartrige that can handle up to deer sized game. However, if all you intend to shoot is prairie dogs inside of 250 yards, the .204 is pure fun. I still prefer something bigger there too because I like to shoot out to 400-600 yards.