The key to keeping turtles in is tricking them into thinking they're not really penned in. In my experience if a turtle feels confined or comes to an obstacle they are either going to A) go around it B) dig under it or C) climb over it.
I've found that wire fencing that has large gaps in it works best. Here's a close up of the fencing I use. It's made up of 2"x3" rectangles. I think the dark color green also helps it blend in with the surroundings as well. I've never had any of my box turtles try to dig or climb out of my pen.