There's a risk if the chestnut hasn't been properly cured (ie: spent a couple of years submerged in a stream with moderate current). Unless you can accurately ID the species of chestnut, be aware that the horse chestnut is highly toxic in all parts of the tree.