I have seen an awesome cave system done by siliconing layers of that pink foam you can get from home depot together and carving caves and what not out of it...then using a concrete mixture layering it over the foam and it looked sick!
I made 2 prettty big caves using just pieces of slate, which me and my roomate got for free at a slate/rock/tile place. They typically deal in tons and much larger amounts for construction and stuff and didnt get a crap if we wanted a meager 5-10 lbs of random pieces.
i made my rock caves insane by cutting off their eyelids and forcing to stay awake for weeks at a time and playing loud music.. constant beatings also helped to push them over the edge... a strobe light was also helpful...