I honestly think it would be hard to tear off a background that is siliconed in place on all sides. In fact, you would have trouble tearing one out yourself, if it is installed properly. The front of these backgrounds are about a 1/4" thick befor it hits the foam. That layer is made of some sort of mortar i think. Very hard and rock like.
I think it would be easy for them to rip smaller things down though like the cave you mentioned.
I'll be trying it anyway with my ray tank, i'm very confident they wouldn't be able to budge it. There is no give pushing or pulling.
Prior to me owning one and getting it in the tank. I honestly would have thought the same thing as yourself.