Semiclassical gravity is a description of gravitational physics in terms of local observables evolving on a classical spacetime manifold. This appears to be at odds with diffeomorphism invariance, which forbids the existence of truly local physical degrees of freedom. To resolve this, one needs to replace the standard notion of locality with a relational version, in which some of the fields are used as reference frames. In the quantum theory, the reference fields must fluctuate, and in this talk I will argue that understanding the consequences of these fluctuations could provide a general route towards going beyond the semiclassical regime in perturbative gravity. To illustrate this, I will explain how to extend the generalised second law (which states that the generalised entropy of a black hole horizon never decreases) beyond the semiclassical regime.