A self-averaging physical property of a disordered system is one that can be described by averaging over a sufficiently large sample. The concept was introduced by Ilya Mikhailovich Lifshitz.
Frequently in physics one comes across situations where quenched randomness plays an important role. Any physical property X of such a system, would require an averaging over all disorder realisations. The system can be completely described by the average [X] where [...] denotes averaging over realisations (“averaging over samples”) provided the relative variance RX = VX / [X]2 → 0 as N→∞, where VX = [X2] − [X]2 and N denotes the size of the realisation. In such a scenario a single large system is sufficient to represent the whole ensemble. Such quantities are called self-averaging. Away from criticality, when the larger lattice is built from smaller blocks, then due to the additivity property of an extensive quantity, the central limit theorem guarantees that RX ~ N−1 thereby ensuring self-averaging. On the other hand, at the critical point, the question whether is self-averaging or not becomes nontrivial, due to long range correlations.
At the pure critical point randomness is classified as relevant if, by the standard definition of relevance, it leads to a change in the critical behaviour (i.e., the critical exponents) of the pure system. It has been shown by recent renormalization group and numerical studies that self-averaging property is lost if randomness or disorder is relevant. Most importantly as N → ∞, RX at the critical point approaches a constant. Such systems are called non self-averaging. Thus unlike the self-averaging scenario, numerical simulations cannot lead to an improved picture in larger lattices (large N), even if the critical point is exactly known. In summary, various types of self-averaging can be indexed with the help of the asymptotic size dependence of a quantity like RX. If RX falls off to zero with size, it is self-averaging whereas if RX approaches a constant as N → ∞, the system is non-self-averaging.
There is a further classification of self-averaging systems as strong and weak. If the exhibited behavior is RX ~ N−1 as suggested by the central limit theorem, mentioned earlier, the system is said to be strongly self-averaging. Some systems shows a slower power law decay RX ~ N−z with 0 < z < 1. Such systems are classified weakly self-averaging. The known critical exponents of the system determine the exponent z.
It must also be added that relevant randomness does not necessarily imply non self-averaging, especially in a mean-field scenario.  The RG arguments mentioned above need to be extended to situations with sharp limit of Tc distribution and long range interactions.