At evaluation time, the paper corrupts each head’s quiet reads before any gate acts and measures the increase in validation loss relative to that same model without corruption. · CiteArk
At evaluation time, the paper corrupts each head’s quiet reads before any gate acts and measures the increase in validation loss relative to that same model without corruption.
来源：fixed-paper:PDF p. 13, Appendix C, Procedure and Junk types
