Make your agent earn the right to fix a flaky test: a shuffle-seed harness and a commit guard
Tell an agent a test is flaky and it will add a retry. This setup makes it measure the failure rate, replay a failing seed and prove the fix, and it blocks the commit if the 'fix' is a retry, a sleep or a skip. About 15 minutes to try.
13 min read