Reading the Other Tasks' Files on a Shared Eval Runner

The shared runner as a tenancy problem

A shared evaluation runner is a tenancy problem wearing a scheduling costume. Many jobs execute on one machine, each in its own workspace, and "its own" is the operative question. If jobs run as separate containers, isolation is a wall: separate filesystems, separate namespaces, nothing of the neighbour's visible. If they run as separate directories under one user — the common CI-agent layout — isolation is a convention: the runner places each job in its folder and trusts it to stay there. Conventions do not enforce anything; the only thing between you and the neighbour's directory is a path prefix and politeness.

What stays visible under the convention is instructive. Other jobs' workspaces, with their checkouts, their '.env' files, their results; shared caches and build directories; the agent's own state — tokens, queues, logs. And beyond visibility, the shared surfaces: a writable '/tmp', a shared volume mounted into every job, a cache keyed by project name that the previous tenant populated.

The architectural fix is real separation — per-job containers or VMs, ephemeral workspaces wiped between runs, no shared writable state — and the diagnostic question is simple: can a job, using only its own privileges, name a file that belongs to another job? If the answer is yes, tenancy is a convention, whatever the marketing says.