Submission Diversity Requirements

🆕 New submission diversity requirements

We're introducing new diversity requirements for Project Terminus moving forward. We will restrict the kinds of submissions we accept in order to guide the dataset toward ideal variance in submission data.

To support this, we are introducing new automated evaluations. These checks are designed to block submission of tasks with certain properties. Here is what to know about what we want and will allow:

Restrictions and preferences

• Codebase size: Only small and large will be accepted - minimal will be blocked.
• Milestones: Non-milestone tasks won't be blocked, but milestone tasks are preferred (and pay more! 🎉).
• Model difficulty: Only medium and hard will be accepted - easy will be blocked.

Note: “Easy,” “medium,” and “hard” here refer to model performance (pass rates on frontier agents), not the difficulty field you set in task.toml.

No change in these areas

• Category: No restrictions.
• Subcategory: No restrictions.
• Languages: All languages are accepted; Python tasks must be hard model difficulty to be accepted.

💡 Tasks already in flight

Already have a task in progress? These diversity checks do not apply to submissions that are waiting for review or in your revision queue. They apply only to brand new submissions moving forward.

As always, let us know if you have any questions - we're happy to help! 😊