CI Feedback Training

Learn to iterate on CI and LLMaJ feedback to get your task passing all checks.

Practice Notebook

Download the Jupyter notebook for hands-on practice:

What You'll Learn

Understanding CI check output
Interpreting LLMaJ feedback
Iterating on failures efficiently
Common patterns and fixes

CI Iteration Workflow

Run checks → Identify failures → Fix issues → Re-run → Repeat

Step 1: Run All Checks

harbor tasks check <task-folder> -m openai/@openai/gpt-5.2

Step 2: Review Output

CI Checks:
  ✓ pinned_dependencies
  ✗ check_task_absolute_path
  ✓ validate_task_fields

LLMaJ Checks:
  ✓ behavior_in_tests
  ✗ informative_test_docstrings
  ✓ anti_cheating_measures

2 checks failed

Step 3: Fix One Issue at a Time

Focus on one failure at a time. The error message will tell you exactly what's wrong:

✗ check_task_absolute_path
  Line 3: "Edit config/settings.json"
  Use absolute path: /app/config/settings.json

For detailed fix instructions: See CI Checks Reference and LLMaJ Checks Reference

Step 4: Re-run and Verify

harbor tasks check <task-folder> -m openai/@openai/gpt-5.2

Step 5: Repeat

Continue fixing issues one at a time until all checks pass.

Understanding Error Messages

Error messages are designed to be helpful. They typically include: - Which check failed (e.g., check_task_absolute_path) - Where the issue is (file and line number) - What's wrong (specific problem description) - How to fix it (suggestion or requirement)

Read the error message carefully - it tells you exactly what needs to be fixed.

Tips for Efficient Iteration

Fix easy issues first - Start with CI checks (syntax/structure) before LLMaJ (quality)
Run locally before submitting - Saves time and helps you catch issues early
Read error messages carefully - They tell you exactly what's wrong and often how to fix it
One fix at a time - Easier to track what worked and debug if something breaks
Use the reference guides - CI Checks Reference and LLMaJ Checks Reference have detailed explanations for each check

Checklist Before Final Submission

All CI checks pass
All LLMaJ checks pass
Oracle agent passes
Real agents tested (difficulty verified)
Code reviewed for quality

Next Steps

Submit via Platform