Safe Autonomous Code Fixing interview questions

25 questions from the Safe Autonomous Code Fixing course, each with the correct answer and an explanation. Try to answer before you open the answer.

Why Autonomous Fixing Needs Safety

1. Which task is the best fit for an autonomous fix bot?

  1. Making the app "feel faster" without a metric
  2. Redesigning the system architecture
  3. A failing unit test with a clear reproduction
  4. Changing production data by hand
Show answer

Answer: A failing unit test with a clear reproduction

Objective checks make automation safe.

Lesson: Where Automated Fixes Help

2. Why might an automated fixer delete a failing test?

  1. Deleting the test also makes the check pass, unless the pipeline forbids it
  2. Tests are always wrong
  3. Models cannot read tests
  4. Deleting tests improves coverage
Show answer

Answer: Deleting the test also makes the check pass, unless the pipeline forbids it

Guard against gaming the check.

Lesson: What Can Go Wrong

3. What should come first in a safe fix pipeline?

  1. Rewriting the module
  2. Merging a patch
  3. Deleting old tests
  4. Reproducing the bug with a failing test
Show answer

Answer: Reproducing the bug with a failing test

Reproduce before fixing.

Lesson: The Safe Fix Pipeline

Reproduce and Localise

4. Why must the new test fail before the fix?

  1. Tests should always fail
  2. It proves the test actually captures the bug
  3. It makes CI faster
  4. Passing tests cannot be committed
Show answer

Answer: It proves the test actually captures the bug

A test that never failed proves nothing.

Lesson: Reproducing With a Failing Test

5. Which frame of a Python traceback shows where the exception was raised?

  1. The first frame listed
  2. The last frame listed
  3. Any random frame
  4. The import statements
Show answer

Answer: The last frame listed

Read traces from the bottom.

Lesson: Reading Stack Traces and Logs

6. What does git bisect run need from the test script?

  1. A new branch per commit
  2. A screenshot
  3. A pull request
  4. Exit code 0 for good commits and non-zero for bad ones
Show answer

Answer: Exit code 0 for good commits and non-zero for bad ones

Exit codes drive the binary search.

Lesson: Finding the Breaking Commit With git bisect

Generating Minimal Patches

7. Why not give the fixer the whole repository by default?

  1. It wastes context and invites unrelated changes
  2. Repositories are always secret
  3. Models cannot read code
  4. It makes tests fail
Show answer

Answer: It wastes context and invites unrelated changes

Focused context, focused patch.

Lesson: Giving the Fixer the Right Context

8. Why reject large diffs from an automated fixer?

  1. Git cannot store them
  2. Large diffs never compile
  3. They are harder to review and more likely to include unrelated or risky changes
  4. They always fail tests
Show answer

Answer: They are harder to review and more likely to include unrelated or risky changes

Small changes are reviewable changes.

Lesson: Minimal Diffs and Size Limits

9. What should happen to a candidate patch that passes tests but edits a test file?

  1. It should be preferred
  2. It should be merged first
  3. It should be rejected by the pipeline
  4. It should skip review
Show answer

Answer: It should be rejected by the pipeline

Passing tests is not enough on its own.

Lesson: Multiple Candidates and Selection

Verifying Patches

10. Why verify each candidate in a fresh copy?

  1. So attempts cannot interfere with each other or with the developer's workspace
  2. It makes tests pass automatically
  3. Git requires it
  4. It removes the need for tests
Show answer

Answer: So attempts cannot interfere with each other or with the developer's workspace

Isolation keeps verdicts trustworthy.

Lesson: Isolated Verification of Each Candidate

11. A test fails 5 times out of 20 with no code change. What should an auto-fixer do?

  1. Merge a random patch
  2. Rewrite the function under test
  3. Delete the test silently
  4. Treat it as flaky and not change product code for it
Show answer

Answer: Treat it as flaky and not change product code for it

Fix the flakiness, not the code.

Lesson: Detecting Flaky Tests

12. What does a surviving mutant tell you?

  1. The code is perfect
  2. The tests did not detect a change in behaviour, so they are too weak there
  3. The mutant is the correct fix
  4. Tests should be deleted
Show answer

Answer: The tests did not detect a change in behaviour, so they are too weak there

Strengthen tests where mutants survive.

Lesson: Are the Tests Strong Enough? Mutation Checks

13. What should happen if a patch disables a lint rule to pass?

  1. Accept it as a fix
  2. Treat it as a protected change and reject it from automated fixing
  3. Prefer it over other candidates
  4. Ignore lint entirely
Show answer

Answer: Treat it as a protected change and reject it from automated fixing

Do not weaken the checks.

Lesson: Static Checks, Types and Builds

Guardrails on Patches

14. Which change should a tampering check flag?

  1. Renaming a local variable in source
  2. Changing > to >= in source code
  3. Adding a docstring to a function
  4. Adding @unittest.skip to the failing test
Show answer

Answer: Adding @unittest.skip to the failing test

Skipping a test hides the failure.

Lesson: Detecting Test Tampering

15. Why protect CI workflow files from automated fixes?

  1. Editing CI can disable the very checks that verify the fix
  2. They are too small to edit
  3. Git cannot track them
  4. They contain only comments
Show answer

Answer: Editing CI can disable the very checks that verify the fix

The fixer must not change its own judge.

Lesson: Protected Paths and File Allowlists

16. A real API key was committed and then removed in a later commit. What must also happen?

  1. Rename the variable
  2. Nothing, removal is enough
  3. Rotate the key, because it remains in history and may have been copied
  4. Make the repository bigger
Show answer

Answer: Rotate the key, because it remains in history and may have been copied

Exposed secrets must be rotated.

Lesson: Secret Scanning on Every Patch

Sandboxes and Permissions

17. Why disable network access in a fix sandbox?

  1. It makes git faster
  2. Tests never need files
  3. To stop exfiltration and unexpected downloads by untrusted code
  4. Containers require it
Show answer

Answer: To stop exfiltration and unexpected downloads by untrusted code

Contain what you cannot fully trust.

Lesson: Sandboxed Execution

18. Which permission should an auto-fix bot NOT have?

  1. Merging its own pull requests into main
  2. Creating a branch
  3. Opening a pull request
  4. Reading the target repository
Show answer

Answer: Merging its own pull requests into main

Humans and CI approve merges.

Lesson: Least-Privilege Tokens and Branch Protection

19. Which defence still works if injected text persuades the model?

  1. Hiding the logs
  2. A stronger instruction in the prompt alone
  3. Trusting the issue author
  4. Mechanical gates and minimal permissions
Show answer

Answer: Mechanical gates and minimal permissions

Enforcement does not depend on the model's judgement.

Lesson: Untrusted Instructions in Issues and Logs

Review, Merge and Revert

20. Which item should an automated PR always include?

  1. Only the word "fix"
  2. The root cause and evidence that the new test failed before and passes after
  3. The bot's API key
  4. A request to skip review
Show answer

Answer: The root cause and evidence that the new test failed before and passes after

Evidence speeds up trustworthy review.

Lesson: Pull Requests With Evidence

21. Which change is least suitable for autonomous fixing?

  1. Fixing a lint warning
  2. A change to authentication logic
  3. A patch-level dependency bump with passing tests
  4. Correcting a typo in a log message
Show answer

Answer: A change to authentication logic

High-risk areas need human ownership.

Lesson: Review and Merge Policy

22. Why prefer git revert over rewriting history on main?

  1. It skips CI
  2. It deletes the bug permanently from all clones
  3. It is the only command git has
  4. It undoes the change with a new commit and keeps shared history intact
Show answer

Answer: It undoes the change with a new commit and keeps shared history intact

Shared history should not be rewritten.

Lesson: Reverting Safely

Dependency Fixes, Metrics and a Checklist

23. Which dependency update needs the most human attention?

  1. A major-version upgrade
  2. A patch version with passing tests
  3. A minor version of a dev-only tool
  4. No update at all
Show answer

Answer: A major-version upgrade

Major versions can break APIs.

Lesson: Automated Dependency and Security Fixes

24. What does a high revert rate on merged bot fixes suggest?

  1. Reviews are too slow
  2. The bot is perfect
  3. Gates or tests are not catching bad fixes
  4. There are too few bugs
Show answer

Answer: Gates or tests are not catching bad fixes

Strengthen verification before widening scope.

Lesson: Measuring an Auto-Fix System

25. Which item belongs on a safe auto-fix checklist?

  1. Production credentials in the sandbox
  2. The bot can merge to main without review
  3. Test files are read-only for the bot
  4. Skip the full test suite to save time
Show answer

Answer: Test files are read-only for the bot

Constrain, verify, review.

Lesson: A Safe Auto-Fix Checklist

← Back to the course