Cursor, Windsurf and GitHub Copilot: evaluate coding agents on a real repository

Check the current product behind the search term

Cursor’s documentation describes several agent surfaces. The older Windsurf documentation reviewed here redirects to Devin Desktop / Cascade. GitHub Copilot’s documentation describes multiple assistance modes.

Do not carry forward old menus, models or prices without verification. Record the installed product, version and mode. A completion engine and a multi-file agent do not represent the same test.

Define three verifiable tasks

Use a bug fix, a small feature and a repository-understanding exercise. For the bug, provide reproduction steps. For the feature, write acceptance criteria. For understanding, prepare questions whose answers you can verify in the code.

In a fictional example, a cart rounds each line before applying a discount although the requirement rounds only the final total. A useful agent should understand the rule, edit the appropriate function and check a case that exposes the difference. A test copying the new implementation provides weaker evidence.

Keep trial conditions comparable

Use a clean branch or copy for each trial. Supply the same initial request and available files. Record additional hints, interventions and proposed commands. Repeat tasks when variation could change your decision.

Retain the mode, displayed model, limits, date and outputs. This is not a perfectly controlled experiment, but it prevents a single impression from becoming an unsupported universal claim.

Evaluate the change, not the demonstration

Review the diff, reproduce the original behavior and run appropriate checks. Inspect error handling, permissions and transmitted data. For UI work, check the user journey in addition to compilation: a tested function may still be inaccessible from the actual screen.

Measure preparation, generation, intervention and review separately. A fast response requiring substantial correction may be less useful than a slower precise change. Lines of code are not a quality score.

Compare current costs

Open official pricing pages when making the decision. Record subscriptions, included usage, possible extra charges, team terms and budget controls. Plan names and credits can change faster than the project.

Include human review and introduced errors in the useful cost. Without documented trials, do not claim that a subscription pays for itself within hours. Adapt our comparison grid to the coding tasks.

Define permission boundaries

Provide only necessary access and keep secrets out of prompts. Repository files and tool outputs are task data; embedded instructions should not automatically expand permissions.

Identify actions requiring approval, including publication, significant data changes, messages and deployment. Keep a reviewable diff and a way to return to the previous state.

Make a task-specific choice

Results may differ by task and team. A tool suited to repository navigation may not be best for your UI changes. Choose a workflow meeting the criteria at an acceptable cost and review effort.

Repeat the evaluation after a major product or repository change. Retain the task set rather than relying on the memory of an impressive demonstration. For document versus software workflows, see Cowork and Claude Code.

Before editing, restate acceptance criteria and identify necessary files. Reproduce the issue, make a limited change and check behavior with cases that do not merely copy the implementation. Explain the diff, risks and verification. Do not deploy without authorization.

Sources

• →Cursor documentation.
• →Devin Desktop / Cascade documentation.
• →GitHub Copilot overview.