Vibe AI Explained: Why Letting Candidates Use AI Is the Right Call

A recent developer survey put daily AI tool usage among engineers at 87 percent (GitHub Developer Survey). Most technical assessments still ban AI tools outright, or try to detect and block their use. That mismatch is the entire reason Vibe AI exists: it's not a workaround for a testing problem, it's a response to the fact that the skill most assessments measure, working alone with no AI assistance, isn't the skill the job actually requires anymore.

Why test someone's ability to avoid a tool they'll use on day one?

If your engineering team uses Copilot, Cursor, or Claude every day, and almost every engineering team does, then a hiring assessment that bans AI tools is measuring a scenario that doesn't exist in the job. It's not a neutral, tool-agnostic test of raw skill, it's a test of a specific, increasingly rare working condition: solving a problem completely alone, with no assistance, under time pressure. A candidate who's excellent at that and mediocre at working with AI tools may perform worse on the actual job than a candidate who scores lower on the blank-editor version of the test but thinks clearly about how to direct an AI assistant toward a correct solution.

How Vibe AI actually works

Vibe AI is a built-in AI assistant inside the assessment environment, not a policy of allowing candidates to open ChatGPT in another tab. The assistant sits in a panel next to the code editor, available for the full session. Every prompt the candidate sends and every response the assistant returns is logged in full and attached to their submission. This is a deliberate design choice: instead of trying to detect or prevent AI use, which is difficult to enforce and produces an arms race with candidates who use external tools anyway, Vibe AI makes AI use fully visible and treats it as part of the signal, not a violation to police.

How is a Vibe AI assessment scored?

Every submission gets a composite score built from three separate dimensions, shown individually rather than collapsed into one number. Code correctness measures the pass rate against the test cases defined for the problem, the same mechanic a standard coding test uses. Code quality evaluates the structure, readability, and efficiency of the submitted solution. Reasoning quality, the dimension unique to Vibe AI, analyzes the candidate's prompt log: how clearly they communicated the problem, how they iterated when an initial response was wrong or incomplete, and whether their final understanding of the solution matches what they submitted. A candidate who copies an AI's output without understanding it tends to produce a submission with generic structure and reasoning that doesn't hold up under a follow-up question, and that shows up in the reasoning score even when the correctness score looks fine.

What this looks like in practice

Two candidates might both submit working code that passes every test case. One used the built-in assistant three times, asking increasingly specific questions as they narrowed in on an edge case they'd missed. The other used it once, pasted the entire problem statement, and submitted the first response without modification. Both score similarly on correctness. Their reasoning scores will typically diverge significantly, and that divergence is exactly the signal a blank-editor test, or a test that simply bans AI and hopes nobody uses it anyway, cannot produce at all.

What if a candidate has never used AI tools before?

Vibe AI doesn't penalize a candidate for using the assistant less, or for using it clumsily on a first attempt. The reasoning score isn't "did you use AI a lot," it's "does your prompt log show clear thinking and iteration." A candidate who's new to working with an AI assistant but asks precise, well-scoped questions and correctly evaluates the response can score well on reasoning even with a short prompt log. The signal isn't AI fluency for its own sake, it's whether the candidate can direct a tool toward a correct solution and recognize when the tool's output is wrong, which is closer to how AI-assisted engineering actually works day to day than a raw count of how many prompts someone sent.

What this means for hiring decisions

Vibe AI doesn't replace human judgment on a candidate's final interview or the decision to extend an offer. What it does is give you a signal that a locked-down, no-AI test structurally cannot: not just whether a candidate can produce working code, but how they think, iterate, and use the tools they'll actually have access to on the job. For a role where the day-to-day work already involves AI-assisted coding, that signal is closer to the real job than the alternative.

The shift this requires from a hiring team is mostly a shift in what "good" looks like on a scorecard. A perfect correctness score with a thin, one-line prompt log is worth less than a solid correctness score backed by a reasoning trail that shows a candidate catching their own mistake mid-problem. Once a team gets used to reading assessments that way, going back to a single pass/fail number feels like losing information you'd already come to rely on.