Menu / 05
← Signal feed

Search intelligence / Argument

The coding test is becoming a judgment test

HackerRank's AI interviewer now scores how engineers work with AI, while 62% of organisations still ban AI in technical interviews. What that means for assessing engineers.

Field note / Published

Written by
Matteo Muscio
Published
  1. 01 AI
  2. 02 Recruitment
  3. 03 Hiring
Article contents 4
  1. 01 The numbers behind the launch
  2. 02 Where most companies still stand
  3. 03 The code is the easy part now. Test the judgment.
  4. 04 The open question

On 5 October, HackerRank made Chakra, its AI interviewer, generally available to customers. The product is worth a look less for the automation and more for what it chooses to measure. Candidates get a task based on a real code repository and work alongside an AI assistant. Chakra then asks follow-up questions: why they picked one approach, how a new constraint would change the solution. It scores critical thinking, judgment and what HackerRank calls “AI fluency”: how well someone frames a problem for the AI, judges what comes back and steers it toward a solution. (TechCrunch, 5 Oct 2026)

HackerRank’s CEO, Vivek Ravisankar, summed up the shift: “The previous modality of evaluation was evaluating the output.”

The numbers behind the launch

500,000+ interviews. According to TechCrunch, Chakra ran more than 500,000 interviews in roughly six months of beta testing, with Snowflake, Snorkel and Capgemini among the companies that tried it. One session merges three steps that usually sit apart: the recruiter screen, the take-home assessment and the follow-up with an engineer. Humans still make the final hiring decision.

70–80% fewer suspicious-activity flags. HackerRank says flags were 70% to 80% lower in Chakra interviews than in comparable traditional assessments, varying by geography and seniority. This is a company-reported figure. The reasoning behind it is simple: if candidates can use AI openly, there is less reason to use it secretly.

Where most companies still stand

The launch arrives while many employers are still keeping AI out of the room. Karat surveyed 400 engineering leaders across the US, India and China and reported that (Karat, Jan 2026):

  • 71% say AI is making technical skills harder to assess.
  • 62% of organisations still prohibit AI use in technical interviews.
  • Allowing AI in live interviews differs sharply by market.

Chart: Share of engineering leaders whose live technical interviews allow AI use

Market Allow AI use
US 38%
China 68%

Source: Karat, “Engineering Interviews in 2026”, survey of 400 engineering leaders in the US, India and China.

Karat sells human-plus-AI interviews itself, so its conclusions deserve the same caution as HackerRank’s. Still, the direction is consistent across both companies: when AI can produce working code quickly, the code alone says less about the person who submitted it (Karat, Aug 2026).

The code is the easy part now. Test the judgment.

My view is that this is the right question for engineering interviews to be asking. What’s worth testing is how someone thinks: how they frame the problem, whether they question an answer that looks plausible, and how they change course when a constraint moves. Those are the things that separate engineers once everyone has the same assistant.

Tools like Chakra can watch that process at scale and apply consistent criteria. That has value, particularly early in a funnel. What a tool can’t decide for you is what good judgment looks like for your team: your codebase, your product’s trade-offs, the mistakes that would actually hurt you.

That’s the reason we built assessment the way we did on MVRK’s tech searches. Technical assessment is led by the CTO of an Italian scale-up, who built its tech stack and team from scratch. Rather than a LeetCode-style test, they look at how candidates reason through the kind of problems a growing engineering team actually faces. The aim is the same one HackerRank describes: assess the process, not just the output, with the final call made by people who have done the job.

The open question

84% of developers say they use or plan to use AI tools (Stack Overflow’s 2025 survey, as cited by Karat). Yet most organisations in Karat’s sample still ban AI in the interview. How each company closes that gap will say a lot about what it actually values in an engineer.

If AI is allowed in the job, should it be allowed in the interview?

Sources