Guides
How to evaluate projects in the era of AI and vibecoding
How to set up criteria when one person builds over a weekend what a team used to make, and how to distinguish a product from a wrapper.
Old evaluation criteria break down against AI. When a team builds in one evening with Cursor what used to take a whole weekend, evaluating code volume becomes pointless, and traditional scales start rewarding the wrong people. Let's break down how to judge a hackathon where everyone is vibecoding.
What stopped working
Evaluating by the amount of work done no longer reflects effort or skill. One person with AI can produce the output of an entire team, and a beautiful wrapper over a chat model looks no worse on a demo than a well-thought-out product. If you keep the old scale, you will reward those who generated a plausible picture the fastest, rather than those who actually built something.
Where to shift the weight
- Functionality. Not 'looks ready', but works on a live example chosen by the jury, not the participant.
- Depth behind the wrapper. Is there a custom task, custom data, and logic, or is it just an interface on top of someone else's model.
- Understanding. The team explains how the key part works, rather than just pointing to the AI.
- Usefulness and originality of the approach, rather than the mere fact of using AI.
A useful trick during the pitch: ask the team to explain a piece of their own code or show the project on a new example. This separates those who built it from those who just pressed a button in under a minute.
Banning AI is pointless
A ban is unenforceable; participants will use AI anyway, just more quietly. It is wiser not to ban it, but to level the playing field and hold them accountable for the result. If you give all teams equal access to the necessary models, no one loses due to an empty limit, and the evaluation becomes fairer. On Stavleak, organizers distribute sponsors' API keys to participants through their dashboard, and the jury scores based on your custom criteria from a separate login, so you configure the scale for the AI era yourself.
Write down the rules in advance
Whatever you decide, tell the participants before the start: can they use AI, do they need to disclose what was generated, is code understanding evaluated. Transparent rules eliminate half of the disputes at the award ceremony.
We have a general breakdown of criteria without reference to AI separately, and you can set up evaluation according to your rules on the Stavleak demo.