Skip to content
Embercode
✕
Contest setup and the judgeLesson 1 of 8

35 min read

How contests work

Why this matters when you build with AI

An AI tool can write a working solution in seconds. A contest does not reward a working solution as such. It rewards the points or solved problems you have when the clock stops, and that depends on the scoring rules, on which problems you pick, and on when you submit.

Many people who use AI for coding learn this the hard way. They ask a model for a solution, run it on the sample input from the statement, see the right answer, and submit. The verdict then says the solution fails on some hidden test. The fix is not to trust the model more or less. The fix is to know what the contest measures, to check the code on cases the sample does not cover, and to decide before the round starts when to stop working on a problem and move on.

This lesson covers the rules of a round. Later lessons cover how to read limits, avoid overflow, stress test and read a verdict.

Words used in this lesson

A round is one contest session with a fixed start, a fixed length and a set of problems. A problem is one task with a statement, an input format and an output format. A submission is one upload of your source code to the judge. The judge is the system that compiles your code, runs it on test inputs and compares the output with the correct answer. A verdict is the judge's result for one submission, for example "accepted" or "wrong answer".

A test is one input file with its correct output. A sample test is one or two tests shown in the statement. A hidden test is a test you cannot see. A subtask is a group of tests that share a rule, such as "all numbers are at most 100". A scoreboard is the table that shows every team or person, the problems they solved and the total.

Two common round shapes

Most rounds you will meet follow one of two shapes.

Olympiad style is used by national and international informatics olympiads. Each person works alone. A round has a few problems, often three or four, and each problem is worth a number of points. Points come from tests or subtasks, so you can earn part of a problem's value even when the solution is not perfect. The person with the most points wins.

ICPC style is used by the ICPC (International Collegiate Programming Contest) and by many university contests. Regional rounds usually use teams of three people who share one computer. A problem counts as solved only if it is fully accepted. The team with the most solved problems wins, and penalty time breaks ties. Rules differ between regions, so read the rules of the round you enter.

The two shapes reward different habits. In an olympiad, a partial solution that passes the easy subtask still earns points, so it pays to write something safe first. In an ICPC round, a half-correct solution earns nothing, and the time spent on it is lost.

Scoring modes

A scoring mode says how a problem is turned into points. There are three common modes.

All or nothing

A problem is worth its full value only when every hidden test passes. Anything less is worth zero. ICPC rounds work this way. It is simple to read, and it rewards finishing.

Subtask points

A problem is split into subtasks. Each subtask has its own points and its own rule on the inputs. Olympiad rounds often use this mode.

The usual rule, as written in the IOI rules for 2026, is that a submission's score for a subtask is the lowest score among its tests in that subtask. The final score for a subtask is the best score over all of your submissions. The rule can be changed by the statement of a single problem, so always read the statement.

Here is a worked example with numbers made up for this lesson. A subtask is worth 25 points and has three tests. Suppose three submissions score like this.

SubmissionTest scoresLowest test scorePoints for the subtask
first attempt1.0, 0.4, 1.00.4010.00
second attempt1.0, 0.6, 0.80.6015.00
third attempt1.0, 1.0, 1.01.0025.00

The final score for this subtask is 25.00, because the third attempt has the best value. The first attempt is not wasted, but it does not add to the second or third.

The next block is the calculation, run as a program so the numbers are exact.

Python · editable

The output of that program, run with Python 3.14.4, is:

solved at minute   0: 300.00 points
solved at minute  60: 200.00 points
solved at minute 120: 100.00 points
solved at minute 150:  90.00 points
solved at minute 170:  90.00 points
first attempt   min test 0.40 -> 10.00 points
second attempt  min test 0.60 -> 15.00 points
third attempt   min test 1.00 -> 25.00 points
final subtask score: 25.00 points (best over all submissions)

Time decay

In a time-decay mode, a problem is worth less the later you solve it. Some rounds use this to reward speed. A simple linear version works like this. The problem starts at its full value at minute 0. It loses value in a straight line until the end of the round. It never drops below a floor, here 30 percent of the start value.

The program above shows the rule for a 300-point problem in a 180-minute round. At minute 60 the value is 200 points, at minute 120 it is 100, and from minute 150 on the floor of 90 points applies. Real rounds use their own formulas and floors. The round's rules say which one, and some rounds use no decay at all. Do not carry a formula from one round to another.

Time decay changes the plan. A problem that you could solve in fifteen minutes is worth a lot at minute 20 and much less at minute 150. In that kind of round, a moderate problem solved early can beat a hard problem solved late.

Penalty rules

ICPC-style rounds use penalty time to break ties between teams with the same number of solved problems. The standard rule, as written in the Universal Cup 2024 to 2025 season rules and in other ICPC-style rulebooks, is this: the penalty for a problem is the minute of its first accepted submission, plus 20 minutes for every wrong submission on that problem before the first accepted one. A wrong submission counts for this purpose unless it is a compile error. A problem you never solve adds nothing to the penalty, and its wrong attempts are ignored.

The team total is the sum of the penalty times of all solved problems. Ranking is by solved count first, and then by lower total penalty.

A concrete case helps. Suppose a team solves problem A at minute 40, after two wrong submissions on A. The penalty for A is 40 + 2 × 20 = 80. Suppose it solves problem B at minute 55 with no wrong submission. The penalty for B is 55. The team has solved two problems with a total penalty of 135. A team that solved two problems with a total penalty of 100 ranks above it.

Two things follow from this rule. First, a wrong submission costs 20 minutes only if you later solve the problem. A team that tries a problem five times and never solves it loses nothing on the penalty, but it also gains nothing. Second, the minute of your first accepted submission is the largest part of the penalty. Submitting a fast, wrong answer is not free: it costs 20 minutes if the problem is later solved.

The worked example in the next section is an implementation of this rule.

Hidden tests and what you see

When you submit, the judge runs your program on test files. Some of those files are shown to you, usually the sample tests in the statement. Most are hidden, so you learn only the verdict.

Some judges run in two phases. During the round, a submission is checked on a smaller set of tests, called pretests on Codeforces. After the round, the full set is run, and that second run is often called the system test. A submission can pass the pretests and still fail a system test, so passing during the round does not prove your solution is right. Check the rules of your round: they say which tests run when, and whether a late verdict changes your rank.

The hidden tests are built to break wrong ideas. They include the largest inputs, the smallest inputs, equal values, values that are zero or negative when the statement allows it, and inputs that trigger overflow or slow code. The sample test is usually the easiest case in the set. A solution that matches the sample tells you very little.

This is why the lesson on stress testing, later in this module, asks you to write a brute force and a random generator. You will need them for hidden tests, because you will not see those tests.

How ratings move

A rating is a number that estimates how strong a person or team is in rated rounds. Codeforces uses an Elo-style rating, which Wikipedia describes as Elo. In an Elo system, you gain points when you do better than the rating of the field predicted, and you lose points when you do worse. Beating strong opponents gains more than beating weak ones. Rating changes come after the round, from the final ranking.

Two points are useful for planning. First, a rating reflects your results over many rounds, not one. A single bad round moves the number a little. Second, the exact formula, and whether a round is rated at all, depends on the platform and the round. The platform's own FAQ and the round's announcement are the only sources for that. This lesson does not state the formula, because the sources I could open did not give it in a form I could check.

The idea in practice: a scoreboard problem

Here is an original problem of the ICPC kind, so you can see the rule above turned into code.

Scoreboard. The first line has an integer m, the number of submissions, from 1 to 10^5. Each of the next m lines has four items: a team number, a problem number, the minute of the submission from 0 to 300, and a verdict that is either OK or BAD. The lines are in the order the submissions were made. For each team that appears in the input, print one line with its number of solved problems and its penalty, in the form team T solved S penalty P. Print the teams in increasing order of team number. A solved problem counts with its penalty as described above. Submissions to a problem already solved do not count.

The idea is a single pass through the log. For each team and problem, keep a count of the BAD submissions seen so far. When an OK arrives for a problem not yet solved, add its minute plus 20 times the count to the penalty, and add one to the solved count.

// Scoreboard (ICPC-style penalty). Input: m submissions, then m lines "team problem minute verdict".
// Verdict is OK or BAD. Minutes are integers from 0 to 300, in the order they were made.
// For each team print: solved count and penalty. A solved problem costs its first OK minute
// plus 20 for every BAD submitted on that problem before the first OK. Unsolved problems cost 0.
// Output: one line per team that appears, sorted by team id.
#include <bits/stdc++.h>
using namespace std;
 
int main() {
    int m = 0;
    if (scanf("%d", &m) != 1) return 0;
    map<int, map<int, int>> bad_before;      // team -> problem -> BAD count so far
    map<int, map<int, bool>> solved;         // team -> problem -> solved?
    map<int, int> solved_count, penalty;     // team -> totals
    for (int i = 0; i < m; i++) {
        int team = 0, problem = 0, minute = 0;
        char verdict[8] = {0};
        if (scanf("%d %d %d %7s", &team, &problem, &minute, verdict) != 4) break;
        penalty.try_emplace(team, 0);
        solved_count.try_emplace(team, 0);
        if (solved[team][problem]) continue;  // later submissions on a solved problem do not count
        if (string(verdict) == "OK") {
            solved[team][problem] = true;
            solved_count[team]++;
            penalty[team] += minute + 20 * bad_before[team][problem];
        } else {
            bad_before[team][problem]++;
        }
    }
    for (const auto &[team, cnt] : solved_count)
        printf("team %d solved %d penalty %d\n", team, cnt, penalty[team]);
    return 0;
}

Compile it with g++ -std=c++17 -O2 -Wall -Wextra -o scoreboard scoreboard.cpp. The compile is clean with those warnings turned on. Run it on this sample, which is a small input you can type in yourself.

7
1 1 10 BAD
1 1 15 OK
1 2 20 OK
2 1 5 BAD
2 1 9 BAD
2 1 30 OK
3 4 1 BAD

The output is:

team 1 solved 2 penalty 55
team 2 solved 1 penalty 70
team 3 solved 0 penalty 0

Check it by hand. Team 1 solves problem 1 at minute 15 after one BAD, so 15 + 20 = 35. It solves problem 2 at minute 20 with no BAD, so 20. The total is 55 with two solved problems. Team 2 solves problem 1 at minute 30 after two BAD submissions, so 30 + 40 = 70. Team 3 never solves anything, so its penalty is 0.

A brute force is a slow, obviously correct version used to check the fast one. For this problem the brute force takes each team and each problem in turn, scans the whole log again, and stops at the first OK. Here it is in Python.

# Brute force: for every (team, problem) pair, scan the whole log again in order.
import sys
tokens = sys.stdin.read().split()
m = int(tokens[0])
subs = []
for i in range(m):
    team, problem, minute, verdict = tokens[1 + 4 * i: 5 + 4 * i]
    subs.append((int(team), int(problem), int(minute), verdict))
teams = sorted({s[0] for s in subs})
for team in teams:
    count, penalty = 0, 0
    problems = sorted({s[1] for s in subs if s[0] == team})
    for p in problems:
        own = [s for s in subs if s[0] == team and s[1] == p]  # already in submission order
        bads = 0
        for _, _, minute, verdict in own:
            if verdict == "OK":
                count += 1
                penalty += minute + 20 * bads
                break
            bads += 1
    print(f"team {team} solved {count} penalty {penalty}")

The two programs were compared on 400 random logs with up to 25 submissions each, with teams and problems numbered 1 to 4 and minutes that never go down. They printed the same lines in all 400 cases. This checks the logic on small inputs. It does not check speed on the largest input, which is a separate question covered in later lessons.

Complexity. The C++ program reads each line once and does a constant number of map operations per line, so it runs in about m times log of the number of teams and problems. The brute force does a full scan for each team and problem pair, so it is much slower on large logs.

Common mistakes

Trusting the sample test. The sample is usually the easiest input in the set. Symptom: the program prints the right answer on the sample, and the judge reports a wrong answer on a hidden test. The exact wording of the verdict depends on the judge. The fix is a random small-input generator and a brute force, both covered later.

Spending the first hour on the hardest problem. Symptom: at the end of the round you have one problem attempted and none solved. The fix is to read every problem statement in the first ten or fifteen minutes, then choose the order by how many points or solved problems each one is worth to you.

Counting wrong submissions on unsolved problems as penalty. In the ICPC rule above, a problem you never solve adds no penalty. Symptom: a team with five wrong submissions on one unsolved problem and one solved problem appears to lose time that the scoreboard does not show. The fix is to add penalty only at the first accepted submission.

Forgetting that later submissions do not count on a solved problem. A submission after OK on the same problem does not change the penalty in the rule above. Symptom: your local calculation gives a higher penalty than the scoreboard. The fix is the continue line in the program above.

Submitting quickly without a test. Each wrong submission on a problem you later solve adds 20 minutes. Symptom: a fast submission that fails costs you 20 minutes for that problem and sometimes more, because you learn nothing new. The fix is a short test on the sample and at least one case you made up before you submit.

Checklist for a round

Keep this list on the desk for the first round you enter.

  • Read the rules of the round: scoring mode, penalty rule, which tests run during the round, and whether a late verdict counts.
  • Read every problem statement in the first ten to fifteen minutes.
  • Write your round plan (below) before the first submission.
  • Before each submission, run the sample and one case you made up.
  • Note the minute of every submission and every verdict in a log. Your log is the evidence when a scoreboard looks wrong.
  • Follow the stop rule you wrote. Do not invent a new one during the round.

Build your round plan

This exercise is for you to do on paper or in a text file before any real round. It takes about fifteen minutes. The result is a plan that you can keep and change after each round.

Write four parts.

  1. Round facts. The length of the round in minutes, the number of problems, the scoring mode, and the penalty rule. Write them from the rules of the round you plan to enter, not from memory.
  2. Reading order. Put each problem in the order you will read it. Give a short reason. A good reason is a number, such as "subtask one is worth 30 points and needs only a loop", or "the statement is short, so it is a fast solve".
  3. Time budget. Reserve some minutes at the end for a final check and for unexpected events. Divide the rest by the number of problems. Write the budget next to each problem.
  4. Stop rule. Write a rule for when you stop working on a problem and move on. Use a number. For example, "I stop after two wrong submissions or when I pass the time budget by 25 percent, whichever comes first. I come back to it after the other problems."

The budget formula is simple. Let T be the round length in minutes, R the reserve, and n the number of problems. The budget per problem is (T - R) / n. For a 300-minute round with eight problems and a 20-minute reserve, the budget is (300 - 20) / 8 = 35 minutes per problem.

Show the answer

A good plan for the round above might look like this. The round is 300 minutes with eight problems, all-or-nothing scoring, and the ICPC penalty rule. The reserve is 20 minutes.

Round facts: 300 minutes, 8 problems, all or nothing, ICPC penalty (20 per wrong submission
on a problem that is later solved). Pretests run during the round; full tests run after.
 
Reading order: read all eight statements in 15 minutes. Then solve in this order:
  1. Problem with the shortest statement and a direct formula (a fast solve).
  2. Problem with a sorting or counting idea that I can name in one sentence.
  3. The rest in order of how clear the idea is to me.
 
Time budget: (300 - 20) / 8 = 35 minutes per problem. Each problem gets a line in the log.
 
Stop rule: after two wrong submissions on one problem, or after 40 minutes on
it without a solution that passes my own case, I stop and move on. I return to it after the
other problems, and I do not submit a guess.

Summary of the rules

Olympiad rounds score by subtask and often reward partial work. ICPC rounds score by solved problems, and penalty time breaks ties. Time-decay rounds lower the value of a problem the later you solve it. Hidden tests decide the verdict, and the sample test is only a first check. A plan with a reading order, a time budget and a stop rule turns these facts into a sequence of decisions you can make during the round.

Practice

Work these on paper or in a file before you look at the answers. Each has a hint in the answer block.

Practice 1. The penalty for one team. A team has this log for one problem each. Problem A: BAD at minute 12, BAD at minute 14, OK at minute 40. Problem B: OK at minute 55. Problem C: BAD at minute 3, BAD at minute 4, BAD at minute 5, and never solved. Find the number of solved problems and the total penalty under the ICPC rule in this lesson.

Show the answer

Hint: problem C adds nothing, because it is never solved. Solved problems: A and B, so the count is 2. Penalty for A is 40 + 2 × 20 = 80. Penalty for B is 55. Total is 135. Answer: 2 solved, penalty 135.

Practice 2. Time decay. A 500-point problem is in a 240-minute round. Its value drops in a straight line from 500 at minute 0 to the round end. The floor is 40 percent of 500. What is the value if you solve it at minute 90? At minute 200?

Show the answer

Hint: first compute the straight-line value, then compare it with the floor. At minute 90 the line gives 500 × (1 - 90 / 240) = 312.5. At minute 200 the line gives 500 × (1 - 200 / 240) = 83.33, which is below the floor of 200. Answer: 312.5 points at minute 90, and 200 points at minute 200.

Practice 3. Subtask scores. A subtask is worth 30 points. Three submissions have these test scores: first [1.0, 0.5], second [0.7, 1.0, 1.0], third [1.0, 1.0, 0.9]. Using the IOI rule from this lesson, what is the final score for the subtask?

Show the answer

Hint: take the lowest test score for each submission, then the best of those. The lowest values are 0.5, 0.7 and 0.9. The best is 0.9. Answer: 0.9 × 30 = 27.0 points.

Practice 4. Make a stop rule. Your round has 10 problems and lasts 240 minutes. You reserve 30 minutes for the final check. Compute the time budget per problem. Then write a stop rule with one number for wrong submissions and one number for time over budget.

Show the answer

Hint: the budget is (240 - 30) / 10. A good rule states both numbers and what you do next. Answer: the budget is 21 minutes per problem. A sample rule: "I stop after three wrong submissions on a problem, or when I am 10 minutes over its budget. I write the problem's name in my log and go to the next one. I return only after the other problems have had their budget."

Practice 5. Find the change in a scoreboard. A student changes the scoreboard program so that every BAD submission adds 20 minutes to the penalty at once, even when the problem is never solved. Run the lesson's program and the changed rule on this input, and give both penalties for team 1.

2
1 1 10 BAD
1 2 20 OK

Show the answer

Hint: problem 1 is never solved, so the lesson's rule adds nothing for it. Problem 2 is solved at minute 20 with no BAD, so it adds 20. The changed rule adds 20 for the BAD on problem 1 as well. Answer: the lesson's program prints team 1 solved 1 penalty 20. The changed rule gives a penalty of 40, which is the error to look for: penalty appears for a problem that the team never solved.

Practice this topic: coding problems for Contest setup and the judge →