In this benchmark we compare Codex vs Claude Code to see how well these AI coding agents understand an unfamiliar codebase.
AI coding tools are evolving rapidly. What started as simple code autocomplete has now become something much more powerful – AI coding agents capable of analyzing entire repositories, refactoring code, and even implementing new features across multiple files.
Two of the most interesting tools in this new category are OpenAI Codex and Claude Code. Both promise to act as intelligent development assistants, but they approach the problem differently: one behaves more like an autonomous coding agent, while the other works closer to an IDE-integrated assistant.
To better understand their strengths and weaknesses, we decided to run a small benchmark using real development tasks. Instead of theoretical comparisons, both agents receive the same prompts and work with the same codebase. The goal is simple: see how they perform when faced with typical development scenarios such as understanding an existing project, detecting bugs, refactoring code, or implementing changes across multiple files.
Because these tasks can vary significantly in complexity, we decided to break this benchmark into a series of small practical tests, each focusing on a specific capability of AI coding agents. This approach allows us to examine their behavior more closely and keep each comparison focused and transparent.
In this first article, we start with one of the most fundamental tasks in real development: understanding an unfamiliar codebase. Before writing or modifying code, developers usually need to understand how a project works and how its components interact.
The results below show how each AI agent analyzed the project and explained its architecture.
Test #1 – Code Understanding
The first and most basic capability of any AI coding agent is the ability to understand an unfamiliar codebase. In real development, this is one of the most common tasks: developers often need to quickly understand how a project works before making changes.
For this test, both AI agents were given access to the same small open-source project. The goal was simple: analyze the codebase and explain how the project works and how its components interact.
Source project used in this test
This project was chosen because it is small enough to analyze quickly but still contains multiple files and logical components, making it suitable for testing architectural understanding.
Prompt used in this test:
Explain how this project works and describe its architecture.
Both agents received exactly the same prompt and the same source project.
Test #1 – Results
Below are the responses generated by each AI agent.
Claude Code – Test #1

OpenAI Codex – Test #1

Project Overview
This is a passphrase-style password generator — a single-file Python CLI tool.
Architecture
The project is minimal: one script (pwgen.py), one word list (words_alpha.txt), and documentation.
File Structure
pwgen.py/
├── pwgen.py # Main script (44 lines)
├── words_alpha.txt # ~370k English words (source data)
├── README.md # Usage docs
├── SPECS.md # Feature checklist / changelog
└── LICENSE # MIT
How It Works
The script follows a linear pipeline of 4 functions:
1. read_words(file_path) — line 4 Reads words_alpha.txt and filters to words with 4–8 characters only.
2. select_random_words(words, count=5) — line 10 Picks 5 words at random (no repeats via random.sample).
3. change_words_to_uppercase(words, count) — line 14 Randomly uppercases 1–4 of the 5 words (adds visual entropy).
4. generate_random_hex() — line 21 Generates a 4-digit hex string (e.g. A3F1) from 0001 to FFFF.
generate_password() — line 25 orchestrates all of the above and joins everything with dashes:
SHITTING-empiry-TENSES-dashy-BIFIDITY-1629
CLI entry point — line 35 Uses argparse with one optional positional argument count (default: 5) to control how many passwords to print.
Password Anatomy
word1-word2-WORD3-word4-WORD5-HEXN
4-8 4-8 4-8 4-8 4-8 4 hex digits
- 5 words (mixed case, 1–4 uppercased randomly)
- 1 hex number appended at the end
- All segments dash-separated
Usage
python pwgen.py # generates 5 passwords python pwgen.py 3 # generates 3 passwords python pwgen.py -h
I’m inspecting the repository structure and key files first, then I’ll summarize how the project behaves and how the pieces fit together.
The repo is small: one Python entrypoint, a word list, and docs. I’m reading the implementation and docs now to map behavior against the stated spec.
I have enough context. I’m pulling line-numbered source so the explanation can point to the exact implementation boundaries instead of paraphrasing loosely.
Ran $i=1; Get-Content pwgen.py | ForEach-Object { ‘{0,4}: {1}’ -f $i, $_; $i++ }
Ran $i=1; Get-Content README.md | ForEach-Object { ‘{0,4}: {1}’ -f $i, $_; $i++ }
Worked for 29s
This project is a small single-file CLI password generator. Its implementation lives entirely in pwgen.py, and it depends on a local dictionary file, words_alpha.txt, plus two documentation files: README.md and SPECS.md.
How It Works
When you run the script, the __main__ block parses one optional positional argument, count, which defaults to 5 and controls how many passwords to print (pwgen.py (line 35) to pwgen.py (line 43)). For each password, it calls generate_password() (pwgen.py (line 25)).
That function is a straight pipeline:
- read_words() loads words_alpha.txt and keeps only words with length 4 through 8 (pwgen.py (line 4) to pwgen.py (line 8)).
- select_random_words() picks 5 distinct words with random.sample() (pwgen.py (line 10) to pwgen.py (line 12)).
- A random integer x from 1 to 4 is chosen, then change_words_to_uppercase() uppercases exactly x of those 5 words in place (pwgen.py (line 14) to pwgen.py (line 19), pwgen.py (line 29)).
- generate_random_hex() creates a 4-digit uppercase hex suffix from 0001 to FFFF (pwgen.py (line 21) to pwgen.py (line 23)).
- The final password is word1-word2-word3-word4-word5-XXXX via ‘-‘.join(…) (pwgen.py (line 31) to pwgen.py (line 32)).
Architecture
The architecture is intentionally minimal and procedural:
- CLI layer: argument parsing and output loop in the __main__ section.
- Generation layer: generate_password() orchestrates the full workflow.
- Utility layer: four helper functions handle file loading, word selection, case mutation, and hex generation.
There are no classes, no packages, no persistent state, and no external dependencies beyond the Python standard library (random, argparse). Data flows one way: file input -> filtered word list -> sampled words -> partial uppercase transform -> hex suffix -> printed string.
One notable implementation detail: the dictionary file is re-read fresh for every generated password because generate_password() calls read_words() each time. That keeps the design simple, but it is less efficient than loading the word list once and reusing it.
Evaluation – Test #1 (Code Understanding)
Codex
Codex correctly identified the structure of the project and explained how the program works.
The response focuses on the execution pipeline and specific implementation details, including references to exact line numbers in the source code. It also highlights a potential inefficiency in the implementation (reloading the word list for every password generation), which shows deeper code inspection.
Overall, Codex provides a more technical and implementation-oriented explanation.
Claude Code
Claude quickly identified the purpose of the project and described the structure of the script and the role of each function. The explanation is very clear and well structured, making it easy to understand how the program works.
However, the explanation focuses more on high-level structure and does not reference specific implementation details or potential design issues.
Quick comparison
To keep the comparison practical, we evaluated both responses based on structure, clarity, and implementation awareness.
| Metric | Codex | Claude Code |
|---|---|---|
| Identifies project components | ✔ | ✔ |
| Architecture explanation | Detailed | Good |
| Execution flow description | ✔ | ✔ |
| Clarity of explanation | Good | Excellent |
| Implementation insights | ✔ | ✖ |
Short verdict
Both AI agents correctly understood the project and described its structure. Claude provided a clearer and more user-friendly explanation, while Codex delivered a more technical analysis and even identified a potential inefficiency in the implementation.