Is Major Cooked? Free AI risk score + roast for your college major
Solo Era solo activities, solo maxxing, sidequests, challenge
Keepsake Project Management, Freelancers, DesignersIntermediate Python projects: 12 builds that teach packaging, concurrency and testing
Intermediate Python projects use third-party packages, run as more than one file and can fail in ways you do not control. Good ones are a price tracker that scrapes and stores history, a REST API with tests, a command-line tool you publish as a package, an async link checker and a spam classifier built from scratch. Below are 12, each with a stack, a first version, a stretch goal and the test that proves it works.
What counts as an intermediate Python project?
A beginner project is one file, runs on your machine and uses only the standard library. An intermediate project has structure, depends on packages, has tests and is meant to be run by someone else. The table shows the shift. If a project below sounds too big, build the beginner version first from Python projects for beginners.
| Beginner | Intermediate | |
|---|---|---|
| Layout | One file | A package with modules, tests and a pyproject.toml |
| Dependencies | Standard library | Pinned third-party packages in a virtual environment |
| Input | What you type | Files, web pages, API responses that change without warning |
| Failure | A traceback you read | Timeouts, bad data and partial runs the program must survive |
| Tests | A few examples | Unit tests, plus a test for each bug you fix |
| Users | You | Someone who has only your README |
Four tools make these projects easier to finish. pytest runs tests with less boilerplate than unittest. Ruff lints and formats code. uv manages Python versions, virtual environments and dependencies. Type hints, checked by an editor or a type checker, catch a class of mistakes before you run anything. Use all four on the first project and they become habit.
Which intermediate Python projects work with web pages and data?
These five take outside data and make it dependable. The skill is not fetching a page. It is surviving the day the page changes, the network drops or the file arrives with a missing column.
01 Price drop tracker
A script that checks one product page each day, stores the price history and tells you when it falls below a number you set.
- Stack
- requests, Beautiful Soup, sqlite3, smtplib or a terminal message, cron.
- Teaches
- Scraping, CSS selectors, a history table, scheduling, defensive parsing.
- First version
- Fetch the page, extract the price, insert (timestamp, price) into SQLite, print an alert under the threshold.
- Stretch goal
- Many pages, a polite delay between requests, an alert when the selector finds nothing, a price chart.
- Time
- 12 to 20 hours.
The failure to design for is silence. When the shop changes its layout, the selector returns nothing, and a naive script reports no price drops forever. Treat “no price found” as an error that sends you a message. Check the site’s terms and its robots.txt before you scrape it, and use urllib.robotparser to read the rules in code.
02 Reading-list REST API with tests
A small web API that creates, lists, updates and deletes books with a status such as to-read, reading or finished.
- Teaches
- HTTP verbs and status codes, validation, database access, testing an API without a server.
- First version
- POST, GET, PATCH and DELETE on /books; invalid input returns 422; ten tests using FastAPI’s test client.
- Stretch goal
- Pagination, filtering by status, an API key header, a Dockerfile.
- Time
- 12 to 20 hours.
FastAPI builds interactive documentation for you, so you can try each endpoint in a browser while you work. The learning is in the tests, described in FastAPI’s testing tutorial: create a book, read it back, patch it, delete it, then check that a second delete returns 404. If you want a front end later, the same API serves a web or mobile app.
Build a reading-list API with FastAPI and SQLite. Book has id, title (1 to 200 characters), author (optional), status (to-read, reading or finished, default to-read). Endpoints: POST /books, GET /books?status=, GET /books/{id}, PATCH /books/{id}, DELETE /books/{id}. Validation errors return 422; unknown ids return 404.
Do not build: accounts, a front end, search.
Write pytest tests with the test client for every endpoint, plus a test that deleting twice returns 404 the second time.03 Async link checker
A tool that reads a list of URLs, checks them all at the same time and reports which are broken, slow or redirected.
- Teaches
- Concurrency, limiting it with a semaphore, timeouts, retries, comparing run times.
- First version
- Read urls.txt, check every URL with a 10-second timeout and at most 20 requests in flight, print status and time.
- Stretch goal
- Crawl a site two levels deep, honor robots.txt, write a JSON or HTML report.
- Time
- 10 to 16 hours.
Measure before and after. If 500 URLs take 0.4 seconds each, checking them one at a time takes 200 seconds. With 20 in flight at once the same work is about 10 seconds, because the time is spent waiting on the network, not computing. Cap concurrency, or you will hammer one server and get blocked. Retry only on timeouts and 5xx errors, never on 404.
04 Personal finance dashboard
Upload your bank CSV and see spending by month and category as charts, with unusual months flagged.
- Teaches
- DataFrames, grouping, charts, a simple interactive app, outlier detection.
- First version
- Upload a CSV, map columns, apply keyword categories from a rules file, show a stacked bar chart per month.
- Stretch goal
- Budget lines per category, and a flag on any month where a category is more than two standard deviations above its average.
- Time
- 10 to 16 hours.
Keep the categorizing rules in a separate file that you edit as plain text, not in code. The test data is the hard part: write a 30-row sample CSV with a refund, a duplicate and a row with a missing amount, and make the dashboard handle all three. Keep your real statements out of the repository.
05 Validated daily data pipeline
A job that downloads a public CSV each day, rejects bad rows with a reason and loads the good ones into SQLite without creating duplicates.
- Stack
- requests, Pydantic, sqlite3, logging, cron.
- Teaches
- Validation, idempotency, upserts, logging, running unattended.
- First version
- Download, validate each row with a Pydantic model, insert with an upsert, write rejected rows and reasons to a second table.
- Stretch goal
- A warning when today’s row count drops by more than 30 percent against yesterday, and a one-page quality report.
- Time
- 12 to 20 hours.
The test that matters is running the whole job twice and getting the same database. That property is called idempotency, and it is what lets you rerun after a failure without fear. Pick a public dataset that updates daily, such as a government open-data feed for your city, so the project has real data and real surprises.
Which Python projects teach structure, packaging and testing?
Writing code that works is the beginner goal. Writing code that someone else can install, extend and trust is the intermediate one. These four are about that second goal.
01 Command-line tool published as a package
Take a script you already wrote, such as the to-do list, and turn it into an installable package with a real command name.
- Stack
- pyproject.toml, a src layout, pytest, the Python Packaging User Guide.
- Teaches
- Package layout, entry points, versions, building and uploading a distribution.
- First version
- pip install -e . gives a working todo command; tests run from the package.
- Stretch goal
- Upload to TestPyPI, install it in a fresh environment, and run tests on every push with a CI workflow.
- Time
- 8 to 12 hours.
The packaging guide walks through building and uploading to TestPyPI, a practice index that is safe to experiment on. Package names must be unique, so pick a distinctive one. Installing your own tool in a clean environment is the real test, because it exposes every file your code quietly depended on being in the current folder.
[build-system] requires = ["setuptools>=61"] build-backend = "setuptools.build_meta" [project] name = "todo-cli-yourname" version = "0.1.0" description = "A command-line to-do list" requires-python = ">=3.11" dependencies = [] [project.scripts] todo = "todo_cli.main:main"
02 Plugin-based file converter
A converter you run as convert in.csv out.json, where each format is a plugin that another package can add.
- Stack
- importlib.metadata entry points, typing.Protocol, argparse, pytest.
- Teaches
- Interfaces, plugin discovery, separating a core from its extensions.
- First version
- A converter registry with CSV to JSON and JSON to CSV built in, chosen by file extension.
- Stretch goal
- A second package, installed separately, that adds a new format and is found automatically through an entry point.
- Time
- 12 to 20 hours.
Define what a converter is before you write one: a name, the extensions it reads and writes, and one function that takes bytes and returns bytes. If the core never imports a plugin by name, your design works. The proof is deleting a plugin and watching the tool still run, with a clear message when you ask for the missing format.
03 Cache decorator library with property-based tests
A decorator that remembers a function’s results for a set number of seconds, published as a small library.
- Stack
- functools, threading, Hypothesis, type hints.
- Teaches
- Decorators, closures, thread safety, typing, property-based testing.
- First version
- @ttl_cache(seconds=30) caches by arguments and expires entries; the cache can be cleared.
- Stretch goal
- A maximum size with least-recently-used eviction, support for async functions, hit and miss statistics.
- Time
- 10 to 16 hours.
Write the tests so they do not wait for real time: pass the decorator a clock function you can move forward in a test. Hypothesis generates hundreds of argument combinations and shrinks a failing one to the smallest example. A good property to check is that a cached call always returns exactly what the uncached function would have returned.
04 Terminal time tracker
A keyboard-driven app in the terminal where you start and stop a timer per project and see today’s totals.
- Stack
- Textual, sqlite3, datetime.
- Teaches
- Event-driven programs, widgets, application state, saving data safely.
- First version
- Press a key to start or stop a project timer; a table shows today’s time per project; data lives in SQLite.
- Stretch goal
- A weekly CSV export and recovery of a timer left running when the program crashed.
- Time
- 12 to 20 hours.
Store start and stop timestamps in UTC and compute totals from them, never from a running counter in memory. If the program is killed, the open row with no stop time tells you exactly what was running. Test the total-calculating function without the screen at all, including a timer that crosses midnight.
Which advanced Python projects should I try next?
Advanced here means the project is limited by an idea, not by a library. You will spend most of the time on a data structure, an algorithm or a measurement. Each one comes with a way to check your answer against something trusted.
01 Log analyzer for files bigger than memory
A command that reads a web server log of any size and reports the top paths, status code counts and the slowest requests.
- Stack
- Generators, re, argparse, concurrent.futures, the profiler.
- Teaches
- Streaming, memory use, regular expressions at scale, profiling, parallelism.
- First version
- Parse a log in nginx’s combined format one line at a time and print a summary; memory use stays flat.
- Stretch goal
- Split the file into byte ranges, process them in parallel and merge the counts, then show the speed-up you measured.
- Time
- 10 to 16 hours.
Generate a test log yourself with a script, 100 MB or more, so you know the true answer. Profile before you optimize: the slow part is often the regular expression, not the file reading. Report the time before and after each change, or you cannot tell which change helped.
02 Sudoku solver with constraint propagation
A program that solves any valid sudoku puzzle in milliseconds and tells you when a puzzle has more than one solution.
- Stack
- Pure Python; Peter Norvig’s essay Solving Every Sudoku Puzzle as a reference after you try alone.
- Teaches
- Backtracking, constraint propagation, choosing the most constrained cell first, benchmarking.
- First version
- Read an 81-character puzzle string, solve it, print the grid, reject invalid puzzles.
- Stretch goal
- Count solutions to prove uniqueness, then generate puzzles with exactly one solution and report puzzles per second.
- Time
- 10 to 18 hours.
Plain backtracking works on easy puzzles and crawls on hard ones. Adding one rule, which is always to try the cell with the fewest remaining candidates, changes the run time more than any micro-optimization. That difference is the lesson, so keep both versions and time them on the same fifty puzzles.
03 Naive Bayes spam classifier from scratch
A text classifier you write without a machine learning library, then compare against scikit-learn on the same data.
- Stack
- Pure Python for the model; the UCI SMS Spam Collection for data; scikit-learn for the comparison.
- Teaches
- Tokenizing, word counts, smoothing, log probabilities, train and test splits, precision and recall.
- First version
- Train on 80 percent of the messages, report accuracy on the other 20 percent.
- Stretch goal
- A confusion matrix, precision and recall for the spam class, and a list of the ten words that most indicate spam.
- Time
- 12 to 20 hours.
Add up logarithms of probabilities instead of multiplying the probabilities. Multiplying hundreds of small numbers underflows to zero. Use add-one smoothing so a word the model never saw in spam does not zero out the whole message. Accuracy alone misleads when most messages are not spam, so report precision and recall too.
Here is my Naive Bayes spam classifier (code below). Do not rewrite it. Review it and list: (1) any place where probabilities are multiplied instead of log-summed, (2) any case where an unseen word could cause a zero or a division by zero, (3) any leak of test data into training. Quote the line for each finding. Then say which metric I should report besides accuracy and why. [paste code here]
How do I make a Python project worth showing to others?
A project is worth showing when a stranger can judge it in five minutes. Do these before you share a repository:
- Put the problem in the first line of the README, then a screenshot or a pasted terminal session.
- Give the install and run commands exactly as you ran them in a fresh environment.
- List what you chose not to build, and why. It shows judgment.
- Show the test command and its output.
- Add a section called “What broke” with two real bugs and how you found them.
If you are stuck for something to build, the developer tools category on the board is full of small problems that fit a Python script or service, and each idea comes with a generated build prompt. For project lists in other languages and at other levels, see coding project ideas and software engineering project ideas.
Frequently asked questions
What are advanced Python projects?
Advanced Python projects are limited by an idea rather than a library: a sudoku solver with constraint propagation, a log analyzer that streams files bigger than memory, a job queue, or a classifier written from scratch. They need profiling, tests and a way to check your answer against a trusted reference. Start with the sudoku solver or the log analyzer above.
How do I move from beginner to intermediate Python?
Change one thing at a time. Add tests to a beginner project, then move it into a package, then add a dependency, then publish or deploy it. The first intermediate project should use one third-party library and one outside data source. The REST API and the price tracker above are good first steps because failures are visible.
Should I learn Flask, Django or FastAPI first?
Pick by the job. FastAPI suits small JSON APIs and uses type hints for validation. Django includes accounts, an admin and a database layer, which suits a full site. Flask is minimal and flexible. For a first API with tests, FastAPI is a short path. For a site with users, Django saves the most work.
How do I test a web scraper?
Save a few real pages as files and test your parser against those files, never against the live site. Add a test where the page is missing the element you look for and assert that the scraper raises a clear error. Then keep one small script that fetches the live page, run by hand, to catch layout changes.
Do I need type hints in Python projects?
You do not need them, but they pay off from the first intermediate project. Editors use them for autocomplete and to flag mistakes before you run the code. They also document what a function expects. Add hints to function signatures first and leave local variables alone. Libraries such as Pydantic and FastAPI use them directly.
Created with AI assistance and edited for this site. Examples are illustrative and do not represent customer research. About the publisher.