Build a URL Shortener from Scratch — Part 1: Requirements and Architecture

Series Overview

We’re building a complete URL shortener (like Bitly or TinyURL) from scratch. This series covers every aspect: design, implementation, caching, analytics, and deployment.

Technology stack: FastAPI (Python), PostgreSQL, Redis, Docker.

Requirements

Functional Requirements

  • Shorten URL: User submits a long URL, gets a short code
  • Redirect: Visiting https://short.ly/abc123 redirects to the original URL
  • Custom aliases: Users can specify their own short code (e.g., /my-portfolio)
  • Expiration: Links can expire after a set time
  • Analytics: Track click count, referrer, and geography per link

Non-Functional Requirements

  • Low latency: Redirects must complete in under 50ms
  • High availability: The service must handle spikes (viral links)
  • Scalability: Support millions of URLs, billions of redirects
  • Uniqueness: No duplicate short codes across different long URLs

Architecture Overview

┌──────────┐      ┌──────────────┐      ┌────────────┐
│  Client  │─────▶│   FastAPI    │─────▶│ PostgreSQL │
│ (Browser)│◀─────│    Server    │◀─────│  (URLs)    │
└──────────┘      └──────┬───────┘      └────────────┘
                         │
                         ▼
                  ┌────────────┐
                  │    Redis   │
                  │  (Cache)   │
                  └────────────┘

Flow:

  1. POST /shorten → Generate short code → Store in PostgreSQL → Cache in Redis → Return short URL
  2. GET /{code} → Check Redis cache → If miss, query PostgreSQL → Redirect to original URL → Increment analytics

Database Design

We’ll define the schema in detail in Part 2. For now, the core entities:

urls table:

  • id (BIGSERIAL)
  • short_code (VARCHAR, UNIQUE)
  • original_url (TEXT)
  • created_at (TIMESTAMP)
  • expires_at (TIMESTAMP, nullable)
  • user_id (optional, for multi-user support)

clicks table (for analytics):

  • id (BIGSERIAL)
  • url_id (FK → urls.id)
  • clicked_at (TIMESTAMP)
  • referrer (TEXT)
  • country (VARCHAR)
  • user_agent (TEXT)

Short Code Generation Strategy

We’ll use Base62 encoding of a counter-based approach:

ID 1 → "b"
ID 100 → "bM"
ID 1,000,000 → "4c92"

Base62 uses characters [a-zA-Z0-9] — 62 possible characters. This gives us:

  • 5 chars: 62^5 ≈ 916 million combinations
  • 7 chars: 62^7 ≈ 3.5 trillion combinations

We’ll use Snowflake-style IDs (time-based, distributed) converted to Base62.

Project Structure

url-shortener/
├── app/
│   ├── main.py
│   ├── config.py
│   ├── models.py
│   ├── schemas.py
│   ├── services/
│   │   ├── shortener.py
│   │   └── analytics.py
│   ├── utils/
│   │   └── base62.py
│   └── api/
│       └── routes.py
├── tests/
├── requirements.txt
├── Dockerfile
└── docker-compose.yml

Next Up

In Part 2, we’ll design the PostgreSQL schema, set up Alembic migrations, and implement the Base62 encoding/decoding logic. Continue to Part 2 →

References

Advertisement