Project Title: Custom HTTP Server & Transparent Proxy Implementation in Go

System Overview

This project implements a custom HTTP server and transparent proxy application developed for Distributed Systems coursework. Built entirely using Go’s low-level TCP primitives (net package), the system bypasses the standard net/http library. The implementation uses the understanding of the HTTP/1.1 protocol specification, concurrent network programming, and manual resource management.

Architecture & Implementation

1. HTTP Server (http_server.go)

The server operates by establishing a TCP listener and manually handling the byte-stream processing for incoming connections.

• Request Parsing: The system reads raw bytes from the connection, extracts the HTTP method (GET/POST) and Target URI, and maps requests to the internal www/ filesystem.
• GET Implementation: Handles static asset retrieval. The server performs manual content-type detection to serve .jpg, .txt, and .css files with appropriate headers.
• POST Implementation: Supports file uploads by reading the request body directly from the TCP stream and committing the binary data to local storage.

2. Transparent Proxy (proxy/proxy.go)

The proxy serves as an intermediary agent, forwarding traffic between clients and upstream servers.

• Request Reconstruction: The proxy interprets incoming request packets, modifies necessary headers for forwarding, and establishes a secondary TCP connection to the destination host.
• Response Relay: Upon receiving data from the upstream server, the proxy captures the full response—including status codes, headers, and the response body—and streams it back to the original client.

Concurrency Management

To prevent resource exhaustion under high load, the system implements a Semaphore Pattern using Go’s buffered channels. This limits the number of active goroutines processing requests simultaneously.

semaphore := make(chan struct{}, 10) // Concurrency limit
// ...
semaphore <- struct{}{} // Acquire token
go func() {
    defer func() { <-semaphore }() // Release token
    handleConnection(conn)
}()

This architecture ensures the server accepts connections non-blocking but processes only a fixed number of concurrent handlers, protecting system stability.

Security Protocols

The application enforces strict access controls to mitigate common web vulnerabilities:

• Path Traversal Protection: The system utilizes filepath.Abs to resolve requested paths, validating that all file operations occur strictly within the designated www/ root directory.
• Extension Whitelisting: File operations are restricted to specific allowed extensions (e.g., .html, .jpg), preventing the execution of unauthorized scripts or access to sensitive system files.

Key Technical Components

• Protocol Compliance: Strict adherence to HTTP statelessness, treating every request as an independent transaction.
• Binary Stream Handling: Precise buffer management to ensure data integrity for images and binary assets during TCP transmission.
• Thread-Safe State: Utilization of channels and atomic operations to manage shared resources in a multi-threaded environment.

Summary

This implementation delivers a functional HTTP server and proxy built from first principles. It serves as a practical application of network theory, validating the ability to architect performant, secure distributed systems without reliance on high-level frameworks.