Architecture

HTOS: High-Throughput Payment System

Financial-grade payment processing with ACID compliance and fraud prevention

Purpose

High-throughput payment processing API guaranteeing transactional integrity, idempotency, and double-spend prevention for financial systems.

Key Metrics

Target Throughput: Millions of transactions per day • Latency: Sub-second processing • Reliability: ACID compliance with strong transactional integrity

Architecture Overview

HTOS uses a multi-layered architecture designed for correctness and scalability:

  • API Layer: ASP.NET Core Web API handling payment endpoints, validation, and orchestration
  • Database Layer: MySQL 8 with ACID transactions and row-level locking for data consistency
  • Cache Layer: Redis 7 for high-speed idempotency key lookups and future rate limiting
  • Domain Layer: Clean domain entities, enums, and exception handling for business logic
  • Infrastructure: Repository pattern, EF Core 9 migrations, and automated database seeding
Client Request
    │
    ▼
┌─────────────────────────────┐
│  ASP.NET Core API Layer     │
│ ├─ Validation               │
│ ├─ Idempotency Check        │
│ └─ Payment Orchestration    │
└──────────┬──────────────────┘
           │
    ┌──────┴──────┐
    │             │
    ▼             ▼
┌─────────┐   ┌─────────┐
│ MySQL   │   │ Redis   │
│ ACID TX │   │ Cache   │
│ Tables  │   │ Keys    │
└─────────┘   └─────────┘

Core Responsibilities

API Layer

Request validation, authentication, idempotency handling, payment orchestration, structured logging and telemetry

MySQL Database

ACID transactions with READ COMMITTED isolation, row-level locking, balance consistency checks, and ledger entries

Redis Cache

O(1) idempotency key lookups, prevents duplicate transactions during retry storms, reduces DB contention

Domain Layer

Business rule enforcement, entity integrity validation, domain exceptions, and type-safe payment operations

Double-Spend Prevention

Prevents unauthorized duplicate charges through atomic database transactions:

  1. Account row is selected with pessimistic locking
  2. Current balance is validated against transaction amount
  3. Balance is decremented atomically
  4. Ledger entries are recorded for audit trails
  5. Payment transaction status is recorded
  6. If any step fails → entire transaction rolls back (no partial updates)

Idempotency Pattern

Ensures same result for duplicate requests, critical for network reliability:

  1. Client sends Idempotency-Key header with unique GUID
  2. API checks Redis for existing key (O(1) lookup)
  3. If found → return cached response immediately (duplicate detected)
  4. If not found:
  5. Process payment within atomic database transaction
  6. Store response in Redis with 24-hour TTL
  7. Return result to client

Technology Stack

Backend

  • .NET 10
  • ASP.NET Core Web API
  • Entity Framework Core 9

Data

  • MySQL 8
  • Redis 7

DevOps

  • Docker
  • Docker Compose

Observability

  • Serilog
  • Structured Logging
  • Metrics

Engineering Decisions

MySQL for ACID

Selected for strong transactional guarantees, row-level locking, and proven reliability in financial systems.

Redis for Speed

Provides O(1) idempotency lookups and drastically reduces database contention under high-volume retry scenarios.

Pessimistic Locking

Guarantees absolute correctness by preventing concurrent modifications to the same account during processing.

Domain-Driven Design

Encapsulates business rules within domain entities, making payment logic clear, testable, and maintainable.

Scalability Considerations

While the current architecture is robust, here are identified bottlenecks and mitigation strategies:

MySQL Row Locks

Issue: Heavy concurrent writes to same account cause lock contention

Mitigation: Horizontal scaling via read replicas, query optimization, and potentially distributed transaction coordinators

Redis Memory

Issue: Unbounded idempotency key storage can exhaust memory

Mitigation: Configurable TTL policies, automatic eviction strategies, and memory monitoring

API Thread Pool

Issue: Thread pool exhaustion under spike traffic without rate limiting

Mitigation: Implement rate limiting (ready with Redis), circuit breakers, and async/await patterns

Key Takeaways

  • Transactional integrity is paramount in financial systems - sacrificed performance for correctness
  • Idempotency is essential for handling network unreliability at scale
  • Smart caching (Redis) dramatically improves performance without sacrificing correctness
  • Domain-driven design makes complex business logic understandable and maintainable
  • Plan for bottlenecks early - design with horizontal scaling in mind from day one
.NET 10 ASP.NET Core MySQL Redis ACID Payment Processing Distributed Systems Docker