Where's the Next 100x?

Beyond 2PL: Rethinking Concurrency

Two-Phase Locking is just one option in a sea of alternatives. The next 100x performance leap demands fundamentally different architectures.

Explore non-locking systems, timestamp ordering, and micro-granularity approaches that defy traditional limits.

The Opportunity Space

In the real world, most schedules aren't conflict-serializable under 2PL. Yet, there are countless valid schedules we can safely execute.

Conflict serializability venn diagram showing the larger space of possible schedules

The blue and white areas represent untapped potential—valid, safe schedules that current locking systems overlook.

Three Paths to the Next 100x

Micro-Locking: Surgical Precision

How it works:
• Fine-grained locks on specific data elements
• Lock only what you need
• Advanced deadlock detection algorithms
• Hierarchical locking for scalability

Performance: Maximum safe concurrency
Trade-off: Complex lock management overhead
Best for: High-contention concurrent systems

LSM-Tree Approach: No Locks Needed

How it works:
• Append-only writes to immutable structures
• No update-in-place operations
• Background compaction merges data
• Reads from multiple levels

Limits: Needs value to be mergeable (e.g., likes += 1)
Performance: Unlimited write throughput
Trade-off: Eventually consistent reads
Best for: Write-heavy workloads (logs, time-series, IoT)

Timestamp Ordering: Global Sequencing

How it works:
• Assign unique timestamps to all operations
• Execute operations in timestamp order
• Abort conflicts, not wait for locks
• Deterministic without coordination

Performance: No lock contention delays
Trade-off: May abort late operations
Best for: Distributed systems, global ordering needs

Choosing Your Path

For Analytics Workloads

Use LSM Trees or columnar stores. Append-only writes with background compaction eliminate lock contention entirely.

Examples: ClickHouse, BigQuery, Snowflake

For Distributed Systems

Use Timestamp Ordering with vector clocks. Global ordering without distributed locking protocols.

Examples: Spanner, Calvin, FaunaDB

For High-Value Transactions

Use Micro-Locking with sophisticated deadlock detection. Maximum concurrency with safety guarantees.

Examples: PostgreSQL, SQL Server, Oracle

Case Studies: Architecture at Scale

One line: When you see "high write throughput," "global coordination," or "fine-grained contention," think beyond 2PL to these architectural alternatives.

10M Ops/sec

TikTok's Viral Video Storm

Problem: Video goes viral during Super Bowl halftime. 10M concurrent users like, comment, share, and follow creator simultaneously. Single video entity becomes hottest contention point in the system.

Approach: Micro-Locking

Design: Separate locks for like_count, comment_list, share_count, follower_count • Field-level locking instead of entity-level • Lock-free atomic counters for simple increments • Sophisticated deadlock detection
Performance: 10M ops/sec on single entity, 99.9% concurrency vs 0.01% with entity locks
Why: Entity-level lock = 1 operation at a time. Field-level locks = 4 operations in parallel. Atomic counters eliminate locks entirely for simple increments.

100M+ Scale

Spotify's Play Counter Chaos

Problem: Ed Sheeran's new song gets 10M plays in first hour. Every play increments counters for: total plays, daily plays, monthly plays, artist stats, genre stats, playlist positions. Traditional locking creates massive contention bottlenecks.

Approach: LSM-Tree (No Locks)

Design: Append all play events to immutable SSTables • Background process aggregates counts • Real-time estimates from recent events • No write-write conflicts possible
Performance: 500K writes/sec, 50ms lag for exact counts, unlimited throughput
Why: 2PL would serialize all counter updates. 10M concurrent incrementers = 10M lock waits vs append-only writes that never conflict.

10K Agents

AI Agent Collaboration Platform

Problem: 10,000 AI agents simultaneously update shared knowledge graph for real-time research. Each agent reads facts, derives new knowledge, and writes back discoveries. Traditional 2PL creates deadlock storms between interdependent agents.

Approach: Timestamp Ordering

Design: Each agent gets unique timestamp when starting operation • All updates applied in timestamp order • Conflicts detected by comparing timestamps • Late agents abort and retry with new timestamp
Performance: 1M operations/sec, 5% abort rate, deterministic results
Why: 2PL with 10K agents creates circular wait chains. Global timestamp ordering eliminates deadlocks - conflicts resolve by "first timestamp wins" rule.