Home / Learn / Concurrency & Parallelism

🔀 Concurrency & Parallelism

Doing many things at once. 51 topics, from beginner to advanced.

Concurrency and parallelism are about running multiple tasks at once. Concurrency structures a program to handle many tasks that may overlap in time, while parallelism runs them literally at the same time on several cores. This area covers threads, locks, race conditions, deadlocks and the asynchronous models used in modern servers and user interfaces.

Who this area is for

Developers writing servers or any program that must do several things at once, and engineers tuning multi-core performance.

Real-world use cases

Handling thousands of network connections in one web server without a thread for each.
Splitting a large matrix multiplication across all CPU cores.
Keeping a user interface responsive while a file downloads in the background.
Handling thousands of network connections in one server without a thread for each.
Splitting an image-processing job across all the cores of a laptop.

Topics in Concurrency & Parallelism

Beginner (17)

Concurrency vs Parallelism — Interleaved progress versus simultaneous execution.
Sequential Bottlenecks — Serial portions limiting overall speedup.
Amdahl's Law — Formula bounding speedup by the serial fraction.
Threads in Practice — Creating and joining threads in programs.
Thread Pools — Reusing a fixed set of worker threads.
Thread Safety — Code behaving correctly when called from multiple threads.
Shared State Hazards — Bugs from unsynchronized access to common data.
Locks in Practice — Using mutual exclusion correctly.
Read-Write Locks — Locks allowing many readers or one writer.
Condition Variables — Waiting threads signaled when state changes.
Producer-Consumer Pattern — Threads exchanging work through a shared buffer.
Atomic Operations — Indivisible hardware-supported updates.
Compare-and-Swap — Primitive updating a value only if unchanged.
Memory Barriers — Instructions constraining reordering of memory operations.
Volatile and Visibility — Ensuring one thread's writes are seen by others.
Starvation and Livelock — Tasks never progressing despite not being blocked.
Priority Inversion — Low-priority task blocking a high-priority one.

Intermediate (17)

Async/Await — Syntax writing asynchronous code in sequential style.
Event-Driven Concurrency — Single-threaded handling of many tasks via an event loop.
Coroutines — Functions that suspend and resume cooperatively.
Futures and Promises Model — Handles for results computed elsewhere.
Green Threads — User-space scheduled lightweight threads.
Actor Model — Isolated entities communicating by messages.
Communicating Sequential Processes — Concurrency via channels between processes.
Go Goroutines and Channels — Lightweight tasks communicating through typed channels.
Lock-Free Data Structures — Structures coordinating via atomics instead of locks.
Wait-Free Algorithms — Guarantees that every operation finishes in bounded steps.
ABA Problem — Hazard when a value changes and changes back unnoticed.
Software Transactional Memory — Treating memory updates like database transactions.
Data Parallelism — Applying the same operation across partitions of data.
Task Parallelism — Running different tasks at the same time.
Fork-Join Model — Splitting work into subtasks then merging results.
Parallel Reduction — Combining many values using a tree of operations.
Parallel Prefix Sum — Scan primitive computing running totals in parallel.

Advanced (17)

OpenMP — Directive-based shared-memory parallel programming.
MPI — Message-passing interface for cluster programming.
CUDA Programming — NVIDIA's model for general-purpose GPU computing.
GPU Memory Hierarchy — Global, shared and register memory on GPUs.
Parallel Sorting — Sorting algorithms designed for many processors.
Load Balancing in Parallel Programs — Distributing work evenly to avoid idle workers.
False Sharing — Performance loss when threads modify data on one cache line.
NUMA — Memory architecture with non-uniform access times.
Parallel Graph Algorithms — Adapting traversals and shortest paths to many cores.
Concurrency Testing — Finding timing bugs with stress and model checking.
Deterministic Parallelism — Designs ensuring reproducible results.
Reactive Programming — Programming with asynchronous data streams.
Backpressure — Signaling producers to slow when consumers lag.
Linearizability — Correctness condition making operations appear atomic.
Memory Models in Languages — Java and C++ rules for concurrent memory behavior.
Hardware Transactional Memory — CPU support for optimistic atomic regions.
Performance Scaling Analysis — Measuring speedup and efficiency as cores increase.

How to study Concurrency & Parallelism

Write a program with two threads that update one counter. Run it many times, observe the wrong results, then fix it with a lock or an atomic operation. Then read the memory-model topics to explain why the bug happened.

Frequently asked questions

What is the difference between concurrency and parallelism?

Concurrency is about structuring a program to manage many tasks that can interleave. Parallelism is about executing tasks at the same instant on multiple cores.

What is a race condition?

A race condition occurs when the outcome depends on the timing of threads reading and writing shared data. Protecting that data with locks or atomic operations prevents it.

What causes a deadlock?

A deadlock happens when threads each hold a resource that another needs and none can proceed. Acquiring locks in a consistent order is a common way to avoid it.

Does more threads always mean faster code?

No. Threads add overhead, and they compete for shared data and locks. Speed improves only when the work is independent and each thread has enough to do.

Why is concurrency bug-prone?

Bugs depend on timing, so they may appear rarely and vanish when you add logging. Shared mutable data and missing synchronization are the usual causes.

Open the interactive Concurrency & Parallelism area Take the Concurrency & Parallelism quiz