Java Concurrency for Backend Services¶

A backend service may handle many requests at once, share caches and clients, and execute work on a mixture of platform and virtual threads. The difficult part is not starting threads. It is preserving correctness while work overlaps, bounding scarce resources, and making task lifetimes predictable during overload and shutdown.

This chapter focuses on Java-specific decisions: the Java Memory Model (JMM), executor behavior, synchronization, and virtual threads. It complements the Go concurrency guide and the Python concurrency guide; it does not repeat their language comparisons.

The examples target Java 21 or later. Virtual threads became permanent in Java 21. API references use Java SE 26 and call out the Java 24 change to virtual-thread monitor pinning.

1. Start with ownership, not locks¶

Before choosing a synchronization primitive, ask who owns each piece of mutable state.

Immutability: construct an object fully, publish it, and do not mutate its fields afterwards. Immutable request metadata, configuration snapshots, and value objects are easier to share.
Thread confinement: keep mutable state inside one request, task, or worker. A local HashMap needs no lock when only one task can access it.
Message passing: hand data to another task through a queue, executor, or other concurrency primitive instead of sharing a mutable object graph.
Shared mutable state: when several threads must update the same state, define the invariant and protect the complete operation with a lock, an atomic operation, or a suitable concurrent collection.

A ConcurrentHashMap makes its individual operations thread-safe; it does not make a sequence such as if (!map.containsKey(k)) map.put(k, v) atomic. Prefer putIfAbsent, computeIfAbsent, or a transaction-like operation supplied by the collection. Even then, keep mapping functions short and avoid performing remote I/O inside them.

2. Understand the Java Memory Model¶

The JMM defines which writes one thread is allowed to observe from another. It is not enough for a field update to be atomic at the hardware level: another thread also needs a defined visibility and ordering relationship.

A data race exists when two threads access the same variable concurrently, at least one access is a write, and the accesses are not ordered by a happens-before relationship. A data race can produce stale or surprising observations even when each individual read or write appears simple.

The most useful happens-before rules for application code are:

Actions in one thread happen-before later actions in that same thread.
Unlocking a monitor happens-before a later lock of that same monitor.
A write to a volatile field happens-before a subsequent read of that field.
Calling Thread.start() happens-before actions in the started thread; actions in a thread happen-before another thread successfully returns from join() on it.
Actions before submitting a task to an Executor happen-before that task begins. For an ExecutorService, task actions happen-before a successful result retrieval through Future.get().

These edges compose transitively.

Visibility is not atomicity¶

volatile is useful for a state flag or a reference whose individual reads and writes need visibility. It does not make a compound operation atomic:

private volatile int requests;

void recordRequest() {
    requests++; // read, add, write: another thread can interleave
}

Use AtomicInteger.incrementAndGet() for an independent counter, or a lock if the counter participates in a larger invariant. Atomic classes provide atomic operations on one value; they do not make several related fields update as one transaction.

synchronized protects a critical section and establishes visibility through the monitor's unlock/lock relationship. The lock must be the same object on both paths. Avoid synchronizing on public objects, strings, boxed primitives, or objects that callers can also lock.

final fields help make immutable objects safely usable after construction, provided this does not escape from the constructor. They do not make referenced mutable objects immutable. A final List reference can still point to a list that another thread changes.

Safe publication in a service¶

Publish shared state through a mechanism with a defined memory-consistency guarantee: a volatile reference, a synchronized hand-off, a concurrent collection, or task submission through an executor. For example, build a configuration snapshot completely and then replace one volatile reference. Readers see either the old complete snapshot or the new complete snapshot, rather than a partially updated set of fields.

Do not treat volatile as a general replacement for locks. If updating a balance requires checking a limit and then changing two fields, the entire invariant needs one coordinated operation.

3. Choose an executor that matches the workload¶

An Executor separates task submission from the mechanics of running it. ExecutorService adds task results, cancellation, and lifecycle management. This is usually preferable to creating raw threads throughout application code.

Choose the execution model from the bottleneck:

Workload	Starting point	Main constraint
CPU-bound transformations	Bounded platform-thread pool near available processor capacity	CPU cores and allocation/GC cost
Blocking I/O with a modest fixed concurrency	Bounded platform-thread pool	Thread and downstream connection limits
Many concurrent blocking I/O tasks	Virtual thread per task	Database, remote API, file descriptor, and memory limits
Durable work that must survive process restart	Persistent queue or job system	Durable acceptance, retries, ordering, and idempotency

A thread pool is not automatically bounded. Executors.newFixedThreadPool(n) uses a shared unbounded queue. It caps active worker threads but does not cap queued work. If producers can submit faster than workers complete tasks, the queue can grow until latency and memory use become unacceptable.

When the queue must be bounded, configure ThreadPoolExecutor explicitly:

var executor = new ThreadPoolExecutor(
    8, 8,
    0L, TimeUnit.MILLISECONDS,
    new ArrayBlockingQueue<>(64),
    new ThreadPoolExecutor.AbortPolicy()
);

This admits at most eight running tasks and 64 queued tasks. When both limits are full, AbortPolicy throws RejectedExecutionException. Translate that into an explicit overload response at the service boundary—for example, a 503 with a bounded Retry-After where retrying is safe—or apply a documented fallback. Do not silently discard work that the caller believes was accepted.

The numbers above are illustrative. Size a pool from measured service time, CPU or I/O behavior, downstream capacity, and the request latency budget. A large queue does not create throughput; it stores waiting time and memory. If a request can wait only 300 ms, admitting it behind several seconds of queued work is not useful capacity.

CallerRunsPolicy can slow a producer by making it execute rejected work, but it changes where latency is paid. It can block a request thread, event loop, or scheduler that should remain responsive. Use it only when the submitting thread is safe to block and the feedback effect is intentional. DiscardPolicy and DiscardOldestPolicy are unsuitable for work whose completion matters unless loss is explicitly part of the contract.

4. Use virtual threads for blocking I/O—not as a capacity limit¶

Virtual threads are lightweight Java threads scheduled by the runtime over a smaller set of carrier platform threads. They make a thread-per-request style practical for services that spend much of their time waiting on blocking I/O. They do not make CPU instructions execute faster and do not add CPU cores.

For independent blocking operations, Java 21+ can create one virtual thread per task:

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    Future<Customer> customer = executor.submit(() -> customerClient.fetch(id));
    Future<Orders> orders = executor.submit(() -> orderClient.fetchFor(id));

    return new Dashboard(customer.get(), orders.get());
}

The executor is intentionally not a pool of reusable virtual threads. Create a virtual thread for each task; do not pool them to imitate the resource management strategy used for platform threads.

But virtual threads are cheap threads, not unlimited downstream capacity. If every request can open a database connection or call a partner API at once, a large burst can still exhaust the connection pool or overload the partner. Put the limit around the scarce resource: use a connection pool for database connections, a semaphore for a partner's concurrent-call budget, and a bounded durable queue for work that must wait across process restarts.

A semaphore can cap concurrent access while many virtual threads wait. If waiting must be bounded, use tryAcquire(timeout, unit) and reject or degrade when a permit is not available in time. Release acquired permits in finally; do not hold unrelated locks while waiting for network I/O.

Virtual-thread pinning is version-sensitive. In Java 21–23, blocking while holding an intrinsic monitor could pin a virtual thread to its carrier. Java 24's JEP 491 changed monitor implementation so synchronized code no longer causes that form of pinning. Native methods and foreign-function calls can still pin a virtual thread. Measure on the actual JDK and dependencies you deploy; do not mechanically replace every synchronized block because of old pinning advice.

Virtual threads also change the economics of ThreadLocal: a high number of live virtual threads can make per-thread retained objects expensive. Avoid storing large request state in long-lived thread locals, and do not assume that thread identity is a durable request or security context.

5. Keep critical sections small and design for interruption¶

A lock should protect the smallest operation that preserves an invariant. Do not hold a monitor while calling a database, making an HTTP request, waiting on a future, or sleeping. Those operations can block for a long time while every other thread that needs the lock waits behind them.

Deadlocks commonly arise when two threads acquire the same locks in different orders. Establish one global lock order, avoid nested locks where possible, and use time-bounded acquisition only when the failure policy is clear. A timeout that breaks a deadlock attempt does not automatically repair partially changed state.

Prefer higher-level primitives when they express the relationship directly:

BlockingQueue for producer/consumer hand-off.
Semaphore for a finite number of permits to a resource.
CountDownLatch for one-time coordination.
CompletableFuture for composing completion stages; pass an explicit executor for blocking work rather than placing it on the common pool by accident.
Concurrent collections and atomic classes for supported per-key or per-value operations.

Interruption is Java's cooperative cancellation signal. Blocking methods such as sleep, wait, join, and semaphore acquisition can throw InterruptedException. If a method cannot propagate that checked exception, restore the interrupt flag before returning or translating the error:

catch (InterruptedException interrupted) {
    Thread.currentThread().interrupt();
    throw new RequestCancelledException(interrupted);
}

Do not catch InterruptedException and continue as if nothing happened. Do not assume shutdownNow() forcibly kills code; it normally interrupts running tasks and stops queued tasks from starting. Tasks must cooperate by checking interruption and using interruptible operations.

6. Make executor lifecycle part of application lifecycle¶

An executor owns threads and queued work. Create it at a deliberate application scope, share it only among the components that share its capacity policy, and close it during service shutdown. Avoid constructing a new pool per request.

shutdown() stops accepting new tasks but lets already submitted tasks finish. awaitTermination() waits for completion. shutdownNow() makes a best-effort interruption attempt and returns tasks that never started. On interruption during shutdown, call shutdownNow() and restore the current thread's interrupt status.

A Future is a failure boundary: observe task results and exceptions, and decide whether sibling tasks should be cancelled. For CompletableFuture, remember that async methods without an explicit executor normally use the common ForkJoin pool; blocking JDBC or HTTP work can starve unrelated tasks if it is placed there.

If work must survive a process crash, an in-memory executor is not a durable queue. Persist the job or publish it through a durable messaging system before acknowledging acceptance. Then design retries, idempotency, expiry, and poison-message handling as a separate delivery contract.

7. Diagnose concurrency in production¶

Measure the queue and the work, not only the number of threads. Useful signals include:

active, pool, and peak thread counts;
queue depth and oldest queued-task age;
task wait time before execution and task execution time;
rejection count by executor and task type;
semaphore wait time and downstream connection-pool wait time;
CPU utilization, context switching, allocation rate, and GC pauses;
virtual-thread pinning and submit-failure events when virtual threads are used.

A growing queue with stable worker utilization often means tasks are waiting on a dependency or the pool is undersized for the actual work. High CPU and low queueing may point to CPU saturation instead. A pool with all workers blocked on tasks submitted to that same pool can deadlock through thread starvation: reserve capacity for dependent work or avoid synchronously waiting for child tasks on the same saturated executor.

Use thread dumps and Java Flight Recorder to investigate blocked threads, lock ownership, virtual-thread pinning, and long-running tasks. Correlate task names and request identifiers in logs, but avoid putting high-cardinality request IDs into metric labels. Alert on user-visible latency and saturation, not merely on a thread count crossing an arbitrary threshold.

8. Interview questions with reasoned answers¶

1. What does happens-before guarantee?

It defines visibility and ordering between actions. If one action happens-before another, the later action is allowed to observe the earlier action's effects. It is established by program order and synchronization edges such as monitor unlock/lock, volatile write/read, thread start/join, and executor submission/result retrieval. It is not a claim that the CPU must physically execute every instruction in that order.

2. Why is volatile int count; count++ still unsafe?

count++ is a read-modify-write sequence. Volatile gives visibility and ordering for each access, but two threads can both read the same value and overwrite one another's increment. Use an atomic increment or protect the whole operation with a lock.

3. Why can newFixedThreadPool(16) still exhaust memory?

The factory uses a fixed number of workers and an unbounded shared queue. If arrival rate stays above completion rate, waiting tasks accumulate without a queue-capacity limit. A bounded ThreadPoolExecutor with an explicit rejection policy makes overload visible.

4. When would you choose virtual threads over a platform-thread pool?

For a large number of mostly blocking I/O tasks where straightforward synchronous code is useful. They improve throughput by allowing carriers to run other virtual threads while one is blocked; they do not make CPU-bound code faster. Limit database and remote-service concurrency separately.

5. Does a semaphore make a method thread-safe?

Only if every access that must be coordinated uses the same semaphore and the permit scope covers the entire invariant. A semaphore can cap concurrent calls to a partner API, but it does not automatically protect unrelated shared fields or make multiple operations atomic.

6. What causes thread-starvation deadlock in a pool?

All workers can block waiting for child tasks that are queued to the same pool. No worker remains to execute those children. Avoid blocking parent tasks on work submitted to the same saturated executor, separate execution domains, or use non-blocking composition.

7. What is the difference between shutdown() and shutdownNow()?

shutdown() rejects new tasks and lets submitted tasks finish. shutdownNow() attempts to interrupt running tasks and prevents queued tasks from starting, but it cannot forcibly stop code that ignores interruption. Both require an explicit wait if the caller must know when termination completes.

8. How would you investigate rising request latency in a Java service?

Separate queue wait from task execution time, then inspect active workers, queue age, rejection counts, CPU, downstream pool waits, and lock contention. Use thread dumps/JFR to find blocked or pinned threads. Change one capacity or synchronization policy at a time and compare p95/p99 latency and error rates under a representative load.

9. Practical review checklist¶

Identify which state is immutable, confined, or shared.
Write down the invariant before choosing volatile, an atomic, a lock, or a concurrent collection.
Select platform threads or virtual threads from the workload, not habit.
Bound scarce downstream resources even when using virtual threads.
Bound queues and define what rejection means to the caller.
Keep locks away from network calls, database waits, and future joins.
Propagate interruption and observe task failures.
Own executor creation and shutdown at application scope.
Measure queue age, wait time, task time, saturation, and rejections.
Test overload, cancellation, shutdown, and failure paths—not only successful parallel execution.