Beyond Physical Cores: Virtual Threads in Java

While physical CPU cores provide the foundation for parallelism, traditional operating system threads can become resource-intensive for applications with a high number of concurrent tasks. Enter virtual threads in Java 16, a lightweight alternative that offers several advantages

Initial Setup:

Suppose we have 2 core machines and we create 4 virtual threads.

• JVM creates 4 virtual threads: VirtualThread1, VirtualThread2, VirtualThread3, and VirtualThread4.
• The JVM schedules them on the available OS threads:
  • VirtualThread1 runs on OSThread1.
  • VirtualThread2 runs on OSThread2.

VirtualThread1 encounters I/O:

• VirtualThread1 makes an I/O call and gets blocked.

1. Work Stealing Opportunity: When VirtualThread1 gets blocked on I/O, the JVM recognizes an opportunity for work stealing to utilize the idle OS thread (OSThread1).
2. Initial Target: VirtualThread2: The JVM might first check if VirtualThread2, which is running on the other OS thread (OSThread2), has any suitable work units in its queue that can be stolen and executed on OSThread1.
3. Busy VirtualThread2: If VirtualThread2 is actively processing and doesn’t have readily stealable work units, the JVM needs to explore other options.
4. Expanding the Search: The JVM can then turn to VirtualThread3 and VirtualThread4. They might be running on the same OS thread as VirtualThread1 (OSThread1) or on the other OS thread (OSThread2) depending on the scheduling decisions.
5. Checking Queues: The JVM examines the queues of VirtualThread3 and VirtualThread4 for suitable work units. This suitability depends on factors like the size of the work unit, its dependencies, and potential cache locality benefits.
6. Stealing from Available Option: If the JVM finds suitable work units in the queue of either VirtualThread3 or VirtualThread4, it can steal that work and run it on the idle OSThread1 (where VirtualThread1 was previously running).

Key Points:

• Work stealing is not limited to just the other actively running thread (VirtualThread2 in this case).
• The JVM explores all available virtual threads to find suitable work units for efficient utilization of CPU cores.
• Scheduling decisions might influence which virtual thread’s queue gets checked first, but ultimately, all are potential targets for work stealing.

In essence, the JVM strives to maximize resource utilization by stealing work from any available virtual thread that has suitable work units, not just the initially busy thread. This ensures efficient use of CPU cores even when some virtual threads are blocked on I/O operations.

Here's how to create virtual threads in Java 16:

There are two main approaches to create virtual threads:

1. Using the Thread.ofVirtual() factory method:

This is the simplest and most common way. Here's the syntax:

Thread thread = Thread.ofVirtual().start(() -> {
  // Your task to be executed in the virtual thread
  System.out.println("Running in a virtual thread!");
});

2. Using the Executors.newVirtualThreadPerTaskExecutor() method:

This approach is useful when you want to create a thread pool with virtual threads. Each submitted task will be executed in a separate virtual thread.

ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

executor.submit(() -> {
  // Your task to be executed in a virtual thread
  System.out.println("Running in a virtual thread from the pool!");
});

executor.shutdown(); // Gracefully shut down the executor when done