From e60ebc2348d1d11791edfbd8c84279d254aca304 Mon Sep 17 00:00:00 2001 From: Cay Horstmann Date: Sat, 15 Jul 2023 07:37:21 +0200 Subject: [PATCH 1/6] Added virtual-threads --- .../02_new-features/virtual-threads.md | 367 ++++++++++++++++++ 1 file changed, 367 insertions(+) create mode 100644 app/pages/learn/01_tutorial/02_new-features/virtual-threads.md diff --git a/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md b/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md new file mode 100644 index 0000000..d6a225c --- /dev/null +++ b/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md @@ -0,0 +1,367 @@ +--- +id: new_features.virtual_threads +title: Virtual Threads +slug: learn/new-features/virtual-threads +type: tutorial +category: awareness +category_order: 2 +layout: learn/tutorial.html +main_css_id: learn +subheader_select: tutorials +description: "Virtual Threads: What, Why, and How?" +author: ["cayhorstmann"] +--- + + +## Why Virtual Threads? + +When Java 1.0 was released in 1995, its API had about a hundred classes, among them `java.lang.Thread`. Java was the first mainstream programming language that directly supported concurrent programming. + +Since Java 1.2, each Java thread runs on a *platform thread* supplied by the underlying operating system. (Up to Java 1.1, on some platforms, all Java threads were executed by a single platform thread.) + +Platform threads have nontrivial costs. They require a few thousand CPU instructions to start, and they consume a few megabytes of memory. Server applications can serve so many concurrent requests that it becomes infeasible to have each of them execute on a separate platform thread. In a typical server application, these requests spend much of their time *blocking*, waiting for a result from a database or another service. + +The classic remedy for increasing throughput is a non-blocking API. Instead of waiting for a result, the programmer indicates which method should be called when the result has become available, and perhaps another method that is called in case of failure. This gets unpleasant quickly, as the callbacks nest ever more deeply. + +JEP 425 introduced *virtual threads* in Java 19. Many virtual threads run on a platform thread. Whenever a virtual thread blocks, it is *unmounted*, and the platform thread runs another virtual thread. (The name “virtual thread” is supposed to be reminiscent of virtual memory that is mapped to actual RAM.) Virtual threads became a preview feature in Java 20 (JEP 436) and are final in Java 21. + +With virtual threads, blocking is cheap. When a result is not immediately available, you simply block in a virtual thread. You use familiar programming structures—branches, loops, try blocks—instead of a pipeline of callbacks. + +Virtual threads are useful when the number of concurrent tasks is large, and the tasks mostly block on network I/O. They offer no benefit for CPU-intensive tasks. For such tasks, consider parallel streams or [recursive fork-join tasks](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/RecursiveTask.html). + +## Creating Virtual Threads + +The factory method `Executors.newVirtualThreadPerTaskExecutor()` yields an `ExecutorService` that runs each task in a separate virtual thread. For example: + +``` +import java.util.concurrent.*; + +public class VirtualThreadDemo { + public static void main(String[] args) { + final int NTASKS = 100; + ExecutorService service = Executors.newVirtualThreadPerTaskExecutor(); + for (int i = 0; i < NTASKS; i++) { + service.submit(() -> { + long id = Thread.currentThread().threadId(); + LockSupport.parkNanos(1_000_000_000); + System.out.println(id); + }); + } + service.close(); + } +} +``` + +By the way, the code uses `LockSupport.parkNanos` instead of `Thread.sleep` so that we don't have to catch the pesky `InterruptedException`. + +Perhaps you are using a lower-level API that asks for a thread factory. To obtain a factory for virtual threads, use the new `Thread.Builder` class: + +``` +Thread.Builder builder = Thread.ofVirtual().name("request-", 1); +ThreadFactory factory = builder.factory(); +``` + +Now, calling `factory.newThread(myRunnable)` creates a new (unstarted) virtual thread. The `name` method configures the builder to set thread names `request-1`, `request-2`, and so on. + +You can also use a builder to create a single virtual thread: + +``` +Thread t = builder.unstarted(myRunnable); +``` + +Alternatively, if you want to start the thread right away: + +``` +Thread t = builder.started(myRunnable); +``` + +Finally, for a quick demo, there is a convenience method: + +``` +Thread t = Thread.startVirtualThread(myRunnable); +``` + +Note that only the first approach, with an executor service, works with result-bearing tasks (callables). + +## Thread API Changes + +After a series of experiments with different APIs, the designers of Java virtual threads decided to simply reuse the familiar `Thread` API. A virtual thread is an instance of `Thread`. Cancellation works the same way as for platform threads, by calling `interrupt`. As always, the thread code must check the “interrupted” flag or call a method that does. (Most blocking methods do.) + +There are a few differences. In particular, all virtual threads: + +* Are in a single thread group +* Have priority `NORM_PRIORITY` +* Are daemon threads + +There is no API for constructing a virtual thread with another thread group. Trying to call `setPriority` or `setDaemon` on a virtual thread has no effect. + +The static `Thread::getAllStackTraces` method returns a map of stack traces of all *platform* threads. Virtual threads are not included. + +A new `Thread::isVirtual` instance method tells whether a thread is virtual. + +Note that there is no way to find the platform thread on which a virtual thread executes. + +Java 19 has a couple of changes to the `Thread` API that have nothing to do with virtual threads: + +* There are now instance methods `join(Duration)` and `sleep(Duration)`. +* The non-final `getId` method is deprecated since someone might override it to return something other than the thread ID. Call the final `threadId` method instead. + +As of Java 20, the `stop`, `suspend`, and `resume` methods throw an `UnsupportedOperationException` for both platform and virtual threads. These methods have been deprecated since Java 1.2 and deprecated for removal since Java 18. + +## Capturing Task Results + +You often want to combine the results of multiple concurrent tasks: + +``` +Future f1 = service.submit(callable1); +Future f2 = service.submit(callable2); +result = combine(f1.get(), f2.get()); +``` + +Before virtual threads, you might have felt bad about the blocking `get` calls. But now blocking is cheap. Here is a sample program with a more concrete example: + +``` +import java.util.concurrent.*; +import java.net.*; +import java.net.http.*; + +public class VirtualThreadDemo { + public static void main(String[] args) throws InterruptedException, ExecutionException { + ExecutorService service = Executors.newVirtualThreadPerTaskExecutor(); + Future f1 = service.submit(() -> get("https://horstmann.com/random/adjective")); + Future f2 = service.submit(() -> get("https://horstmann.com/random/noun")); + String result = f1.get() + " " + f2.get(); + System.out.println(result); + service.close(); + } + + private static HttpClient client = HttpClient.newHttpClient(); + + public static String get(String url) { + try { + var request = HttpRequest.newBuilder().uri(new URI(url)).GET().build(); + return client.send(request, HttpResponse.BodyHandlers.ofString()).body(); + } catch (Exception ex) { + var rex = new RuntimeException(); + rex.initCause(ex); + throw rex; + } + } +} +``` + +If you have a list of tasks with the same result type, you can use the `invokeAll` method and then call `get` on each `Future`: + +``` +List> callables = ...; +List results = new ArrayList<>(); +for (Future f : service.invokeAll(callables)) + results.add(f.get()); +``` + +Again, a more concrete sample program: + +``` +import java.util.*; +import java.util.concurrent.*; +import java.net.*; +import java.net.http.*; + +public class VirtualThreadDemo { + public static void main(String[] args) throws InterruptedException, ExecutionException { + ExecutorService service = Executors.newVirtualThreadPerTaskExecutor(); + List> callables = new ArrayList<>(); + final int ADJECTIVES = 4; + for (int i = 1; i <= ADJECTIVES; i++) + callables.add(() -> get("https://horstmann.com/random/adjective")); + callables.add(() -> get("https://horstmann.com/random/noun")); + List results = new ArrayList<>(); + for (Future f : service.invokeAll(callables)) + results.add(f.get()); + System.out.println(String.join(" ", results)); + service.close(); + } + + private static HttpClient client = HttpClient.newHttpClient(); + + public static String get(String url) { + try { + var request = HttpRequest.newBuilder().uri(new URI(url)).GET().build(); + return client.send(request, HttpResponse.BodyHandlers.ofString()).body(); + } catch (Exception ex) { + var rex = new RuntimeException(); + rex.initCause(ex); + throw rex; + } + } +} +``` + +## Rate Limiting + +Virtual threads improve application throughput since you can have many more concurrent tasks than with platform threads. That can put pressure on the services that the tasks invoke. For example, a web service may not tolerate huge numbers of concurrent requests. + +With platform threads, an easy (if crude) tuning factor is the size of the thread pool for those tasks. But you should not pool virtual threads. Scheduling tasks on virtual threads that are then scheduled on platform threads is clearly inefficient. And what is the upside? To limit the number virtual threads to the smallish number of concurrent requests that your service tolerates? Then why are you using virtual threads in the first place? + +With virtual threads, you should use alternative mechanisms for controlling access to limited resources. Instead of an overall limit on concurrent tasks, protect each resource in an appropriate way. For database connections, the connection pool may already do the right thing. When accessing a web service, you know your service, and can provide appropriate rate limiting. + +As an example, on my personal web site, I provide demo services for producing random items. If a large number of requests comes at an instant from the same IP address, the hosting company blacklists the IP address. + +The following sample program shows rate limiting with a simple semaphore that allows a small number of concurrent requests. When the maximum is exceeded, the `acquire` method blocks, but that is ok. With virtual threads, blocking is cheap. + +``` +import java.util.*; +import java.util.concurrent.*; +import java.net.*; +import java.net.http.*; + +public class RateLimitDemo { + public static void main(String[] args) throws InterruptedException, ExecutionException { + ExecutorService service = Executors.newVirtualThreadPerTaskExecutor(); + List> futures = new ArrayList<>(); + final int TASKS = 250; + for (int i = 1; i <= TASKS; i++) + futures.add(service.submit(() -> get("https://horstmann.com/random/word"))); + for (Future f : futures) + System.out.print(f.get() + " "); + System.out.println(); + service.close(); + } + + private static HttpClient client = HttpClient.newHttpClient(); + + private static final Semaphore SEMAPHORE = new Semaphore(20); + + public static String get(String url) { + try { + var request = HttpRequest.newBuilder().uri(new URI(url)).GET().build(); + SEMAPHORE.acquire(); + try { + Thread.sleep(100); + return client.send(request, HttpResponse.BodyHandlers.ofString()).body(); + } finally { + SEMAPHORE.release(); + } + } catch (Exception ex) { + ex.printStackTrace(); + var rex = new RuntimeException(); + rex.initCause(ex); + throw rex; + } + } +} +``` + +## Pinning + +The virtual thread scheduler mounts virtual threads onto carrier threads. By default, there are as many carrier threads as there are CPU cores. You can tune that count with the `jdk.virtualThreadScheduler.parallelism` VM option. + +When a virtual thread executes a blocking operation, it is supposed to be unmounted from its its carrier thread, which can then execute a different virtual thread. There are two situations in which this does *not* happen: + +1. When executing a `synchronized` method or block +2. When calling a native method or foreign function + +A thread that does either of these is called *pinned*. Being pinned is not bad in itself. But when a pinned thread blocks, there is a problem. The thread cannot be unmounted, and the carrier thread is blocked. That leaves fewer carrier threads for running virtual threads. + +In some situations, the virtual thread scheduler will compensate by starting another carrier thread. For example, in JDK 21, this happens for many file I/O operations, and when calling `Object.wait`. But this is an implementation detail that may change. You can control the maximum number of carrier threads with the `jdk.virtualThreadScheduler.maxPoolSize` VM option. + +Pinning is harmless if `synchronized` is used to avoid a race condition in an in-memory operation. However, if there are blocking calls, it would be best to replace `synchronized` with a `ReentrantLock`. This is of course only an option if you have control over the source code. + +You always have the option whether or not to use virtual threads. In particular, heavy file I/O work may be better suited for platform threads. + +To find out whether pinned threads are blocked, start the JVM with one of the options + +``` +-Djdk.tracePinnedThreads=short +-Djdk.tracePinnedThreads=full +``` + +You get a stack trace that shows when a pinned thread blocks: + +``` +... +org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) <== monitors:1 +... +``` + +Note that you get only one warning per pinning location! + +Alternatively, record with Java Flight Recorder, view with your favorite mission control viewer, and look for `VirtualThreadPinned` and `VirtualThreadSubmitFailed` events. + +The JVM will eventually be implemented so that `synchronized` methods or blocks no longer lead to pinning. Then you only need to worry about pinning for native code. + +The following sample program shows pinning in action. We launch a number of virtual threads that sleep in a synchronized method, blocking their carrier threads. A number of virtual threads are added that do no work at all. But they can't be scheduled because the carrier thread pool has been completely exhausted. Note that the problem goes away when you + +* use a `ReentrantLock` +* don't use virtual threads + +``` +import java.util.concurrent.*; +import java.util.concurrent.locks.*; + +public class PinningDemo { + public static void main(String[] args) throws InterruptedException, ExecutionException { + ExecutorService service = + Executors.newVirtualThreadPerTaskExecutor(); + // Executors.newCachedThreadPool(); + + final int TASKS = 20; + long start = System.nanoTime(); + for (int i = 1; i <= TASKS; i++) { + service.submit(() -> block()); + // service.submit(() -> rblock()); + } + for (int i = 1; i <= TASKS; i++) { + service.submit(() -> noblock()); + } + service.close(); + long end = System.nanoTime(); + System.out.printf("%.2f%n", (end - start) * 1E-9); + } + + public static synchronized void block() { + System.out.println("Entering block " + Thread.currentThread()); + LockSupport.parkNanos(1_000_000_000); + System.out.println("Exiting block " + Thread.currentThread()); + } + private static Lock lock = new ReentrantLock(); + public static void rblock() { + lock.lock(); + try { + System.out.println("Entering rblock " + Thread.currentThread()); + LockSupport.parkNanos(1_000_000_000); + System.out.println("Exiting rblock " + Thread.currentThread()); + } finally { + lock.unlock(); + } + } + public static void noblock() { + System.out.println("Entering noblock " + Thread.currentThread()); + LockSupport.parkNanos(1_000_000_000); + System.out.println("Exiting noblock " + Thread.currentThread()); + } +} +``` + +## Thread Locals + +A *thread-local variable* is an object whose `get` and `set` methods access a value that depends on the current thread. Why would you want such a thing instead of using a global or local variable? The classic application is a service that is not threadsafe, such as `SimpleDateFormat`, or that would suffer from contention, such as a random number generator. In these cases, a per-thread shared instance makes sense. + +However, if the shared instance is resource-intensive, there can be a problem when migrating to virtual threads. There will likely be far more of them than threads in a thread pool, and now you have many more expensive instances. In such a situation, you should rethink your sharing strategy. + +Another common use for thread locals is to provide “implicit” context, such as a database connection, that is properly configured for each task. Instead of passing the context from one method to another, the task code simply reads the thread-local variable. This is actually hard to get right with thread pools, since you don't want another thread use your database. But tasks are mapped 1:1 to virtual threads, so a thread-local context makes sense. + +An *inheritable thread-local variable* is a special kind of thread-local variable. When a new thread is started, it receives a copy of the parent's inheritable thread locals. This is just the behavior that you want for context information, but making these copies can be expensive. The copies are necessary because the child thread might mutate the thread-local values. + +Such mutation is uncommon. Run with the VM flag `jdk.traceVirtualThreadLocals` to get a stack trace when a virtual thread mutates a thread-local variable. + +## Conclusion + +* Use virtual threads to increase throughput when you have many tasks that mostly block on network I/O +* The primary benefit is the familiar “synchronous” programming style, without callbacks +* Don't pool virtual threads; use other mechanisms for rate limiting +* Check for pinning and mitigate if necessary +* Minimize thread-local variables in virtual threads + + + From 60c035d26b1f36904da83acdfc27f5e7f2b5f2ee Mon Sep 17 00:00:00 2001 From: Cay Horstmann Date: Sat, 15 Jul 2023 07:57:08 +0200 Subject: [PATCH 2/6] added me as author --- app/data/authors.yaml | 11 ++++++++++- .../01_tutorial/02_new-features/virtual-threads.md | 7 ++++--- package-lock.json | 3 ++- package.json | 3 ++- 4 files changed, 18 insertions(+), 6 deletions(-) diff --git a/app/data/authors.yaml b/app/data/authors.yaml index dfbf31d..8de9dd4 100644 --- a/app/data/authors.yaml +++ b/app/data/authors.yaml @@ -38,4 +38,13 @@ twitter: venkat_s website: https://agiledeveloper.com/ description: | - Venkat Placeholder. \ No newline at end of file + Venkat Placeholder. + +- name: Cay Horstmann + email: cay@horstmann.com + photo_url: https://horstmann.com/cay-pfh.jpg + github: cayhorstmann + twitter: cayhorstmann + website: https://horstmann.com + description: | + Cay Placeholder. diff --git a/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md b/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md index d6a225c..e7d1338 100644 --- a/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md +++ b/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md @@ -1,15 +1,16 @@ --- id: new_features.virtual_threads title: Virtual Threads -slug: learn/new-features/virtual-threads +slug: learn/virtual-threads type: tutorial category: awareness category_order: 2 -layout: learn/tutorial.html +group: refactoring-to-functional-style +layout: learn/tutorial-group-top.html main_css_id: learn subheader_select: tutorials description: "Virtual Threads: What, Why, and How?" -author: ["cayhorstmann"] +author: ["CayHorstmann"] --- diff --git a/package-lock.json b/package-lock.json index 7a9c501..6029cab 100644 --- a/package-lock.json +++ b/package-lock.json @@ -34,7 +34,8 @@ }, "devDependencies": { "dotenv": "^16.0.3", - "gulp-markdown-toc": "^1.1.0" + "gulp-markdown-toc": "^1.1.0", + "gulp-util": "^3.0.8" } }, "node_modules/-": { diff --git a/package.json b/package.json index 8fa098a..62f4815 100644 --- a/package.json +++ b/package.json @@ -29,6 +29,7 @@ }, "devDependencies": { "dotenv": "^16.0.3", - "gulp-markdown-toc": "^1.1.0" + "gulp-markdown-toc": "^1.1.0", + "gulp-util": "^3.0.8" } } From 5bb7ebe0f98779e63d7434eeec67dd8bf0782195 Mon Sep 17 00:00:00 2001 From: Chad Arimura Date: Thu, 20 Jul 2023 08:21:09 -0700 Subject: [PATCH 3/6] adding back top-level sections --- app/templates/pages/learn/index.html | 48 ++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/app/templates/pages/learn/index.html b/app/templates/pages/learn/index.html index bafeb2c..e3a8eeb 100644 --- a/app/templates/pages/learn/index.html +++ b/app/templates/pages/learn/index.html @@ -28,6 +28,18 @@

Running Your First Java Application

+
+

Staying Aware of New Features

+
    + {% for tutorial in file.data.tutorials.awareness | sort(false, false, "category_order") %} +
  • + {{ tutorial.title_html | safe }}
    +
    {{ tutorial.description_html | safe }}
    +
    • + {% endfor %} +
+
+

Getting to Know the Language

@@ -41,6 +53,42 @@

Getting to Know the Language

+
+

Mastering the API

+
    + {% for tutorial in file.data.tutorials.api | sort(false, false, "category_order") %} +
  • + {{ tutorial.title_html | safe }}
    +
    {{ tutorial.description_html | safe }}
    +
    • + {% endfor %} +
+
+ +
+

Organizing your Application

+
    + {% for tutorial in file.data.tutorials.organizing | sort(false, false, "category_order") %} +
  • + {{ tutorial.title_html | safe }}
    +
    {{ tutorial.description_html | safe }}
    +
    • + {% endfor %} +
+
+ +
+

Getting to know the JVM

+
    + {% for tutorial in file.data.tutorials.jvm | sort(false, false, "category_order") %} +
  • + {{ tutorial.title_html | safe }}
    +
    {{ tutorial.description_html | safe }}
    +
    • + {% endfor %} +
+
+ {{ contents | safe }} From 17b544efad48f8f60475ec22b6c8f8ec69eb6d70 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9?= Date: Thu, 20 Jul 2023 17:49:06 +0200 Subject: [PATCH 4/6] Fixed: syntax coloring. Fixed: table of content is not generated. Fixed: the page is not displayed on the learn page. --- ...rtual-threads.md => 00_virtual-threads.md} | 51 ++++++++++++------- 1 file changed, 33 insertions(+), 18 deletions(-) rename app/pages/learn/01_tutorial/02_new-features/{virtual-threads.md => 00_virtual-threads.md} (96%) diff --git a/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md similarity index 96% rename from app/pages/learn/01_tutorial/02_new-features/virtual-threads.md rename to app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md index e7d1338..9cc9cca 100644 --- a/app/pages/learn/01_tutorial/02_new-features/virtual-threads.md +++ b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md @@ -1,19 +1,27 @@ --- id: new_features.virtual_threads title: Virtual Threads -slug: learn/virtual-threads +slug: learn/new-features/virtual-threads type: tutorial category: awareness -category_order: 2 -group: refactoring-to-functional-style -layout: learn/tutorial-group-top.html +category_order: 1 +layout: learn/tutorial.html main_css_id: learn subheader_select: tutorials +toc: + - Why Virtual Threads? {why} + - Creating Virtual Threads {creating} + - Thread API Changes {api-changes} + - Capturing Task Results {task-results} + - Rate Limiting {rate-limiting} + - Pinning {pinning} + - Thread Locals {thread-locals} + - Conclusion {conclusion} description: "Virtual Threads: What, Why, and How?" author: ["CayHorstmann"] --- - +  ## Why Virtual Threads? When Java 1.0 was released in 1995, its API had about a hundred classes, among them `java.lang.Thread`. Java was the first mainstream programming language that directly supported concurrent programming. @@ -30,11 +38,12 @@ With virtual threads, blocking is cheap. When a result is not immediately availa Virtual threads are useful when the number of concurrent tasks is large, and the tasks mostly block on network I/O. They offer no benefit for CPU-intensive tasks. For such tasks, consider parallel streams or [recursive fork-join tasks](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/RecursiveTask.html). +  ## Creating Virtual Threads The factory method `Executors.newVirtualThreadPerTaskExecutor()` yields an `ExecutorService` that runs each task in a separate virtual thread. For example: -``` +```java import java.util.concurrent.*; public class VirtualThreadDemo { @@ -57,7 +66,7 @@ By the way, the code uses `LockSupport.parkNanos` instead of `Thread.sleep` so t Perhaps you are using a lower-level API that asks for a thread factory. To obtain a factory for virtual threads, use the new `Thread.Builder` class: -``` +```java Thread.Builder builder = Thread.ofVirtual().name("request-", 1); ThreadFactory factory = builder.factory(); ``` @@ -66,24 +75,25 @@ Now, calling `factory.newThread(myRunnable)` creates a new (unstarted) virtual t You can also use a builder to create a single virtual thread: -``` +```java Thread t = builder.unstarted(myRunnable); ``` Alternatively, if you want to start the thread right away: -``` +```java Thread t = builder.started(myRunnable); ``` Finally, for a quick demo, there is a convenience method: -``` +```java Thread t = Thread.startVirtualThread(myRunnable); ``` Note that only the first approach, with an executor service, works with result-bearing tasks (callables). +  ## Thread API Changes After a series of experiments with different APIs, the designers of Java virtual threads decided to simply reuse the familiar `Thread` API. A virtual thread is an instance of `Thread`. Cancellation works the same way as for platform threads, by calling `interrupt`. As always, the thread code must check the “interrupted” flag or call a method that does. (Most blocking methods do.) @@ -109,11 +119,12 @@ Java 19 has a couple of changes to the `Thread` API that have nothing to do with As of Java 20, the `stop`, `suspend`, and `resume` methods throw an `UnsupportedOperationException` for both platform and virtual threads. These methods have been deprecated since Java 1.2 and deprecated for removal since Java 18. +  ## Capturing Task Results You often want to combine the results of multiple concurrent tasks: -``` +```java Future f1 = service.submit(callable1); Future f2 = service.submit(callable2); result = combine(f1.get(), f2.get()); @@ -121,7 +132,7 @@ result = combine(f1.get(), f2.get()); Before virtual threads, you might have felt bad about the blocking `get` calls. But now blocking is cheap. Here is a sample program with a more concrete example: -``` +```java import java.util.concurrent.*; import java.net.*; import java.net.http.*; @@ -153,7 +164,7 @@ public class VirtualThreadDemo { If you have a list of tasks with the same result type, you can use the `invokeAll` method and then call `get` on each `Future`: -``` +```java List> callables = ...; List results = new ArrayList<>(); for (Future f : service.invokeAll(callables)) @@ -162,7 +173,7 @@ for (Future f : service.invokeAll(callables)) Again, a more concrete sample program: -``` +```java import java.util.*; import java.util.concurrent.*; import java.net.*; @@ -198,6 +209,7 @@ public class VirtualThreadDemo { } ``` +  ## Rate Limiting Virtual threads improve application throughput since you can have many more concurrent tasks than with platform threads. That can put pressure on the services that the tasks invoke. For example, a web service may not tolerate huge numbers of concurrent requests. @@ -210,7 +222,7 @@ As an example, on my personal web site, I provide demo services for producing ra The following sample program shows rate limiting with a simple semaphore that allows a small number of concurrent requests. When the maximum is exceeded, the `acquire` method blocks, but that is ok. With virtual threads, blocking is cheap. -``` +```java import java.util.*; import java.util.concurrent.*; import java.net.*; @@ -253,6 +265,7 @@ public class RateLimitDemo { } ``` +  ## Pinning The virtual thread scheduler mounts virtual threads onto carrier threads. By default, there are as many carrier threads as there are CPU cores. You can tune that count with the `jdk.virtualThreadScheduler.parallelism` VM option. @@ -272,14 +285,14 @@ You always have the option whether or not to use virtual threads. In particular, To find out whether pinned threads are blocked, start the JVM with one of the options -``` +```shell -Djdk.tracePinnedThreads=short -Djdk.tracePinnedThreads=full ``` You get a stack trace that shows when a pinned thread blocks: -``` +```shell ... org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) <== monitors:1 ... @@ -296,7 +309,7 @@ The following sample program shows pinning in action. We launch a number of virt * use a `ReentrantLock` * don't use virtual threads -``` +```java import java.util.concurrent.*; import java.util.concurrent.locks.*; @@ -344,6 +357,7 @@ public class PinningDemo { } ``` +  ## Thread Locals A *thread-local variable* is an object whose `get` and `set` methods access a value that depends on the current thread. Why would you want such a thing instead of using a global or local variable? The classic application is a service that is not threadsafe, such as `SimpleDateFormat`, or that would suffer from contention, such as a random number generator. In these cases, a per-thread shared instance makes sense. @@ -356,6 +370,7 @@ An *inheritable thread-local variable* is a special kind of thread-local variabl Such mutation is uncommon. Run with the VM flag `jdk.traceVirtualThreadLocals` to get a stack trace when a virtual thread mutates a thread-local variable. +  ## Conclusion * Use virtual threads to increase throughput when you have many tasks that mostly block on network I/O From f8dc8393fb07edc9cc97950e798be8a3ebf7e2fd Mon Sep 17 00:00:00 2001 From: Cay Horstmann Date: Sat, 22 Jul 2023 08:15:21 +0200 Subject: [PATCH 5/6] virtual threads article ready for review --- .../02_new-features/00_virtual-threads.md | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md index 9cc9cca..c4eeda3 100644 --- a/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md +++ b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md @@ -36,7 +36,7 @@ JEP 425 introduced *virtual threads* in Java 19. Many virtual threads run on a p With virtual threads, blocking is cheap. When a result is not immediately available, you simply block in a virtual thread. You use familiar programming structures—branches, loops, try blocks—instead of a pipeline of callbacks. -Virtual threads are useful when the number of concurrent tasks is large, and the tasks mostly block on network I/O. They offer no benefit for CPU-intensive tasks. For such tasks, consider parallel streams or [recursive fork-join tasks](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/RecursiveTask.html). +Virtual threads are useful when the number of concurrent tasks is large, and the tasks mostly block on network I/O. They offer no benefit for CPU-intensive tasks. For such tasks, consider [parallel streams](https://dev.java/learn/api/streams/parallel-streams/) or [recursive fork-join tasks](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/RecursiveTask.html).   ## Creating Virtual Threads @@ -270,19 +270,17 @@ public class RateLimitDemo { The virtual thread scheduler mounts virtual threads onto carrier threads. By default, there are as many carrier threads as there are CPU cores. You can tune that count with the `jdk.virtualThreadScheduler.parallelism` VM option. -When a virtual thread executes a blocking operation, it is supposed to be unmounted from its its carrier thread, which can then execute a different virtual thread. There are two situations in which this does *not* happen: +When a virtual thread executes a blocking operation, it is supposed to be unmounted from its its carrier thread, which can then execute a different virtual thread. However, there are situations where this unmounting is not possible. In some situations, the virtual thread scheduler will compensate by starting another carrier thread. For example, in JDK 21, this happens for many file I/O operations, and when calling `Object.wait`. You can control the maximum number of carrier threads with the `jdk.virtualThreadScheduler.maxPoolSize` VM option. + +A thread is called *pinned* in either of the two following situations: 1. When executing a `synchronized` method or block 2. When calling a native method or foreign function -A thread that does either of these is called *pinned*. Being pinned is not bad in itself. But when a pinned thread blocks, there is a problem. The thread cannot be unmounted, and the carrier thread is blocked. That leaves fewer carrier threads for running virtual threads. - -In some situations, the virtual thread scheduler will compensate by starting another carrier thread. For example, in JDK 21, this happens for many file I/O operations, and when calling `Object.wait`. But this is an implementation detail that may change. You can control the maximum number of carrier threads with the `jdk.virtualThreadScheduler.maxPoolSize` VM option. +Being pinned is not bad in itself. But when a pinned thread blocks, it cannot be unmounted. The carrier thread is blocked, and, in Java 21, no additional carrier thread is started. That leaves fewer carrier threads for running virtual threads. Pinning is harmless if `synchronized` is used to avoid a race condition in an in-memory operation. However, if there are blocking calls, it would be best to replace `synchronized` with a `ReentrantLock`. This is of course only an option if you have control over the source code. -You always have the option whether or not to use virtual threads. In particular, heavy file I/O work may be better suited for platform threads. - To find out whether pinned threads are blocked, start the JVM with one of the options ```shell @@ -362,14 +360,14 @@ public class PinningDemo { A *thread-local variable* is an object whose `get` and `set` methods access a value that depends on the current thread. Why would you want such a thing instead of using a global or local variable? The classic application is a service that is not threadsafe, such as `SimpleDateFormat`, or that would suffer from contention, such as a random number generator. In these cases, a per-thread shared instance makes sense. -However, if the shared instance is resource-intensive, there can be a problem when migrating to virtual threads. There will likely be far more of them than threads in a thread pool, and now you have many more expensive instances. In such a situation, you should rethink your sharing strategy. - Another common use for thread locals is to provide “implicit” context, such as a database connection, that is properly configured for each task. Instead of passing the context from one method to another, the task code simply reads the thread-local variable. This is actually hard to get right with thread pools, since you don't want another thread use your database. But tasks are mapped 1:1 to virtual threads, so a thread-local context makes sense. An *inheritable thread-local variable* is a special kind of thread-local variable. When a new thread is started, it receives a copy of the parent's inheritable thread locals. This is just the behavior that you want for context information, but making these copies can be expensive. The copies are necessary because the child thread might mutate the thread-local values. Such mutation is uncommon. Run with the VM flag `jdk.traceVirtualThreadLocals` to get a stack trace when a virtual thread mutates a thread-local variable. +Resource-intensive thread locals can be a problem when migrating to virtual threads. There will likely be far more of them than threads in a thread pool, and now you have many more expensive instances. In such a situation, you should rethink your sharing strategy. +   ## Conclusion @@ -378,6 +376,3 @@ Such mutation is uncommon. Run with the VM flag `jdk.traceVirtualThreadLocals` t * Don't pool virtual threads; use other mechanisms for rate limiting * Check for pinning and mitigate if necessary * Minimize thread-local variables in virtual threads - - - From a1c02974d3f364572ab2ecd616ea0bf53a6a94e5 Mon Sep 17 00:00:00 2001 From: Cay Horstmann Date: Mon, 31 Jul 2023 07:49:24 +0200 Subject: [PATCH 6/6] Reworded thread locals section --- .../01_tutorial/02_new-features/00_virtual-threads.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md index c4eeda3..bd5cfca 100644 --- a/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md +++ b/app/pages/learn/01_tutorial/02_new-features/00_virtual-threads.md @@ -358,15 +358,13 @@ public class PinningDemo {   ## Thread Locals -A *thread-local variable* is an object whose `get` and `set` methods access a value that depends on the current thread. Why would you want such a thing instead of using a global or local variable? The classic application is a service that is not threadsafe, such as `SimpleDateFormat`, or that would suffer from contention, such as a random number generator. In these cases, a per-thread shared instance makes sense. +A *thread-local variable* is an object whose `get` and `set` methods access a value that depends on the current thread. Why would you want such a thing instead of using a global or local variable? The classic application is a service that is not threadsafe, such as `SimpleDateFormat`, or that would suffer from contention, such as a random number generator. Per-thread instances can perform better than a global instance that is protected by a lock. -Another common use for thread locals is to provide “implicit” context, such as a database connection, that is properly configured for each task. Instead of passing the context from one method to another, the task code simply reads the thread-local variable. This is actually hard to get right with thread pools, since you don't want another thread use your database. But tasks are mapped 1:1 to virtual threads, so a thread-local context makes sense. +Another common use for thread locals is to provide “implicit” context, such as a database connection, that is properly configured for each task. Instead of passing the context from one method to another, the task code simply reads the thread-local variable whenever it needs to access the database. -An *inheritable thread-local variable* is a special kind of thread-local variable. When a new thread is started, it receives a copy of the parent's inheritable thread locals. This is just the behavior that you want for context information, but making these copies can be expensive. The copies are necessary because the child thread might mutate the thread-local values. +Thread locals can be a problem when migrating to virtual threads. There will likely be far more virtual threads than threads in a thread pool, and now you have many more thread-local instances. In such a situation, you should rethink your sharing strategy. -Such mutation is uncommon. Run with the VM flag `jdk.traceVirtualThreadLocals` to get a stack trace when a virtual thread mutates a thread-local variable. - -Resource-intensive thread locals can be a problem when migrating to virtual threads. There will likely be far more of them than threads in a thread pool, and now you have many more expensive instances. In such a situation, you should rethink your sharing strategy. +To locate uses of thread locals in your app, run with the VM flag `jdk.traceVirtualThreadLocals`. You will get a stack trace when a virtual thread mutates a thread-local variable.   ## Conclusion