When ReadyNow Can Only Compile Under Traffic Loads

This is the fifth post in a series about faster Java application warmup with ReadyNow. If you haven’t been following this series, head back to the first post, Faster Java Warmup: CRaC with ReadyNow. That post explored how to address situations where traffic is loaded and redirected to your application before ReadyNow has finished compiling and optimizing the bytecode.

ReadyNow enables applications to compile bytecode to native code shortly after startup to optimally handle the load. It also optimizes the compiled code by removing unused paths, inlining, etc. But this compilation and optimization cannot be fully performed in certain situations before traffic is redirected to the application.

This post explores such a scenario and provides detailed solutions for development teams facing this challenge.

Compilation only occurs on traffic load

In some cases, ReadyNow does not achieve the expected warmup improvements, and many compilations are performed only when the application first loads. The most common reason is that traffic triggers the first execution of the code, and certain classes are only loaded and initialized at this time.

The first chart below shows a perfect example of the first set of compilations shortly after startup, and the spike in the compiler queue when traffic starts [Figure 1]. There is a period of time left before this event, during which the compiler has time to precompile the code and be ready when needed, but cannot fully execute this process.

The second chart shows an example where most of the compilation is handled after startup, and only a very small amount of compilation is required when traffic loads [Figure 2].

ReadyNow waits for class loading

In order to compile and optimize the bytecode to native code, ReadyNow needs to initialize the class. This happens when the class is “first accessed” and does things like calling static initializers (static {} blocks in the class) and initializing all static variables. Classes are loaded by the class loader in the JVM only when they are needed.

As a result, some code cannot be compiled in advance and can only be compiled when it is first needed by production load on the system. This prevents ReadyNow from achieving its main goal, which is to compile code before it is needed.

How to solve this problem?

Identify the problem

The solution is simple: create a warmup script that calls your static code before traffic starts. This loads the necessary classes so that ReadyNow can optimize the code right away. Figuring out which methods need to be called can be difficult, but logs and profiling tools can help.

A fully optimized system can be achieved before traffic starts through an iterative process of 2 to 3 iterations, with the following steps: 

1. To get the Zing garbage collector log, see the documentation.
2. Get the ReadyNow output files from a run, preferably from a production system with real load. This can be done by:
   • Use -XX:ProfileLogOut=<path> on the system itself.
   • When using the ReadyNow Orchestrator service of Optimizer Hub:
     • -XX:ProfileLogDumpOutputToFile=<name>, see the ReadyNow Orchestrator JVM options documentation.
     • If the JVM already writes logs for the service, this is done via the /vmInstances API.
3. Determines the exact time (at X seconds) that load is routed to the application.
4. Open the Garbage Collector log in GCLA (Add link) and load the ReadyNow profile on it using “Add File”.
5. Select the ReadyNow chart (e.g. First Call Events -> All Events) and examine the RAW data.
6. Search for “CLASS_INIT” or “FIRST_CALL” and filter for only those log entries.
7. Continue filtering log entries (for example, by organization name, then application name. For example: “Acme”, then “Payroll”).   Alternatively, you can filter out lambda, reflection, frameworks, and other non-application methods.
8. If necessary, return to step 5 and repeat.
9. Once you have a small enough list, check the methods starting at X seconds (as described in point #2 above) and find 1 or 2 methods to add to your startup script.

Call the identified code

Once you have determined the methods that will be called when traffic starts, you can create a script or automation to trigger the compilation of all the necessary code. Depending on your use case, this could be a script in Java, Bash, Python, etc. that contains mock requests, integration tests, or any other solution that triggers your application to call these methods.

Readiness check

At this point, we know which methods must be called and have a solution to execute them to trigger code compilation. The next step is to understand when compilation is complete so that we can send traffic to the application. To maximize the usability of our application, we want to minimize the time between the last compilation and the start of traffic.

In Zing 24.06.0.0, the MXBean extension has several new methods that can request several metrics from a running JVM to help you determine when your application is ready to handle traffic:

• getTotalOutstandingCompiles()
• getTotalPerformedTier1Compiles()
• getTotalPerformedTier2Compiles()

These methods return the total number of queued and in-progress compilations, and can return the total number of Tier 1 and Tier 2 compilations on request. Again, you can define thresholds to determine application readiness based on your use case.

For more information, see the Zing MXBeans documentation.

Spring Petclinic example

Let’s use the famous Spring Petclinic demo application to illustrate the problem and solution. With the help of the JMeter tests included in this project, we can simulate the load of the application and check the compilation queue using the garbage collector log file.

In this example, after the application starts, the JMeter test is started after a fixed time, and after the test is finished, after another waiting period, the test is executed again.

Run without a ReadyNow profile

In the first run, we did not provide a ReadyNow profile. This meant that the Zing runtime had no compilation information available and could not perform any compilation in advance. As you can see in the chart, at the start of the application, there was a small spike in the compiler queue, as the “base” classes were loaded. When the first JMeter test started, simulating the beginning of the application traffic load, there was a significant spike in the compiler queue number. When the second test executed, the spike was much smaller.

This shows that when we use the JMeter test as a “dummy” load, we can already prepare the application to handle traffic [Figure 3].

Run with the ReadyNow profile

While executing the above test, I instructed the application to generate a ReadyNow profile using -XX:ProfileLogOut=readynow.log and used it as input for the second test. As expected, we got the desired effect: a larger spike in the compiler queue immediately after the application started, which enabled most of the code to run in its native format thanks to ReadyNow. With the first JMeter test, we were able to trigger the compilation of the remaining methods as a warm-up step for improvement and to prepare the application for real traffic. As you can see, the number of compiler queues decreased further during the second test run, indicating that the application was well prepared to handle real traffic [Figure 4].

Generational ReadyNow profiles

As explained in a previous blog post, “How to Train ReadyNow for Optimal Java Performance,” ReadyNow can take a profile as input and output a new generation, providing more and better information for the next run. In the tests for this blog post, I did just that.

The chart you see below is the result of the third generation profile. Below that are the results of the first, second, and third profiles. As you can see, the initial compiler queue peaks right after startup and continues to grow as ReadyNow learns more about the code to be compiled [Figure 5]. More importantly, the spike gets smaller as traffic starts to arrive from the second run of the JMeter test.

Result

Because these “missing” classes are now loaded and initialized by the warmup script after startup, ReadyNow can perform its intended task from startup or before real traffic starts, and can compile all necessary native code up front. This moves compilation to the very beginning of the application, but it still takes some time to complete. The exact amount of time can be determined through testing and in conjunction with the Readiness Probe to allow the Falcon compiler in Zing or the Cloud Native Compiler in the Optimizer Hub to complete most optimizations.  

Based on experience from many customer production systems, it is generally not necessary to achieve 100% of Tier 2 optimizations at startup. The most important compilations will be completed first, and for most customers, it is acceptable to do subsequent optimizations after traffic starts.

Conclusion

Uninitialized classes can sometimes hinder the effectiveness of ReadyNow in reducing Java application warmup time. By understanding this limitation and identifying and proactively invoking static code, you can leverage the full power of ReadyNow to optimize application performance from the start.

By following these best practices and properly configuring ReadyNow and the application startup process, organizations can achieve the best balance between startup time and runtime performance, ensuring their Java applications are truly “ready now” when production traffic arrives.

To read the entire ReadyNow series, start at the beginning with the following blog posts:

Faster Java Warmup: CRaC vs. ReadyNow
How ReadyNow reduces Java warmup time
How to train ReadyNow for optimal Java performance
Use Optimizer Hub to speed up Java application startup and compilation
When ReadyNow can only compile under traffic load