Montag, 27. Mai 2019

Jakarta EE without javax: the world won't end this time either

If you missed the news, Oracle is donating the Java EE specification to the Eclipse foundation. This decisions has followed a rather long period of hibernation in the specification process where people rightfully suspected a loss of strategic interest in Java EE by Oracle. At first, the decision to donate the specification was well-met by the Java EE and broader Java community. Without Oracle slowing down the process, those involved in Java EE could once again attempt to close up to non-standardized APIs. Until today, the donation process is however incomplete due to Oracle and the Eclipse foundation being in disagreement about several details of the donation being made.

While overturning all intellectual property, Oracle was less generous when it comes to the use of its Java brand within the specification’s new home. Java EE does of course contain the name of the open-source, yet trademark-protected platform that is owned by Oracle. And this renders a problem in a legal context: if you grant a third party the use of your brand's name, you have yielded rights to restrict it in the future. To make things worse, you are potentially weakening your position in court when it comes to lawsuits regarding your brand. With the history of Oracle and Google arguing about Java licensing for years, one could therefore have anticipated that branding would be a difficult discussion point. And without pretending to know much about international trademark law, I was told by people more involved that "use it or lose it" is a good enough approximation for understanding the general motto of such disagreements. As a first result, the specification was therefore renamed from Java EE into Jakarta EE to avoid a clash of interests.

The real shocker for the newly formed Jakarta EE community was however yet to come. After months of discussing formalities of the donation, the Eclipse foundation learned that it could neither assume ownership of the current javax namespace that hosts the APIs that are defined within Java EE. Instead, this namespace is now planned to be hibernated for all donated APIs. Any new API that will be created within the specification process of Jakarta EE is therefore supposed to be hosted within a new namespace to avoid infringing Oracle's trademark.

At this point, it is important to clarify what this means. Jakarta EE and the Eclipse foundation are not prohibited of using the javax namespace or implementing its API. Neither will APIs that currently exist be deleted. But any API that is created or updated within the newly formed Jakarta EE specification process will need to live in a new namespace that is most likely imitating the one in existence but with jakarta as its prefix instead of javax. As an example, if a new method was to be added to the javax.servlet.Servlet interface, the next version of the servlet specification would need to publish a new interface named jakarta.servlet.Servlet instead of adding this method to the existing API.

I am not a Java EE user, why would I care?


Formally, the Java platform as most people know it is split into two parts. The first part being Java SE where all API is defined within packages that are prefixed with java. In addition to that, Java EE specifies extending APIs within the javax namespace. Those APIs do not imply a particular implementation but only define behavior that is implemented by different vendors of Java EE-compliant components.

In this context, Java EE is a blanket term for several API specifications that are not dependent on one another. As an example, the Java messaging specification (JMS) defines an API for interacting with message queues whereas the Java servlet specification defines an API for dispatching calls to a web server. In practice, no Java EE application runtime that I know of implements the entirety of APIs that are defined within the Java EE specification process. And some Java frameworks even focus on only implementing a single specification. As an example, the Jetty web server only implements the Java servlet specification. Therefore, if you are using Jetty via Spring Boot, you are formally a user of Java EE even if you are not interacting with the specification directly or consider yourself a Java EE user.

Despite this formal distinction, you have probably encountered Java EE and its javax namespace even if you only ever programmed vanilla Java without including any external dependencies. This is because of selected Java EE APIs being bundled with the standard image of the JVM. In addition to the API, the JVM also ships a default implementation of this API, to offer users the convenience of solving common tasks without any additional efforts. As an example, JAXP is a Java EE specification that defines an API for processing XML in Java. With XML-processing being such a common task, especially on the enterprise-oriented Java platform, its inclusion it was a reasonable choice. For JAXP, its assumed common usage is still factual today but other JVM-bundled Java EE specifications did not age equally well. As an example, SOAP messaging is no longer the first choice for most Java developers such that the JVM-bundled JAX-WS implementation has become dead weight for the majority of users. In order to cut down on the JVM footprint and with introducing the Java module system in Java 9, several Java EE APIs were moved to deprecated modules that are scheduled for removal in a future release. Of course, this does not imply that a module's API itself is deprecated. JAX-WS is still alive and actively used by many. But as a result of this module deprecation, JAX-WS needs to be added as an explicit dependency by those that want to continue using it in a future Java release.

In our age of running microservices on virtualized hardware, a reduced JVM footprint has become an obvious goal of evolving the JVM. But removing Java EE API from the base JVM image has another advantage. By asking users to include an explicit dependency on Java EE API, upgrading the Java runtime and Java EE are no longer tied together. Until Java 8, managing such version interdependencies has always been tedious. This is especially true if you do not control the exact version of the JVM that you are deploying your applications to. Until Java 8, the JVM only allowed you to override an implicit Java EE dependency by placing a jar file into the JVM's extension folder. But this is of course problematic when you are sharing a JVM installation with other Java processes that would also be affected. Also, you are still required to have some control over the JVM installation being used. To remedy this problem, the Java module system does no longer resolve the deprecated Java EE modules by default what makes it possible to include an explicit version with the JVM despite the continued on-demand bundling while also offering a simple way of activating legacy compatibility.

To make things even more complicated, a small subset of Java EE APIs grew into Java SE in a way that does not allow their simple separation. As an example, the JDBC specification is split into "client-side" and "server-side" requirements where the former bit formally belongs to Java SE whereas the latter is part of Java EE. This distinction comes from the original Java philosophy where one would use Java SE for user-facing desktop applications but Java EE for server applications that are used by multiple concurrent users. In this spirit, the JDBC Connection interface is for example defined in the java.sql package. After all, a desktop user might of course want to connect to a database. On the other side, the JDBC DataSource interface is defined in the javax.sql package as connection pooling was considered a requirement only for multi-threaded server applications. From today's perspective, this separation does of course no longer make much sense but the namespace and formal distinction remains even today.

Of course, it would not be meaningful to have the JDBC API evolve separately within both Java SE that is managed by the OpenJDK project and by Jakarta EE that is now managed by the Eclipse foundation. Therefore, not all parts of the Java EE specification were donated to Eclipse such that the javax.sql namespace will be retained for the JDBC API that is now considered to be a part of only Java SE. Other examples of such API retention are the JMX API which relies heavily on native JVM-support. And of course all other APIs that were always considered to be part of Java SE such as the Swing API that also ended up in the Java extension namespace will remain in their original packages.

What about backwards compatibility?


The important thing to keep in mind is that no currently existing javax API will disappear, neither today or in the future. Personally, I also expect the specifications that are now part of Jakarta EE to support the javax namespace for many years to come. As a matter of fact, dealing with multiple namespaces is nothing new to most Java EE implementations but has always been an important topic to deal with. The Hibernate library for example has already successfully completed a similar migration when gradually replacing its own annotations with those defined by the JPA specification. In another example, the Spring framework supports the Java EE CDI specification in parallel to its native annotations. Doing so, it is for example possible to request a bean injection by using either the javax.inject.Inject annotation or Spring's native Autowired annotation. Once the Inject annotation is transferred to the jakarta package, I would therefore expect the Spring framework to support both the Java EE and Jakarta EE namespace as I also expect it from other implementers of Java enterprise APIs.

With Jakarta EE being the successor of Java EE, I would neither expect this support to be overly costly to implement or maintain as application server vendors can simply implement Jakarta EE wrapper classes that delegate to the now outdated Java EE API. For example, a Java EE servlet can internally be treated as a Jakarta EE servlet by defining a wrapper class similar to the following:

public class LegacyServlet implements jakarta.servlet.Servlet {

  private final javax.servlet.Servlet delegate;

  public LegacyServlet(javax.servlet.Servlet delegate) {
    this.delegate = delegate;
  }

  @Override
  public void service(jakarta.servlet.ServletRequest req, jakarta.servlet.ServletResponse resp) {
 delegate.service(new LegacyServletRequest(req), new LegacyServletResponse(resp));
  }
}

This should be fairly simple if Jakarta EE targets (logical) backwards compatibility to the current specification and API. If this principle is adhered, this would also require users of the API to only update to the Jakarta namespace in case that they want to take new features into use what already requires code changes. I would therefore expect the changed namespace not to affect future Jakarta EE users too much but to mainly be the concern of those implementing its APIs. Looking back to other more fundamental changes to the Java platform in the past, this was also true when for example the Java module system was introduced which mostly concerned library and framework developers but rarely end users of Java.

Of course, support for both namespaces will never be universal, especially not in the long run and therefore users of Java EE APIs will need to react to the transition eventually. Given that the specification retains binary compatibility of its API with the exclusion of the change in namespace prefix, I do however believe that porting software should be easy to overcome and should even be automatable. Any Java class references its imported types in each class file's constant pool. It would be trivial for a tool to patch all relevant type references in all constant pools of an artifact with the new jakarta prefix. Doing so, legacy users of Java EE could avoid source code changes for their applications under passive maintenance and only apply such changes after compilation, or even patch an artifact during deployment.

What drives Oracle?


I am of course a software consultant and not an expert of international trademark jurisdiction. Neither do I have any insights into Oracle's decision making process. Therefore, please take this last section as an educated speculation mixed with my personal opinion rather than a summary of facts.

Several voices in the Java community are currently accusing Oracle of acting against the interest of Java and its users by restricting the use of the javax namespace. There have also been heated debates in the Eclipse foundation to the point where it was suggested that the donation of Java EE in this way might be rejected due to its incompatibility with the goals and values of the organization.

Given that this change does impose significant work on the users of Java, one can of course quickly arrive at this opinion. I can however not imagine that Oracle took this decision lightly. Oracle is and has been investing heavily in the Java platform – Java has rarely been as alive as it is today – but it has also changed its strategic orientation. To me, the idea that Oracle does “not care” about the Java community while making these investments simply does not fit.

So what do I think motivated this decision? To me, the restriction has little to do with Java EE but is about Oracle protecting its interest in Java SE. At the end of the day, Oracle is investing in Java to turn a profit. And by allowing the use of its trademark, Oracle would be giving up a share of the control of its brand what sets this goal into jeopardy. Of course, Oracle is relying on Java for its own products and maintains it also for this reason. But at the same time the company attempts to create a strategic model that justifies the funding of the many hundred full-time and highly qualified employees that work on Java.

Oracle is quite obviously pushing into selling cloud solutions and given the company's current dominance in runtimes and databases in addition to its deep pockets, I believe that their chances to gain significant market share in this field are better than many people anticipate. Another indication of Oracle planning with the success of the core platform are its investments into the Graal VM and its compiler which does of course also offer the Java language a much broader scope of application in resource-constrained environments as within containers.

But while investing in certain areas, Oracle is surely also looking into ways of cutting costs and terminates ventures that are no longer of strategic interest or not sufficiently profitable. And while it saddens users – including myself – that a successful project team like that of Java flight recorder is laid off, it makes sense given that the vast majority of Java developers are not asking for such tooling. I believe that Java EE did neither fit into Oracle’s plans or cost profile for the platform and suffered a similar faith.

In this light, Oracle probably considered a trade-off between giving up the specification versus donating it for other people to maintain. And while the choice to donate Java EE might seem to come without costs, Oracle is of course taking a risk by making this donation. By allowing competing organizations to continue their efforts in Java EE, these efforts might also strengthen their ability to compete against Oracle in the the Java SE segment. This is especially true with Red Head and IBM being among those organizations which are of course also competing in the market for cloud solutions. By protecting its branding rights, Oracle is simply aiming to reduce the risk that Java EE is weaponized by a competitor to fight for Java SE market share in the future. And to be fair, Oracle had offered a way for the Eclipse foundation to continue using the javax namespace. However, this would have required the foundation to restrict itsself to bundling its products with Java SE-certified implementations of the JVM rather than for example its own IBM-donated OpenJ9. Doing so, Oracle would have retained sufficient control of its brand despite Eclipse's use of the javax namespace but at the same time, it is very understandable that signing such a broad agreement was not in the interest of the foundation either. It just was not meant to be and in this light one could even argue that Eclipse was the wrong choice for receiving the donation of Java EE in the first place.

What comes next?


In the open source community, we discuss loud and often that our work is underfunded. And while the lacking profitability is a problem for individual developers hacking away their nights, it is of course also an issue for large corporations, be it Oracle or any other company involved in the current discussion. In my opinion, by Oracle donating the intellectual property of Java EE, the Java community was already handed over the most important value of the specification and we should focus on what we have and not get overly distracted by the strings attached. Personally, I’d worry more about the future of Java if Oracle had lost the interest in the Java brand instead of taking its stand.

As for Jakarta EE, I do neither think that the coming namespace migration is the biggest problem the specification is facing. Many developers have been frustrated with the dustiness of Java EE even before the recent stagnation period. In my opinion, the problem is part of the process. In reality, a Java EE specification is often derived from a leading framework’s implementation. If another framework wanted to reinvent how the same problem can be solved with a better API, it needs to face the constant criticism that it does not adhere to the standard. This all despite the fact that this standard was typically nothing but a snapshot of a previous best practice. For this reason, I hope that Jakarta EE can rather focus on its evolution instead of holding too much onto its past. With a compelling offering of state of the art API, I would not worry much about adjusting my code if it saved me from implementing iterations of minimally altered jakarta.servlet.Servlet API.

Sonntag, 5. August 2018

Hands on Java 11's constantdynamic

With the intention of making the JVM more appealing to dynamic languages, the seventh version of the platform had introduced invokedynamic to its instruction set. Java developers do not normally take note of this feature as it is hidden in Java byte code. In short, by using invokedynamic it has become possible to delay the binding of a method call until its first invocation. This technique is for example used by the Java language to implement lambda expressions which are only manifested on demand upon their first use. Doing so, invokedynamic has evolved into an essential language feature which I have described in detail in a previous blog posting. With constantdynamic a similar mechanism was introduced to Java 11, only that it delays the creation of a constant value. This posting describes the purpose and the inner workings of this feature and shows how to generate code that makes use of this new instruction using the Byte Buddy library.

What are constant values in Java?


Before Java 5, constant values in a Java programs could only be strings or of a primitive type. Those constants were built into the language as literals and are even assumed by the javac compiler to reduce the size of a class file. For example, in the following code snippet the value of the only field is never actually read but instead copied to its use site during compilation:

class ConstantSample {
  final String field = “foo”;
  void hello() {
    System.out.print(field);
  }
}

Instead of reading the field within the hello method, the generated byte code will contain a direct reference to the constant value foo. As a matter of fact, the above class will not ever attempt to read the field’s value what can be validated by altering it using Java reflection after which invoking hello would still print foo.

To represent such constant values, any Java class file contains a constant pool which can be thought of as a table that writes out any constant values that exist within the scope of a class. This implies constants that are used within methods or as field values but also other immutable information that describes a class such as the class’s name or names of invoked methods and their declaring type names. Once a value is recorded in the class’s constant pool, values can be referenced by an offset pointing to a specific entry within the constant pool. Doing so, values that are repeated throughout a class only need to be stored once because an offset can of course be referenced multiple times.

Therefore, when the field is read in the above source code, javac emits a byte code that refers to the offset of the value foo in the constant pool instead of emitting a read instruction for the field. This can be done as the field is declared final where javac ignores the edge-case of a reflective value change. By emitting an instruction to read a constant, javac also saves some bytes compared to an instruction for a field read. This is what makes this optimization lucrative, especially since string and numeric values are fairly commonly in any Java class. Smaller class files help the Java runtime to load classes quicker and an explicit notion of constantness helps the the JVM’s JIT and AOT compilers to apply further optimizations.

The described reuse of offsets for the same constant also implies an identity of reused values. As a consequence of representing an equal string value by a single instance the following statement will assert true in Java:

assert “foo” == “foo”;

Under the hood, both values of foo point to the same constant pool offset in the defining class’s constant pool. Additionally, the JVM even deduplicates constant strings across classes by interning strings that are found in constant pools.

Limitations of constant pool storage


Such tabular representation of values within a class file’s constant pool works well for simple values such as strings and numeric primitives. But at the same time, it can have non-intuitive consequences when javac is not discovering a value as being constant. For example, in the following class the only field’s value is not treated as a constant within the hello method:

class NoConstantSample {
  final String field = “foo”.toString();
  void hello() {
    System.out.print(field);
  }
}

While the toString method is trivial for strings, this circumstance remains unknown to javac which does not evaluate Java methods. Therefore, the compiler can no longer emit a constant pool value as the input to the print statement. Instead, it must emit a field read instruction of the field which requires additional bytes as it was mentioned before. This time, if the field’s value was changed by using reflection, invoking hello would therefore also print the updated value.

Of course, this example is contrived. But it is not difficult to imagine how limiting the classical approach to constants in Java plays out in practice. For example, imagine an integer value that is defined as Math.max(CONST_A, CONST_B). Of course, the maximum of two compile-time constants would itself be constant. Yet, due to javac’s inability of evaluating Java methods, the derived value is not discovered as a constant but only computed at runtime.

Another problem of declaring constant values in a class file’s constant pool, is its limitation to simple values. Strings and numerical values are of course trivial to represent, but more complex Java objects require more flexibility than the classical approach. To support additional constants, the Java class file format already added class literal constants in Java 5 where values such as String.class would no longer be compiled to a call to Class.forName("java.lang.String") but to a constant pool entry containing a class reference. And also the Java 7 release added new constant pool types to the class file specification to allow for a constant representation of MethodType and MethodHandle instances.

In contrast to strings, classes and primitive values, the Java programming language does however not offer a literal for creating those latter constants. Rather, the possibility for such constants was added to better support invokedynamic instructions where javac required an efficient way of representation. In essence, a lambda expression is described by the lambda’s expressions type signature - a MethodType - and a reference to its implementation - a MethodHandle. If both values had to be created as explicit, non-constant arguments for each call to a lambda expression, the performance overhead of using such expressions would surely have outweighed their benefit.

While this solution eased some intermediate pain, it implied a dissatisfying perspective onto Java’s future with regards of adding further constant types. A constant pool entry’s type is encoded by a single byte what severely limits the total number of possible constant types in a class file. As an additional hassle, changes to the class file format require a cascading adjustment of any tool that processes class files what makes a more generic approach for expressing constant values desirable. By introducing constantdynamic, such a mechanism is finally supported by the Java virtual machine with the upcoming release of Java 11.

Introducing dynamic constants


A dynamic constant is not created by processing a literal expression but by invoking a so-called bootstrap method that produces the constant value as its result. This is fairly similar to the invokedynamic instruction that binds method call sites by invoking a bootstrap method during runtime where a pointer to a target implementation for the dynamically bound call site is returned. As key difference, a bootstrapped constant is however immutable whereas dynamically bound method calls can be redirected to another implementation at a later point.

In essence, bootstrap methods are nothing more but Java methods with some requirements to their signature. As a first argument, any bootstrapping method receives a MethodHandles.Lookup instance that is automatically provided by the JVM. Such lookups give access with the privileges of the class that a particular instance of the class represents. For example, when MethodHandles.lookup() is invoked from any class, the caller-sensitive method returns an instance that for example allows for reading private fields of the calling class what would not be possible for a lookup instance that was created from within another class. In the case of a bootstrap method, the lookup represents the class that defines the dynamic constant under creation rather then the class that is declaring the boostrap method. Doing so, the bootstrap methods can access the same information as if the constant was created from within the constant-defining class itself. As a second argument, the bootstrap method receives the constant’s name and as a third argument, it receives the constants expected type. A bootstrap method must be static or a constructor where the constructed value represents the constant.

In many cases, none of these three arguments are required for implementing a bootstrap method but their existence allows for the implementation of more generic bootstrapping mechanisms to facilitate allow for the reuse of bootstrap methods for the creation of multiple constants. If desired, the last two arguments can also be omitted when declaring a bootstrap method. Declaring a MethodHandles.Lookup type as the first parameter is however required. This is done to allow potentially allowing further invocation modes in the future where the first parameter serves as a marker type. This is another difference to invokedynamic which allows allows for the omission of the first parameter.

With this knowledge, we can now express the previous maximum of two constants that was previously mentioned as a derived constant. The value is computed trivially by the following bootstrap method:

public class Bootstrapper {
  public static int bootstrap(MethodHandles.Lookup lookup, String name, Class type) {
    return Math.max(CONST_A, CONST_B);
  }
}

Since the lookup instance that is the first argument comes with the privileges of the class that defines the constant, it would also be possible to acquire the values of CONST_A and CONST_B by using this lookup, even if they were not normally visible to the bootstrap method, for example because they were private. The class’s javadoc explains in detail what API needs to be used for locating a field and to read their values.

In order to create a dynamic constant, a bootstrap method must be referenced within a class’s constant pool as an entry of type dynamic constant. As of today, the Java language has no way of creating such an entry and to my knowledge no other language is currently making use of this mechanism either. For this reason, we will look into creating such classes using the code generation library Byte Buddy later in this article. In Java pseudo code which hints constant pool values in comments, a dynamic constant and its bootstrap method would however be referred to as follows:

class DynamicConstant {
  // constant pool #1 = 10
  // constant pool #2 = 20
  // constant pool #3 = constantdyamic:Bootstrapper.bootstrap/maximum/int.class
  final int CONST_A = [constant #1], CONST_B = [constant #2];
  void hello() {
    System.out.print([constant #3]);
  }
}

Once the hello method is executed for the first time, the JVM would resolve the specified constant by invoking the Bootstrapper.bootstrap method with maximum as the constant name and int.class as the requested type for the created constant. After receiving a result from the bootstrap method, the JVM would then substitute any reference to the constant with this result and never invoke the bootstrap method again. This would also be true if the dynamic constant was referenced at multiple sites.

Avoiding custom bootstrap methods


For most cases, creating a dynamic constant does not require the implementation of an individual bootstrap method. To cover the majority of use cases, the JVM-bundled class java.lang.invoke.ConstantBootstraps already implements several generic bootstrap methods that can be used for the creation of most constants. As center piece, the class’s invoke method allows to define a constant by providing a method reference as a factory for a constant value. To make such a generic approach work, bootstrap methods are capable of receiving any number of additional arguments which must themselves be constant values. Those arguments are then included as references to other constant pool entries while describing the entry of the dynamic constant.

Doing so, the above maximum can rather be computed by providing a handle to the Math.max method and the two constant values of CONST_A and CONST_B as additional arguments. The implementation of the invoke method in ConstantBootstraps will then invoke Math.max using the two values and return the result where the bootstrap method is roughly implemented as follows:

class ConstantBootstraps {
  static Object invoke(MethodHandles.Lookup lookup, String name, Class type,
          MethodHandle handle, Object[] arguments) throws Throwable {
    return handle.invokeWithArguments(arguments);
  }
}

When additional arguments are provided to a bootstrap method, they are assigned in their order to every additional method parameter. To allow for more flexible bootstrap methods such as the invoke method above, the last parameter can also be of an Object array type to receive any excess arguments, in this case the two integer values. If a bootstrap method does not accept a provided argument, the JVM will not invoke the bootstrap method but throw a BootstrapMethodError during the failed constant resolution.

Using this approach, the pseudo code to using ConstantBootstraps.invoke would no longer required an individual bootstrap method and rather look as in the following pseudo code:

class AlternativeDynamicConstant {
  // constant pool #1 = 10
  // constant pool #2 = 20
  // constant pool #3 = MethodHandle:Math.max(int,int)
  // constant pool #4 = constantdyamic:ConstantBootstraps.invoke/maximum/int.class/#3,#1,#2
  final int CONST_A = [constant #1], CONST_B = [constant #2];
  void hello() {
    System.out.print([constant #4]);
  }
}

Nested dynamic constants


As mentioned, the arguments of a bootstrap method are required to be other constant pool entries. With dynamic constants being stored in the constant pool, this allows for nesting dynamic constants what makes this feature even more flexible. This comes with the intuitive limitation that the initialization of dynamic constants must not contain a circles. For example, the following bootstrap methods would be called from top to bottom if the Qux value was resolved:

static Foo boostrapFoo(MethodHandles.Lookup lookup, String name, Class type) {
  return new Foo();
}

static Bar boostrapBar(MethodHandles.Lookup lookup, String name, Class type, Foo foo) {
  return new Bar(foo);
}

static Qux boostrapQux(MethodHandles.Lookup lookup, String name, Class type, Bar bar) {
  return new Qux(bar);
}

When the JVM is required to resolve the dynamic constant for Qux, it would first resolve Bar what would again trigger a previous initialization of Foo as each value depends on the previous.

Nesting dynamic constants can also be required when expressing values that are not supported by static constant pool entry types such as a null reference. Before Java 11, a null value could only be expressed as a byte code instruction but not as a constant pool value where the byte code neither implied a type for null. To overcome this limitation, java.lang.invoke.ConstantBootstraps offers several convenience methods such as nullValue that allows bootstrapping a typed null value as a dynamic constant instead. This null value can then be supplied as an argument to another bootstrap method this method expected null as an argument. Similarly, it is not possible to express a primitive type literal such as int.class in the constant pool which can only represent reference types. Instead, javac translates for example int.class to a read of the static Integer.TYPE field which resolves its value of int.class on startup by a native call into the JVM. Again, ConstantBootstraps offers the primitiveType bootstrap method to represent such values easily as dynamic constants instead.

Why should one care about constant values?


All of the above might sound like a technical finesse that does not add much to the Java platform beyond what static fields already provide. However, the potential of dynamic constants is large but still unexplored. As the most obvious use case, dynamic constants can be used to properly implement lazy values. Lazy values are typically used to represent expensive objects only on-demand when they are used. As of today, lazy values are often implemented by using so-called double checked locking, a pattern that is for example implemented by the scalac compiler for its lazy keyword:

class LazyValue {
  volatile ExpensiveValue value;
  void get() {
    T value = this.value;
    if (value == null) {
      synchronized (this) {
        value = this.value;
          if (value == null) {
            value = new ExpensiveValue();
          }
       }
     }
     return value;
  }
}

The above construct requires a volatile read on every read despite the fact that the value never changes once it is initialized. This implies an unnecessary overhead which can be avoided by expressing the lazy value as a dynamic constant that is only bootstrapped if it is ever used. Especially in the Java core libraries this can be useful for delaying the initialization of many values that are never used, for example in the Locale class which initializes values for any supported language despite the fact that most JVMs only ever use the running machines standard language. By avoiding the initialization of such excess values, the JVM can boot up quicker and avoid using memory for dead values.

Another important use case is the availability of constant expressions to optimizing compilers. It is easy to imagine why compilers prefer processing constant values over mutable values. For example, if a compiler can combine two constants, the result of this combination can permanently replace the previous values. This would of course not be possible if the original values could change over time. And while a just-in-time compiler might still assume that mutable values are factually constant at runtime, an ahead-of-time compiler is dependent on some explicit notion of constantness. By assuring that bootstrap methods are side-effect free, future Java version could for example allow for their compile-time evaluation where constantdynamic could serve as a light-weight macro mechanism to widen the scope of native images written in Java using Graal.

Will I ever work with this feature?


When invokedynamic was introduced in Java 7, this new byte code feature was unused from the perspective of the Java language. However, as of Java 8 invokedynamic instructions can be found in most class files as an implementation of lambda expressions. Similarly, Java 11 does not yet use the constantdynamic feature but one can expect that this will change in the future.

During the latest JVMLS several potential APIs for exposing constantdynamic were already discussed (which would also make invokedynamic accessible via an API). This would be especially useful for library authors to allows them to better resolve critical execution paths but could also unlock some potential to improve javac’s constant detection, for example to widen the scope of non-capturing lambda expressions where field or variable access could be substituted by reading a constant value if a constant value was discovered during compilation. Finally, this new mechanism offers potential for future language enhancements such as a lazy keyword that avoids the overhead of the current equivalents in alternative JVM languages.

The constantdynamic feature can also be useful to Java agents that often need to enhance existing classes with additional information. Java agents cannot normally alter a classes by for example adding static fields as this can both interfere with reflection-based frameworks and since class format changes are forbidden on most JVMs when redefining an already loaded class. Neither restriction does however apply to dynamic constants that are added during runtime where a Java agent can now easily tag classes with additional information.

Creating dynamic constants using Byte Buddy


Despite the lack of language support for constantdynamic, JVMs of version 11 are already fully capable of processing class files that contain dynamic constants. Using the byte code generation library Byte Buddy, we can create such class files and load them into an early access build of the JVM.

In Byte Buddy, dynamic constants are represented by instances of JavaConstant.Dynamic. For convenience, Byte Buddy offers factories for any bootstrap method that is declared by the java.lang.invoke.ConstantBoostraps class such as the invoke method that was discussed previously.

For an easy example, the following code creates a subclass of Callable and defines the return value of the call method as a dynamic constant of the sample class. To bootstrap the constant, we are supplying the constructor of Sample to the mentioned invoke method:

public class Sample {
  public static void main(String[] args) throws Throwable {
    Constructor<? extends Callabable<?>> loaded = new ByteBuddy()
      .subclass(Callable.class)
      .method(ElementMatchers.named("call"))
      .intercept(FixedValue.value(JavaConstant.Dynamic.ofInvocation(Sample.class.getConstructor())))
    .make()
    .load(Sample.class.getClassLoader())
    .getLoaded()
    .getConstructor();

    Callable<?> first = loaded.newInstance(), second = loaded.newInstance();
    System.out.println("Callable instances created");
    System.out.println(first.call() == second.call());
  }
  
  public Sample() { 
    System.out.println("Sample instance created"); 
  }
}

If you run the code, note how only one instance of Sample is created as it was explained in this article. Also note how the instance is only created lazily upon the first invocation of the call method and after the creation of the Callable instances.

To run the above code, you currently have to run Byte Buddy with -Dnet.bytebuddy.experimental=true to unlock support for this feature. This changes once Java 11 is finalized and ready for release where Byte Buddy 1.9.0 will be the first version to support Java 11 out of the box. Also, there are still some rough edges in the latest Byte Buddy release when dealing with dynamic constants. Therefore, it is best to build Byte Buddy from the master branch or to use JitPack. To find more about Byte Buddy, visit bytebuddy.net.

Sonntag, 8. April 2018

JDK 11 and proxies in a world past sun.misc.Unsafe

With JDK 11 the first methods of sun.misc.Unsafe are retired. Among them, the defineClass method was removed. This method has been commonly used by code generation frameworks to define new classes in existing class loaders. While this method was convenient to use, its existence also rendered the JVM inherently unsafe, just as the name of its defining class suggests. By allowing a class to be defined in any class loader and package, it became possible to gain package-scoped access to any package by defining a class within it, thus breaching the boundaries of an otherwise encapsulated package or module.

With the goal of removing sun.misc.Unsafe, the OpenJDK started offering an alternative for defining classes at runtime. Since version 9, the MethodHandles.Lookup class offers a method defineClass similar to the unsafe version. However, the class definition is only permitted for a class that resides in the same package as the lookup's hosting class. As a module can only resolve lookups for packages that are owned by a module or that are opened to it, classes can no longer be injected into packages that did not intend to give such access.

Using method handle lookups, a class foo.Qux can be defined during runtime as follows:
MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodHandles.Lookup privateLookup = MethodHandles.privateLookupIn(foo.Bar.class, lookup);
byte[] fooQuxClassFile = createClassFileForFooQuxClass();
privateLookup.defineClass(fooQuxClassFile);

In order to perform a class definition, an instance of MethodHandles.Lookup is required which can be retrieved by invoking the MethodHandles::lookup method. Invoking the latter method is call site sensitive; the returned instance will therefore represent the privileges of the class and package from within the method is invoked. To define a class in another package then the current one, a class from this package is required to resolve against it using MethodHandles::privateLookupIn. This will only be possible if this target class's package resides in the same module as the original lookup class or if this package is explicitly opened to the lookup class's module. If those requirements are not met, attempting to resolving the private lookup throws an IllegalAccessException, protecting the boundaries that are implied by the JPMS.

Of course, code generation libraries are also constrained by this limitation. Otherwise they could be used to create and inject malicious code. And since the creation of method handles is call-site sensitive, it is not possible to incorporate the new class definition mechanism without requiring users to do some additional work by providing an appropriate lookup instance that represents the privileges of their module.

When using Byte Buddy, the required changes are fortunately minimal. The library defines classes using a ClassDefinitionStrategy which is responsible for the loading a class from its binary format. Prior to Java 11, a class could be defined using reflection or sun.misc.Unsafe using ClassDefinitionStrategy.Default.INJECTION. To support Java 11, this strategy needs to be replaced by ClassDefinitionStrategy.UsingLookup.of(lookup) where the provided lookup must have access to the package in which a classes would reside.

 

Migrating cglib proxies to Byte Buddy


As of today, other code generation libraries do not provide such a mechanism and it is uncertain when and if such capabilities are added. Especially for cglib, API changes have proven problematic in the past due to the libraries old age and widespread use in legacy applications that are no longer updated and would not adopt modifications. For users that want to adopt Byte Buddy as a more modern and actively developed alternative, the following segment will therefore describe a possible migration.

As an example, we generate a proxy for the following sample class with a single method:
public class SampleClass {
  public String test() { 
    return "foo"; 
  }
}

To create a proxy, the proxied class is normally subclassed where all methods are overridden to dispatch the interception logic. Doing so, we append a value bar to the return value of the original implementation as an example.

A cglib proxy is typically defined using the Enhancer class in combination with an MethodInterceptor. A method interceptor supplies the proxied instance, the proxied method and its arguments. Finally, it also provides an instance of MethodProxy which allows to invoke the original code.
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(SampleClass.class);
enhancer.setCallback(new MethodInterceptor() {
  @Override
  public Object intercept(Object obj, Method method, Object[] args, MethodProxy proxy) {
    return proxy.invokeSuper(obj, method, args) + "bar";
  }
});
SampleClass proxy = (SampleClass) enhancer.create();
assertEquals("foobar", proxy.test());

Note that the above code will cause a problem if any other method such as hashCode, equals or toString was invoked on the proxy instance. The first two methods would also be dispatched by the interceptor and therefore cause a class cast exception when cglib attemted to return the string-typed return value. In contrast, the toString method would work but return an unexpected result as the original implementation was prefixed to bar as a return value.

In Byte Buddy, proxies are not a dedicated concept but can be defined using the library’s generic code generation DSL. For an approach that is the most similar to cglib, using a MethodDelegation offers the easiest migration path. Such a delegation targets a user-defined interceptor class to which method calls are dispatched:
public class SampleClassInterceptor {
  public static String intercept(@SuperCall Callable<String> zuper) throws Exception {
    return zuper.call() + "bar";
  }
}

The above interceptor first invokes the original code via a helper instance that is provided by Byte Buddy on demand. A delegation to this interceptor is implemented using Byte Buddy’s code generation DSL as follows:
SampleClass proxy = new ByteBuddy()
  .subclass(SampleClass.class)
  .method(ElementMatchers.named("test"))
  .intercept(MethodDelegation.to(SampleClassInterceptor.class))
  .make()
  .load(someClassLoader, ClassLoadingStrategy.UsingLookup.of(MethodHandles
      .privateLookupIn(SampleClass.class, MethodHandles.lookup()))
  .getLoaded()
  .getDeclaredConstructor()
  .newInstance();
assertEquals("foobar", proxy.test());

Other than cglib, Byte Buddy requires to specify a method filter using an ElementMatcher. While filtering is perfectly possible in cglib, it is quite cumbersome and not explicitly required and therefore easily forgotten. In Byte Buddy, all methods can still be intercepted using the ElementMatchers.any() matcher but by requiring to specify such a matcher, users are hopefully reminded to make a meaningful choice.

With the above matcher, any time a method named test is invoked, the call will be delegated to the specified interceptor using a method delegation as discussed.

The interceptor that was introduced would however fail to dispatch methods that do not return a string instance. As a matter of fact, the proxy creation would yield an exception issued by Byte Buddy. It is however perfectly possible to define a more generic interceptor that can be applied to any method similar to the one offered by cglib's MethodInterceptor:
public class SampleClassInterceptor {
  @RuntimeType
  public static Object intercept(
      @Origin Method method,
      @This Object self,
      @AllArguments Object[] args,
      @SuperCall Callable<String> zuper
  ) throws Exception {
    return zuper.call() + "bar";
  }
}

Of course, since the additional arguments of the interceptor are not used in this case, they can be omitted what renders the proxy more efficient. Byte Buddy will only provide arguments on demand and if they are actually required.

As the above proxy is stateless, the interception method is defined to be static. Again, this is an easy optimization as Byte Buddy otherwise needs to define a field in the proxy class that holds a reference to the interceptor instance. If an instance is however required, a delegation can be directed to a member method of an instance using MethodDelegation.to(new SampleClassInterceptor()).

 

Caching proxy classes for performance


When using Byte Buddy, proxy classes are not automatically cached. This means that a new class is generated and loaded every time the above code is run. With code generation and class definition being expensive operations, this is of course inefficient and should be avoided if proxy classes can be reused. In cglib, a previously generated class is returned if the input is identical for two enhancements what is typically true when running the same code segment twice. This approach is however rather error prone and often inefficient since a cache key can normally be calculated much easier. With Byte Buddy, a dedicated caching library can be used instead, if such a library already is available. Alternatively, Byte Buddy also offers a TypeCache that implements a simple cache for classes by a user-defined cache key. For example, the above class generation can be cached using the base class as a key using the following code:
TypeCache<Class<?>> typeCache = new TypeCache<>(TypeCache.Sort.SOFT);
Class<?> proxyType = typeCache.findOrInsert(classLoader, SampleClass.class, () -> new ByteBuddy()
  .subclass(SampleClass.class)
  .method(ElementMatchers.named("test"))
  .intercept(MethodDelegation.to(SampleClassInterceptor.class))
  .make()
  .load(someClassLoader, ClassLoadingStrategy.UsingLookup.of(MethodHandles
      .privateLookupIn(SampleClass.class, MethodHandles.lookup()))
  .getLoaded()
});

Unfortunately, caching classes in Java brings some caveats. If a proxy is created, it does of course subclass the class it proxies what makes this base class ineligible for garbage collection. Therefore, if the proxy class was referenced strongly, the key would also be referenced strongly. This would render the cache useless and open for memory leaks. Therefore, the proxy class must be referenced softly or weakly what is specified by the constructor argument. In the future, this problem might be resolved if Java introduced ephemerons as a reference type. At the same time, if garbage collection of proxy classes is not an issue, a ConcurrentMap can be used to compute a value on absence.

 

Broaden the usability of proxy classes


To embrace reuse of proxy classes, it is often meaningful to refactor proxy classes to be stateless and to rather isolate state into an instance field. This field can then be accessed during the interception using the mentioned dependency injection mechanism, for example, to make the suffix value configurable per proxy instance:
public class SampleClassInterceptor {
  public static String intercept(@SuperCall Callable<String> zuper, 
        @FieldValue("qux") String suffix) throws Exception {
    return zuper.call() + suffix;
  }
}

The above interceptor now receives the value of a field qux as a second argument which can be declared using Byte Buddy’s type creation DSL:
TypeCache<Class<?>> typeCache = new TypeCache<>(TypeCache.Sort.SOFT);
Class<?> proxyType = typeCache.findOrInsert(classLoader, SampleClass.class, () -> new ByteBuddy()
    .subclass(SampleClass.class)
    .defineField(“qux”, String.class, Visibility.PUBLIC)
    .method(ElementMatchers.named("test"))
    .intercept(MethodDelegation.to(SampleClassInterceptor.class))
    .make()
    .load(someClassLoader, ClassLoadingStrategy.UsingLookup.of(MethodHandles
        .privateLookupIn(SampleClass.class, MethodHandles.lookup()))
    .getLoaded()
});

The field value can now be set on an every instance after its creation using Java reflection. In order to avoid reflection, the DSL can also be used to implement some interface that declares a setter method for the mentioned field which can be implemented using Byte Buddy's FieldAccessor implementation.

 

Weighting Proxy runtime and creation performance


Finally, when creating proxies using Byte Buddy, some performance considerations need to be made. When generating code, there exists a trade-off between the performance of the code generation itself and the runtime performance of the generated code. Byte Buddy typically aims for creating code that runs as efficiently as possible what might require additional time for the creation of such code compared to cglib or other proxing libraries. This bases on the assumption that most applications run for a long time but only creates proxies a single time what does however not hold for all types of applications.

As an important difference to cglib, Byte Buddy generates a dedicated super call delegate per method that is intercepted rather then a single MethodProxy. These additional classes take more time to create and load but having these classes available results in better runtime performance for each method execution. If a proxied method is invoked in a loop, this difference can quickly be crucial. If runtime performance is however not a primary goal and it is more important that the proxy classes are created in short time, the following approach avoids the creating of additional classes altogether:
public class SampleClassInterceptor {
  public static String intercept(@SuperMethod Method zuper, 
        @This Object target, 
        @AllArguments Object[] arguments) throws Exception {
    return zuper.invoke(target, arguments) + "bar";
  }
}

 

Proxies in a modularized environment


Using the simple form of dependency-injection for interceptors rather then relying on a library-specific type such as cglib's MethodInterceptor, Byte Buddy facilitates another advantage in a modularized environment: since the generated proxy class will reference the interceptor class directly rather then referencing a library specific dispatcher type such as cglib's MethodInterceptor, the proxied class’s module does not need to read Byte Buddy’s module. With cglib, the proxied class module must read cglib's module which defines the MethodInterceptor interface rather then the module that implements such an interface. This will most likely be non-intuitive for users of a library that uses cglib as a transitive dependency, especially if the latter dependency is treated as an implementation detail that should not be exposed.

In some cases, it might not even be possible or desirable that the proxied class's module reads the module of the framework which supplies the interceptor. For this case, Byte Buddy also offers a solution to avoid such a dependency altogether by using its Advice component. This component works on code templates such as that in the following example:
public class SampleClassAdvice {
  @Advice.OnMethodExit
  public static void intercept(@Advice.Returned(readOnly = false) String returned) {
    returned += "bar";
  }
}

The above code might not appear to make much sense as it stands and as a matter of fact, it will never be executed. The class merely serves as a byte code template to Byte Buddy which reads the byte code of the annotated method which is then inlined into the generated proxy class. To do so, every parameter of the above method must be annotated to represent a value of the proxied method. In the above case, the annotation defines the parameter to define the method’s return value to which bar is appended as a suffix given the template. Given this advice class, a proxy class could be defined as follows:
new ByteBuddy()
  .subclass(SampleClass.class)
  .defineField(“qux”, String.class, Visibility.PUBLIC)
  .method(ElementMatchers.named(“test”))
  .intercept(Advice.to(SampleClassAdvice.class).wrap(SuperMethodCall.INSTANCE))
  .make()

By wrapping the advice around a SuperMethodCall, the above advice code will be inlined after the call to the overridden method has been made. To inline code before the original method call, the OnMethodEnter annotation can be used.

Supporting proxies on Java versions prior to 9 and past 10


When developing applications for the JVM, one can normally rely on applications that run on a particular version to also run on later versions. This has been true for a long time, even if internal API has been used. However, as a consequence of removing this internal API, this is no longer true as of Java 11 where code generation libraries that have relied on sun.misc.Unsafe will no longer work. At the same time, class definition via MethodHandles.Lookup is not available to JVMs prior to version 9.

As for Byte Buddy, it is the responsibility of a user to use a class loading strategy that is compatible with the current JVM. To support all JVMs, the following selection needs to be made:
ClassLoadingStrategy<ClassLoader> strategy;
if (ClassInjector.UsingLookup.isAvailable()) {
  Class<?> methodHandles = Class.forName("java.lang.invoke.MethodHandles");
  Object lookup = methodHandles.getMethod("lookup").invoke(null);
  Method privateLookupIn = methodHandles.getMethod("privateLookupIn", 
      Class.class, 
      Class.forName("java.lang.invoke.MethodHandles$Lookup"));
  Object privateLookup = privateLookupIn.invoke(null, targetClass, lookup);
  strategy = ClassLoadingStrategy.UsingLookup.of(privateLookup);
} else if (ClassInjector.UsingReflection.isAvailable()) {
  strategy = ClassLoadingStrateg.Default.INJECTION;
} else {
  throw new IllegalStateException(“No code generation strategy available”);
}

The above code uses reflection to resolve a method handle lookup and to resolve it. Doing so, the code can be compiled and loaded on JDKs prior to Java 9. Unfortunately, Byte Buddy cannot implement this code as a convenience since MethodHandles::lookup is call site sensitive such that the above must be defined in a class that resides in the user’s module and not within Byte Buddy.

Finally, it is worth considering to avoid class injection altogether. A proxy class can also be defined in a class loader of its own using the ClassLoadingStrategy.Default.WRAPPER strategy. This strategy is not using any internal API and will work on any JVM version. However, one must keep in mind the performance costs of creating a dedicated class loader. And finally, even if the package name of the proxy class is equal to the proxied class, by defining the proxy in a different class loaders, their runtime packages will no longer be considered as equal by the JVM thus not allowing to override any package-private methods.

Final thoughts


On a final note I want to express my opinion that retiring sun.misc.Unsafe is an important step towards a safer, modularized JVM despite the costs of this migration. Until this very powerful class is removed, any boundaries set by the JPMS can be circumvented by using the privileged access that sun.misc.Unsafe still offers. Without this removal, the JPMS costs all the inconvenience of additional encapsulation without the benefit of being able to rely on it.

Most developers on the JVM will most likely never experience any problems with these additional restrictions but as described, code generation and proxying libraries need to adapt these changes. For cglib, this does unfortunately mean that the end of the road is reached. Cglib was originally modeled as a more powerful version of Java's built-in proxying API where it requires its own dispatcher API to be referenced by the proxy class similar to how Java's API requires referencing of its types. However, these latter types reside in the java.base module which is always read by any module. For this reason, the Java proxying API still functions whereas the cglib model was broken irreparably. In the past, this has already made cglib a difficult candidate for OSGi enviornments but with the JPMS, cglib as a library does no longer function. A similar problem exists for the corresponding proxying API that is provided by Javassist.

The upside of this change is that the JVM finally offers a stable API for defining classes during an application's runtime, a common operation that has relied on internal API for over twenty years. And with the exception of Javaagents that I think still require a more flexible approach, this means that future Java releases are guaranteed to always work once all users of proxies completed this final migration. And given that the development of cglib has been dormant for years with the library suffering many limitations, an eventual migration by today's users of the library was inevitable in any case. The same might be true for Javassist proxies, as the latter library has not seen commits in almost a half year either.

Mittwoch, 10. Mai 2017

Yet another Jigsaw opinion piece

In the past weeks there has been a heated debate around the imminent release of Java 9 and its most famous feature: the Java platform module system – the JPMS which is better known under its project umbrella‘s name Jigsaw. The module system is introduced into the Java ecosystem in form of a formal specification process a JSR which needs to be approved in its final form by its expert group. Among other members of this expert group, representatives of Red Hat and IBM have now voted to reject Java‘s module system in the first ballot which they believe is not yet ready for production.  

What‘s the fuzz all about?


Even today, Java developers are widely familiar with modularity. Build systems like Maven organize code as modules that are compiled against a declared set of dependencies. Only at runtime, these modules are put together on the class path where these compile-time module boundaries vanish. With Jigsaw, the module path is offered as an alternative to this class path for which the JVM retains such compile-time boundaries at runtime. By not using this module path, applications should function just as before. But this comes with the exception of applications that rely on APIs internal to the JVM. The Java standard library is always loaded as a collection of modules, even if the class path is used exclusively such that internal Java APIs are no longer accessible.

This latter limitation in compatibility has raised some concerns among the maintainers of both libraries and end-user applications. And in this context it can be a bit surprising that the recent objections do not relate too much to these concerns. While mentioning issues around compatibility, both Red Hat and IBM predominantly argue that the JPMS requires further extension to allow for a better integration with existing module systems such as JBoss modules and OSGi.

What problem does still need solving?


By jar hell, developers typically describe a situation where a Java application would require two different versions of a library to satisfy different transitive dependencies. Using the class path, this is impossible as one version of a library shadows a second copy. If a class of a given name is loaded for the first time, the system class loader scans jar files in their command line order and loads the first class file it discovers. In the worst case, this can result in Frankenstein functionality if the shadowed jar file contains some exclusive classes that link with classes of the shadowing jar. But more typically, it results in a runtime failure once a feature dependent on a specific version is triggered.

With OSGi and JBoss modules, this problem can be partially resolved. The latter module systems allow to load a library by each its own class loader, thus avoiding the system class loader that is responsible for the class path. With this approach, multiple versions of the same class can coexist by isolation within separate class loaders. Doing so, it is for example possible for two libraries to both depend on their specific version of the commonly breaking Guava API. With class loader isolation, any library would delegate calls to its required version when loading dependent classes.

When using the module path, the JPMS does not (currently) apply such class loader isolation. This means that jar hell is not solved by Java 9. In contrast to using the class path, the JVM does however detect the described version conflict and fails the application at startup, rather than speculating on accidental compatibility. To enforce this constraint, every Java package name is now exclusive to a specific module or the class path. Therefore, it is not possible for two modules to share a package. This restriction also holds if a private package is not meant to be exposed what is considered to be another flaw of the current module design by Jigsaw‘s critics.

A missed chance to escape jar hell?


For class loader isolation to work, it is necessary that versions of the same module never interact. And while two such versions would of course never interact directly, it is unfortunately more than common that two versions are part of the public API of different modules. For example, if two libraries return instances of Guava‘s Function type, a version conflict between each module‘s Guava version can no longer be solved using class loader isolation, even if the Function type did not change between those versions. At runtime, any loaded class is described as a tuple of its name and class loader but since two class loaders do now offer the Function type, which one should be resolved?

This described problem, as a matter of fact, cannot be solved by a module system. Instead, a module system can discover this conflict and inform the user of the need of an explicit resolution. This is accomplished by the current implementation of the JPMS and of course both OSGi and JBoss modules. At the end of the day, version conflicts can only be avoided by evolving APIs in a compatible manner.

Is Jigsaw too simple?


Despite the remaining limitations of a class loader-isolating module system, the current argument against Jigsaw is mainly revolving around this item. Additionally, the expert group members that reject Jigsaw point out the lacking support for circular module dependencies ("module A depends on B depends on C depends on A") and the inability to alter the module graph after its creation.

From a technical perspective, it would of course be possible to add these features. As a matter of fact, Java 9 already ships with a module builder API that allows for loading modules with exclusive class loaders. There is no technical limitation in choosing to retain a single class loader for the module path; rather this decision is considered to be the responsible choice for the JVM by Oracle. And before diving deeper into the arguments, I want to state that I fully agree with the company’s reasoning.

What is wrong with class loader isolation?


As mentioned before, even with class loader isolation, manual version management can often not be avoided. Also, library authors that rely on common APIs with version incompatibilities such as Guava do increasingly shade such dependencies. When shading, the code of a library is copied into a separate name space, thus allowing an application to refer to “its version“ by different names instead of by different class loaders. This approach has of course flaws of its own, especially when a shaded dependency uses JNI. On the other hand, this approach overcomes the just mentioned shortcoming of class loader isolation when using libraries with conflicting shared dependencies. Also, by shading a common dependency, a library author relieves its users from potential conflicts independently of a deployment method.

Allowing for circular dependencies would neither impose a big technical challenge. However, cyclic dependencies are rather uncommon and many build systems like Maven do neither support them. Typically, cyclic dependencies can be refactored to non-cyclic ones by splitting up at least one module into implementation and API. In this context, if a feature seems to be of such little common concern, I do not think corner cases justify its addition, especially when the class path still serves as a backup. And if this decision turns out to be wrong, cyclic dependencies can always be enabled in a future release. Taking this feature away would however not be possible.

Finally, dynamic modules render a feature that could be useful to more than a few applications. When you require the dynamic redeployment of modules with an active life cycle, from my experience in my last project, OSGi is a very good choice. That said, most applications are static and do not have a good reason for its use. But by adding support for a dynamic module graph, the complexity of this feature would translate into the JPMS. Therefore, I think it is the right decision to leave this feature out for now and wait until its use is better understood. Naturally, an approachable module system increases adoption.

Compatibility first.


Does this incompatibility mean the end for OSGi and JBoss modules? Of course not. Quite to the contrary, the introduction of standardized module descriptors gives opportunity to existing module systems. Missing manifest headers to describe bundles is one of the major pain points when using OSGi due to a significant number of libraries that do not consider the proprietary module descriptor. With the introduction of a standardized module descriptor, existing module systems can ease this limitation by using the latter descriptor as a secondary source for a module’s description.

I do not doubt for a second that Red Hat and IBM rejected the JSR with their best intentions. At the same time, I cannot agree with the criticism on the lacking reach of the module system. In my opinion, the existing changes are sufficiently challenging for the Java ecosystem to adopt and especially a last minute introduction of class loader isolation bears the potential of unwanted surprise. In this light, I find the arguments made against the current state of Jigsaw inconsistent as it criticizes the complexity of transition to modules but also demands its extension.

There is no perfect module system


Personally, I think that the current proposal for the JPMS bears two big challenges. Unfortunately enough, they got in the background due to the recent discussion.

Automatic modules

Without a module descriptor, modular code can only refer to a non-modular jar file in form of a so-called automatic module. Automatic modules do not impose any restrictions and are named by their jar file. This works well for developers of end user applications which never release their code for use by another application. Library developers do however lack a stable module name to refer to their dependant automatic modules. If released, they would rely on stable file names for their dependencies which are difficult to assume.

For the adoption of Jigsaw, this would imply a bottom-up approach where any library author can only modularize their software after all dependant code was already modularized. To ease the transition, a manifest entry was added which allows to publish a jar with a stable automatic module name without the need to modularize code or even migrate to Java 9. This allows other libraries users that depend on this first library with a stable name to modularize their code, thus breaking through the bottom-up requirement.

I think it is essential to allow library maintainers to state an explicit module name before their code is migrated to fully use the JPMS and I consider this a more than adequate way to deal with this problem which does unlikely offer a better solution.

Reflection and accessibility

With Jigsaw, it is no longer allowed to access non-public, non-exported members using reflection what is an opportunity that many frameworks currently assume. Of course, with a security manager being set, such access can be impossible even in today’s Java releases but since security managers are used so seldom, this is not thought of much. With Jigsaw, this default is reversed where one needs to explicitly open packages for such reflective access, therefore affecting many Java applications.

Generally, I think that Jigsaw’s encapsulation is a better default than the current general openness. If I want to give Hibernate access to my beans, the JPMS allows me to open my beans to Hibernate only by a qualified export. With a security manager, controlling such fine grained access was difficult if not impossible to implement. However, this transition will induce a lot of growing pain and many libraries are not maintained actively enough to survive adopting these new requirements. Thus, adding this restriction will definitely kill off some libraries that would otherwise still provide a value.

Also, there are use cases of reflection that are still uncovered. For the mocking library Mockito (that I help maintain) we do for example need a way of defining classes in any class loader. This was and still is only possible by the use of internal Java APIs for which no alternative is yet offered. As Mockito is only used in test environments, security should not be of concern in this context. But thanks to the retained openness of sun.misc.Unsafe on which we already rely for instantiating mock classes without constructor calls, we can simply open these APIs by changing their accessibility using its direct memory APIs.

This is of course no good enough solution for the years to come but I am convinced that those concerns can be addressed before removing the Unsafe class entirely. As one possibility, the JVM could be extended with a test module that needs to be resolved explicitly on the command line and which allows such extended access. Another option would be to require the attachment of a Java agent by any test runner because of their ability to break through module barriers. But as for now, any maintained software has an opportunity to resolve its non-standard Java use and continue the discussion on missing APIs in the upcoming years.

Finding consensus.


Considering the stereotype of the socially anxious computer nerd, software development can be a rather emotional business. Oracle has always been a company that Java developers love to hate and the current discussion partly jumps on this bandwagon. Looking at the success of Java as a language and a platform, I do however think that Oracle deserves credit for its objectively good job in its stewardship. Breaking software today with future success in mind is a delicate and grateless task. Anybody who refactored correct but complex code should be sympathetic to this challenge.

Project Jigsaw has often been criticized for being an unnecessary effort and I admit that this thought had crossed my own mind. Yet, it is thanks to the module systems that dead weight like CORBA or RMI can finally be removed from the JVM. With the implied reduction in size of modular Java applications, the JVM has become more attractive for the use within containerized applications and cloud computing what is surely no coincidence given Oracle’s market strategy. And while it would of course be possible to further postpone this effort to a later Java release, the JVM must address the removal of functionality at some point. Now is as good a time as any.

To ease the upcoming transition, it is important to keep the breaking changes down to a minimum. Therefore, I am convinced that extending the scope of Jigsaw is not in the best interest of the broader Java community. Many of the rejecting votes of the recent ballot asked for the involved parties to find consensus on the outstanding issues. Unfortunately, the features in question can either be implemented or discarded where consensus can only be reached by one party giving up their position.

With the typical Java application in mind, I do hope that Oracle does not answer to the demands with a scope extension only to secure a successful vote on the Jigsaw JSR. Rather, I want to appeal to the expert group members who were voting against the JSR to reconsider their vote with the needs of the entire Java ecosystem in mind where the requirements of existing enterprise module solutions are only one factor among many. With the broad usage of Java, ranging from business applications to low-latency systems, it is only natural that different parties identify different priorities for the evolution of the platform. I am convinced that Oracle has found a common denominator for a module system that serves most users.

Donnerstag, 27. Oktober 2016

Generational disparity in garbage collection

For the last year, I have been helping the startup Instana to create a Java agent that traces executions within a Java application. This execution data is collected and jointed to generate traces of user requests as well as the resulting communication between services within the system owner’s hemisphere. This way, unstructured communication can be visualized what significantly simplifies the operation of a distributed system that is composed of multiple interacting services.

In order to generate these traces, the Java agent rewrites all code that reads an external request or initiates one. Obviously, these entries and exits into or out of a system need to be recorded and additionally, meta data is exchanged to identify a request uniquely across systems. For example, when tracing HTTP requests, the agent adds a header containing a unique id which is then recorded by the receiving server as a proof of a request’s origin. Broadly speaking, it is similar to what Zipkin is modeling, but without requiring users to change their code.

In the most simple scenario, such tracing is straightforward to implement. Thanks to my library Byte Buddy which does the heavy lifting, all injected code is written in plain old Java and then copied to the relevant methods at runtime using the Java instrumentation API. For example, when instrumenting a servlet, we know that an entry to a JVM is made whenever the service method is invoked. We also know that the entry is completed when this very same method exits. Therefore, it suffices to add some code to the beginning and the end of the method to record any such entry into a VM process. And it has been the majority of my job to plow through the many Java libraries and frameworks to add support for their ways of communication. From Akka to Zookeeper, over the last year I have hello-worlded my way through the entire Java ecosystem; I even got to write EJBs for all the servers! And I had to make sense of Sun’s CORBA implementation. (Spoiler: There is no sense.)

Things do however quickly become more difficult when tracing asynchronous executions. If a request is received by one thread but is answered from within another thread, it does no longer suffice to only trace entries and exits. Therefore, our agent needs to also track all context switches in concurrent systems made via thread pools, fork join tasks or custom concurrency frameworks. And the same way that debugging asynchronous execution is difficult, this is quite a bit of work for us too. I think that I spend as much time dealing with concurrency as I do recording entries and exits.

The impact on garbage collection


But how does all this impact garbage collection? When implementing a performance monitor, one is facing a trade-off between interpreting the work of a Virtual Machine and causing work for this machine by doing so. While the majority of processing is done in the monitor back-end to which the agent reports its data, we have to do a minimum within the Java process that we share with the monitored application. And you can already guess it: by allocating objects, we inevitably have an impact on the VM’s garbage collection. Fortunately, modern garbage collection algorithms are doing excellent work and by mostly avoiding object allocation and by adaptively sampling our tracing efforts, the effect of our code changes is negligible for the vast majority of users. Ideally, we only burn a few unused processor cycles to do our work. As a matter of fact, very few applications do use their full processing potential and we are happy with grabbing a small portion of this excess.

Writing a garbage collection-friendly application is typically not too difficult. It is obvious that the easiest way of avoiding garbage is to avoid object allocation altogether. However, object allocation in itself isn’t too bad either. Allocating memory is a rather cheap operation and as any processor owns its own allocation buffer - a so-called TLAB - we do not impose an unnecessary synchronization when allocating only a bit of memory from our threads. If an object only lives in the scope of a method, the JVM can even erase the object allocation altogether as if the fields of the objects were put onto the stack directly. But even without this escape-analysis, short-lived objects are captured by a special garbage collection circle called the young generation collection that is processed quite efficiently. To be honest, this is where most of my objects end up as I often value code readability over the small improvements that escape analysis offers. Currently, escape analysis quickly hits its boundary. Yet, I hope for future HotSpots to improve to get the best of both worlds even without changing my code. Fingers crossed!

When writing Java programs, I do not typically think about the impact on garbage collection but the above guidelines tend to manifest in my code. For the majority of our agent, this has been working very well. We are running a whole bunch of example applications and integration tests to assure a good behavior of our agent and I also keep an eye on the GC when running examples. In our modern times, using tools like flight recorder and JIT watch, performance analysis has become quite approachable.

The relativity of short-lived


With an early version of our agent, I one day noticed an application to trigger tenured collection cycles that it did not trigger without it. As a consequence, collection pauses increased by a multitude. The objects that ended up in the tenured collection were however only objects of the monitored application itself. But since our agent runs mostly isolated from the application threads and at first, this did at first not make sense to me.

When digging deeper, I found that our analysis of user objects triggered some additional escapes of objects but the impact was minimal. The application did already produce a fair amount of objects, mostly by using NIO and by using fork join pools. One thing that the latter frameworks have in common is that they rely on the allocation of many short lived objects. For example, a fork-join task often splits itself into multiple subtasks which repeat this procedure until each task’s payload is small enough to be computed directly. Every such task is represented by a single, stateful object. An active fork join pool can spawn millions of such objects every minute. But since the tasks compute fast, the representing object is eligible for collection quickly and therefore captured by the young collector.

So how did these objects end up in the tenured collection all of a sudden? At this time, I was prototyping a new stitching instrumentation to track context switches between such fork join tasks. Following the path of a fork join tasks is not trivial. Each worker thread of a fork join pool applies work stealing and might grab tasks out of the queue of any other task. Also, tasks might provide a feedback to their parent task on completion. As a consequence, tracing the expansion and interaction of tasks is a rather complex process, also because of the existence of so-called continuation threads where a single task might bounce jobs to hundreds of threads within only a few milliseconds. I came up with a rather elegant solution which relied on the allocation of many short-lived objects which were allocated in bursts whenever backtracking a task to its origin. It turned out that these bursts triggered quite a few young collections themselves.

And this is what I did not consider: each young generation collection increases the age of any object that is not eligible for garbage collection at this point. An object does not age by time but by the amount of young collections triggered. This is not true for all collection algorithms but for many of them such as for all default collectors of HotSpot. And by triggering so many collections, the agent threads “prematurely matured” objects of the monitored application despite those objects being unrelated to the agent’s objects. In a way, running the agent “prematurely matured” the target application’s object.

Getting around the problem


I did at first not know how to solve this. In the end, there is no way of telling a garbage collector to treat “your objects” separately. As long as the agent threads were allocating shorter-lived objects at a faster rate than the host process, it would spoil the original objects into the tenured collection causing an increase of garbage collection pauses. In order to avoid this, I therefore started to pool the objects I was using. By pooling, I quickly matured my own objects into the tenured collection and the garbage collection behavior returned to its normal state. Traditionally, pooling was used to avoid the costs of allocation which became cheap in our days. I rediscovered it to erase the impact of our “foreign process” onto garbage collection for the cost of a few kilobytes of memory.

Our tracer is already pooling objects in other places. For example, we represent entries and exits as thread local values that contain a bunch of primitive values that we mutate without allocating a single object. And while such mutable, often procedural and object pooling programming is no longer fashionable, it turns out to be very performance friendly. In the end, mutating bits is closer to what a processor is actually doing. And by using preallocated arrays of a fixed size instead of immutable collections, we save us quite a few round-trips to memory while also preserving our state to be contained in only a few cache lines.

 

Is this a “real world” problem?


You might think that this is a rather specific problem that most people do not need to worry about. But as a matter of fact, the problem that I describe applies to a large number of Java applications. For example, within application containers, we typically deploy multiple applications in a single Java process. Just as in the above case, the garbage collection algorithm does not group objects by application as it has no notion of this deployment model. Therefore, object allocations by two isolated applications that share a container do interfere with the anticipated collection patterns of one another. If each application relies on its objects to die young, the sharing of a heap causes a strong relativity on the duration of short-lived.

I am not an advocate for microservices. As a matter of fact, I think they are a bad idea for most applications. In my opinion, routines that can only exist in interaction should ideally be deployed together unless there are good technical reasons not to. And even if isolated applications ease development, you quickly pay the price in operations. I am just mentioning this to avoid a misinterpretation of the moral of the above experience.

What this experience taught me was that deploying several applications in a single Java process can be a bad idea if those applications are heterogeneous. For example, when running a batch process parallel to a web server, you should consider running each in its own process rather than deploying both of them in the same container. Typically, a batch process is allocating objects at a very different rate than a web server. Yet, many enterprise frameworks still advertise all-in-one solutions for tackling such problems which should not share a process to begin with. In 2016, the overhead of an additional process is not typically a problem and since memory is cheap, rather upgrade your server instead of sharing a heap. Otherwise, you might end up with collection patterns that you did not anticipate when developing, running and testing your applications in isolation.