On Google+ Mark Derricut wrote about a post by RedHat's Scott Stark about Jigsaw, the Java 9 module system, as well as a follow up by Mike Hearn.
I have been following Jigsaw for some years, and I found both posts very interesting. Scott's post highlights some worrying problems in the current Jigsaw implementation. I was already aware of some of them as I read some earlier posts by Neil Bartlett, saw his presentation at EclipseCon Europe 2016, and I have been following Thomas Watson OSGi-Jigsaw interoperability experiments.
In his follow-up post Mike says to lean more toward Jigsaw because OSGi is complicated, poorly documented, uses a crazy versioning approach, and is not that well known. He also said that he'd rather prefer the module declaration to be a source file compiled to byte-code rather than a XML or textual file, and highlights jlink (which allows you to create a statically linked distribution of your application that includes the JRE) as an example of good tools that the JDK will provide on top of Jigsaw. Mike says the RedHat post is not fair, that the authors think “we were here first, so what we did is best practice and you should do it too”, and that Oracle dropped many aspects of other module systems such as OSGi or JBoss Modules to avoid their complexity and reduce the risk.
I personally believe that RedHat builds amongst the most sophisticated products on the market. We must recognize that their people are knowledgeable, their software is robust and runs thousands of applications in production. Writing modular software is certainly more complex than writing monolithic applications. By making Application servers and middleware solutions, companies like RedHat and IBM have to deal with this problem more than any one else. They also have a very large client base, therefore they should be aware of the Java enterprise landscape more than anyone else. Hence, it's my opinion that when RedHat engineers express concerns about the ability of Jigsaw to fulfill today requirements for modularity we should listen to them very carefully. Besides, RedHat JBoss does not officially support OSGi, so they are not really trying to advocate for it; I believe they are simply trying to highlight the issues in Jigsaw. Scott's post includes contributions by Maven committers, and I trust their opinion as Maven is not tied to OGSi or JBoss Modules. Like it or not, Maven is the standard for building Java, therefore I am very worried by the long list of issues between Jigsaw and Maven and if I was Oracle I would have paid more attention to Jigsaw-Maven compatibility.
I also believe that it should be clear to everyone that in recent years Oracle has been working a lot on JavaSE without caring that much about JavaEE, still stuck at version 7. I am afraid we will need to wait ages for JavaEE 9 or that it may happen that JavaEE 9 will be skipped and we will have JavaEE 10 when Jigsaw will have matured in JavaSE 10.
Long story short: future does not look bright for JavaEE developers, who historically always had to use older versions of Java for several years and now it may be even worse.
Is modularity only important for vendors like RedHat or IBM? Of course not. Enterprise applications become more and more complex year after year. For example, it's now very common to have to support native mobile applications as clients, to have to provide API for third parties, to have to provide features such as social integration (Facebook, Twitter, LinkedIn) and collaboration (e.g. chat and video conference capabilities), and many more.
If you are a vendor with a large functional offering there are many chances that not all your clients will want to buy everything, hence you need to modularize your offering to be able to license to your clients and deploy nothing more than what they want and need.
I am not affiliated with RedHat, OSGi but I would like to comment to some of Mike statements.
First few words about myself. I have been coding in Java since 1999, professionally since 2001. I spent the first few years of my career on Java Enterprise, then moved to Eclipse plug-in development for 7 years or so, then back to Java enterprise. In 2001 I was deploying Java Enterprise applications on IBM WebSphere 3, by copying all Jars to some folder on the server file system and by specifying the application class-path in a 20 column (yet scrollable!) text field of the Swing Gui used for administering WebSphere (EAR files were not supported at that time).
Mike' statement #1: OSGi is complicated
When working with large JavaEE applications, the so-called Java "Jar hell" has always been a major issue to me. It's when I started coding for Eclipse 3 that I met OSGi for the first time, and I can tell you that I can not imagine a platform like Eclipse to succeed without a strong modular system. Eclipse plug-ins are "simplified" OSGi bundles: most of them don't use
the OSGi registry (or at least do not require downstream plug-ins to use it), they express dependencies in term of plug-in names rather than packages, and the Equinox OSGi container runs in a more "relaxed" way than strict OSGi. Programming for the Eclipse platform told me that OSGi is not that complex, you can code your first Eclipse plug-in in minutes, and modularity really helps in large applications.
When I moved back to JavaEE I tried to run OSGi on the server. The specification addresses some complex concerns such as dynamicity and the possibility that bundles are installed/uninstalled
on a live application. This requirement is covered by API such as BundleListener and ServiceTracker which certainly add complexity. However most applications will not need that dynamism. Eclipse for example does not allow users to replace individual bundles and always requires a restart after installing/updating some plug-ins. IBM WebSphere Liberty, a JavaEE and OSGi Application Server, does not allow you to hot-swap individual bundles of your OSGi application, which can only be started/stopped as a whole.
Hot-swap makes a lot of sense for IoT or embedded systems, but is not strictly necessary for Enterprise Applications. So if you don't need that level of dynamism most of the perceived OSGi complexity disappears and OSGi becomes just a module system, offering API on top of Java that you may want to use or not. For example, if you like Spring you can get Eclipse Virgo for Tomcat Server and develop your truly modular, Spring based, application with very limited knowledge of OSGi itself (disclaimer: I am a Virgo committer). But you will still get an OSGi environment that enforces true modularity (and can support the most complex scenarios in case you need them).
If you compare it with JavaEE, the OSGi spec is smaller and in my opinion better written. If you started with JavaEE several years ago and saw it growing then you learned JAXB, JAX-RS, JPA gradually. But if you start today as a new developer, JavaEE has a learning curve which is as complex as OSGi. That's why frameworks like Spring are so popular.
Mike's statement #2: OSGi is poorly documented
This may be the case. For plain Java you go to the Oracle website and start from there. For OSGi, information is scattered over the web and it's difficult to pick the right one for starting. The OSGi Alliance web site is not that good, although they are trying to fix that with project enRoute.
Mike's statement #3: OSGi is not that successful
A lot of products out there are built on OSGi. Let me give some examples:
Mike's statement #4: OSGi’s approach to versioning is really crazy
Mike complains that OSGi requires you to give a version not only to the module itself but also to the exported Java packages. Having worked on Eclipse plug-ins first and later on pure OSGi I think the OSGi approach makes a lot of sense. Eclipse plug-ins depend on module names. This is problematic when the code base is refactored and a large plug-in is split in two: your dependency is impacted and you need to change the name of the plug-in you depend upon or add another one. That's why Eclipse contains "compatibility" bundles that re-aggregate and re-export as a whole modules that have been split. Depending on packages makes much more sense, your dependency declaration is not affected even if the maintaners split or rename their bundles. Would you depend on javax.xml.parsers or would you depend on Apache Xerces? Would you depend on the Servlet spec or would you want your code to depend on Tomcat implementation?
Mike's statement #5: module declaration should be a source file
I agree with Mike that the module declaration should be natively supported by the compiler. I believe this is also what the OSGi Alliance would have wanted if they had the ability to change the compiler. Module declarations as source files will allow the compiler to check for errors at compile-time. At the same time I also agree with Scott that the module declaration should be readable to diagnose problem at run-time. Therefore I agree it should not be compiled to byte-code, but rather kept "as is" to be readable, or maybe compiled at run-time, or maybe compiled in a readable format, such as a MANIFEST.MF file or a XML file.
Mike's statement #6: Oracle will build better tools on top of Jigsaw
It's true that today to code for OSGi or JBoss Modules you have to rely on IDEs and third party tools as those module systems are not supported natively by the JDK. So it's clear that the situation will improve when the the Java language will provide modularity out of the box. However this point is a bit stretched in my opinion, because this could have been the case also if Oracle decided to build a different Jigsaw, or to adopt OSGi or JBoss Modules. Plus, I would not bring jlink as an example of a great improvement because that tool will most likely be of little interest to Java enterprise developers, and in my opinion it's in Java Enterprise where modularity is most important. The Eclipse platform of course is not JavaEE but despite being a desktop application, it's nature makes it more similar to a container than a JavaSE application.
Conclusions
To conclude, in my opinion modularity in software is inherently complex. Module systems need to support that complexity and therefore will be complex as well. OSGi not only addresses modularity but also dynamism which adds additional levels of complexity. You can however decide to ignore the dynamic part of OSGi, like the Eclipse platform does, and simply take advantage of it's support for modularity. Jigsaw could do the same, be a subset of OSGi and provide support for modularity. I understand that Oracle is having troubles making the JDK modular, and that Jigsaw is conservative to reduce risks. However it's probably a bit too limited as a modular system. Let's hope this is the first iteration, let's hope it will be adopted by developers, and Oracle will quickly release a 2.0 version that will be feature-wise closer to OSGi or JBoss Modules. Meanwhile, Maven will probably play a big role in making Jigsaw successful or not.
I have been following Jigsaw for some years, and I found both posts very interesting. Scott's post highlights some worrying problems in the current Jigsaw implementation. I was already aware of some of them as I read some earlier posts by Neil Bartlett, saw his presentation at EclipseCon Europe 2016, and I have been following Thomas Watson OSGi-Jigsaw interoperability experiments.
In his follow-up post Mike says to lean more toward Jigsaw because OSGi is complicated, poorly documented, uses a crazy versioning approach, and is not that well known. He also said that he'd rather prefer the module declaration to be a source file compiled to byte-code rather than a XML or textual file, and highlights jlink (which allows you to create a statically linked distribution of your application that includes the JRE) as an example of good tools that the JDK will provide on top of Jigsaw. Mike says the RedHat post is not fair, that the authors think “we were here first, so what we did is best practice and you should do it too”, and that Oracle dropped many aspects of other module systems such as OSGi or JBoss Modules to avoid their complexity and reduce the risk.
I personally believe that RedHat builds amongst the most sophisticated products on the market. We must recognize that their people are knowledgeable, their software is robust and runs thousands of applications in production. Writing modular software is certainly more complex than writing monolithic applications. By making Application servers and middleware solutions, companies like RedHat and IBM have to deal with this problem more than any one else. They also have a very large client base, therefore they should be aware of the Java enterprise landscape more than anyone else. Hence, it's my opinion that when RedHat engineers express concerns about the ability of Jigsaw to fulfill today requirements for modularity we should listen to them very carefully. Besides, RedHat JBoss does not officially support OSGi, so they are not really trying to advocate for it; I believe they are simply trying to highlight the issues in Jigsaw. Scott's post includes contributions by Maven committers, and I trust their opinion as Maven is not tied to OGSi or JBoss Modules. Like it or not, Maven is the standard for building Java, therefore I am very worried by the long list of issues between Jigsaw and Maven and if I was Oracle I would have paid more attention to Jigsaw-Maven compatibility.
I also believe that it should be clear to everyone that in recent years Oracle has been working a lot on JavaSE without caring that much about JavaEE, still stuck at version 7. I am afraid we will need to wait ages for JavaEE 9 or that it may happen that JavaEE 9 will be skipped and we will have JavaEE 10 when Jigsaw will have matured in JavaSE 10.
Long story short: future does not look bright for JavaEE developers, who historically always had to use older versions of Java for several years and now it may be even worse.
Is modularity only important for vendors like RedHat or IBM? Of course not. Enterprise applications become more and more complex year after year. For example, it's now very common to have to support native mobile applications as clients, to have to provide API for third parties, to have to provide features such as social integration (Facebook, Twitter, LinkedIn) and collaboration (e.g. chat and video conference capabilities), and many more.
If you are a vendor with a large functional offering there are many chances that not all your clients will want to buy everything, hence you need to modularize your offering to be able to license to your clients and deploy nothing more than what they want and need.
I am not affiliated with RedHat, OSGi but I would like to comment to some of Mike statements.
First few words about myself. I have been coding in Java since 1999, professionally since 2001. I spent the first few years of my career on Java Enterprise, then moved to Eclipse plug-in development for 7 years or so, then back to Java enterprise. In 2001 I was deploying Java Enterprise applications on IBM WebSphere 3, by copying all Jars to some folder on the server file system and by specifying the application class-path in a 20 column (yet scrollable!) text field of the Swing Gui used for administering WebSphere (EAR files were not supported at that time).
Mike' statement #1: OSGi is complicated
When working with large JavaEE applications, the so-called Java "Jar hell" has always been a major issue to me. It's when I started coding for Eclipse 3 that I met OSGi for the first time, and I can tell you that I can not imagine a platform like Eclipse to succeed without a strong modular system. Eclipse plug-ins are "simplified" OSGi bundles: most of them don't use
the OSGi registry (or at least do not require downstream plug-ins to use it), they express dependencies in term of plug-in names rather than packages, and the Equinox OSGi container runs in a more "relaxed" way than strict OSGi. Programming for the Eclipse platform told me that OSGi is not that complex, you can code your first Eclipse plug-in in minutes, and modularity really helps in large applications.
When I moved back to JavaEE I tried to run OSGi on the server. The specification addresses some complex concerns such as dynamicity and the possibility that bundles are installed/uninstalled
on a live application. This requirement is covered by API such as BundleListener and ServiceTracker which certainly add complexity. However most applications will not need that dynamism. Eclipse for example does not allow users to replace individual bundles and always requires a restart after installing/updating some plug-ins. IBM WebSphere Liberty, a JavaEE and OSGi Application Server, does not allow you to hot-swap individual bundles of your OSGi application, which can only be started/stopped as a whole.
Hot-swap makes a lot of sense for IoT or embedded systems, but is not strictly necessary for Enterprise Applications. So if you don't need that level of dynamism most of the perceived OSGi complexity disappears and OSGi becomes just a module system, offering API on top of Java that you may want to use or not. For example, if you like Spring you can get Eclipse Virgo for Tomcat Server and develop your truly modular, Spring based, application with very limited knowledge of OSGi itself (disclaimer: I am a Virgo committer). But you will still get an OSGi environment that enforces true modularity (and can support the most complex scenarios in case you need them).
If you compare it with JavaEE, the OSGi spec is smaller and in my opinion better written. If you started with JavaEE several years ago and saw it growing then you learned JAXB, JAX-RS, JPA gradually. But if you start today as a new developer, JavaEE has a learning curve which is as complex as OSGi. That's why frameworks like Spring are so popular.
Mike's statement #2: OSGi is poorly documented
This may be the case. For plain Java you go to the Oracle website and start from there. For OSGi, information is scattered over the web and it's difficult to pick the right one for starting. The OSGi Alliance web site is not that good, although they are trying to fix that with project enRoute.
Mike's statement #3: OSGi is not that successful
A lot of products out there are built on OSGi. Let me give some examples:
- Pretty much all IBM Java products (Eclipse, IBM Notes, WebSphere, the Rational suite). I read somewhere that IBM has 100+ products implemented in OSGi. Again, my opinion is that if IBM selected it there must be some good value in OSGi.
- If I am not mistaken, Sonatype Nexus, the Maven repository, is an OSGi application.
- Liferay is an OSGi application.
- At least some WSO2 products are OSGi based (Carbon)
- Glassfish/Payara is OSGi based
- If I remember correcty, Oracle WebLogic is built on OSGi
- Most of the popular Java libraries are also OSGi bundles, for example: Google Guava, Google Guice, Google Gson, most of the Apache libraries, Jackson, Slf4j, EclipseLink and Jersey by Oracle and many many more. I doubt all these projects had OSGi metadata out of the box if no one was asking for them.
Mike's statement #4: OSGi’s approach to versioning is really crazy
Mike complains that OSGi requires you to give a version not only to the module itself but also to the exported Java packages. Having worked on Eclipse plug-ins first and later on pure OSGi I think the OSGi approach makes a lot of sense. Eclipse plug-ins depend on module names. This is problematic when the code base is refactored and a large plug-in is split in two: your dependency is impacted and you need to change the name of the plug-in you depend upon or add another one. That's why Eclipse contains "compatibility" bundles that re-aggregate and re-export as a whole modules that have been split. Depending on packages makes much more sense, your dependency declaration is not affected even if the maintaners split or rename their bundles. Would you depend on javax.xml.parsers or would you depend on Apache Xerces? Would you depend on the Servlet spec or would you want your code to depend on Tomcat implementation?
Mike's statement #5: module declaration should be a source file
I agree with Mike that the module declaration should be natively supported by the compiler. I believe this is also what the OSGi Alliance would have wanted if they had the ability to change the compiler. Module declarations as source files will allow the compiler to check for errors at compile-time. At the same time I also agree with Scott that the module declaration should be readable to diagnose problem at run-time. Therefore I agree it should not be compiled to byte-code, but rather kept "as is" to be readable, or maybe compiled at run-time, or maybe compiled in a readable format, such as a MANIFEST.MF file or a XML file.
Mike's statement #6: Oracle will build better tools on top of Jigsaw
It's true that today to code for OSGi or JBoss Modules you have to rely on IDEs and third party tools as those module systems are not supported natively by the JDK. So it's clear that the situation will improve when the the Java language will provide modularity out of the box. However this point is a bit stretched in my opinion, because this could have been the case also if Oracle decided to build a different Jigsaw, or to adopt OSGi or JBoss Modules. Plus, I would not bring jlink as an example of a great improvement because that tool will most likely be of little interest to Java enterprise developers, and in my opinion it's in Java Enterprise where modularity is most important. The Eclipse platform of course is not JavaEE but despite being a desktop application, it's nature makes it more similar to a container than a JavaSE application.
Conclusions
To conclude, in my opinion modularity in software is inherently complex. Module systems need to support that complexity and therefore will be complex as well. OSGi not only addresses modularity but also dynamism which adds additional levels of complexity. You can however decide to ignore the dynamic part of OSGi, like the Eclipse platform does, and simply take advantage of it's support for modularity. Jigsaw could do the same, be a subset of OSGi and provide support for modularity. I understand that Oracle is having troubles making the JDK modular, and that Jigsaw is conservative to reduce risks. However it's probably a bit too limited as a modular system. Let's hope this is the first iteration, let's hope it will be adopted by developers, and Oracle will quickly release a 2.0 version that will be feature-wise closer to OSGi or JBoss Modules. Meanwhile, Maven will probably play a big role in making Jigsaw successful or not.
