What is OSGi for??
In his daily update at java.net, Chris Adamson writes about the increasing prominence of OSGi. But also admits that he just doesn’t “get” OSGi, and it seems he’s not the only one. Perhaps we in the OSGi community, in our enthusiasm for explaining the cooler features of OSGi, have not done a great job of explaining what OSGi is and why it matters.
In a nutshell, OSGi is a module system for Java, but what does that really mean?
One of the main functions of a module is as a unit of deployment… something that we can either build or download and install to extend the functionality of our application.
The standard unit of deployment in Java is the JAR file. We take these JAR files and put them on our classpath and, some of the time, our application runs. Why only some of the time? Because a JAR file is not a module, it is merely an arbitrary chunk of code. Also, when we load a JAR file at runtime it immediately disappears: the “classpath” is merely a long list of classes. JARs that we place on the classpath contribute to that list, but at runtime the boundaries between the JARs dissolve.
One symptom of this is that JARs have no way to hide implementation packages from users. Another is that classes are loaded from the first possible location in the classpath, which is not always correct. For example, suppose a JAR file contains two classes, org.foo.A and org.foo.B, and A depends on B. We ship these two classes together in a JAR, but there is no way for A to insist that it gets the B from the same JAR. If another JAR happens to provide org.foo.B (e.g. an older version of the same library) and that other JAR is listed earlier on the classpath, then we will always get that other B. Evil classloading errors ensue.
A real module consists not just of implementation code, but also metadata that describes: (a) What this module is for and who provides it etc (b) What dependencies it has, and (c) What it provides to other modules. Also in a real module system, we create an isolated namespace for each module, and only allow cross-module loading to occur in a controlled and well-defined way.
What do we mean by a dependency? Simply, it is a set of assumptions made by a program or block of code about the environment in which it will run. For example, a Java class may assume that it is running on Java 5 and that it has access to the Apache Log4J library. These assumptions are implicit, meaning we don’t know they exist until the first time we try to run the code in an environment where they are false. In other words, we don’t know (except by disassembling) that the code assumes it is running on Java 5 until we first try running it on Java 1.4 and watch it crash. Likewise we don’t know that it makes calls to Log4J, until the first time we run it without Log4J on the classpath and see a NoClassDefFoundError.
Therefore we have no real idea whether a class will work in a particular environment except by trying it out. Even then we may not immediately discover problems, because they don’t occur until a particular execution path is followed.
By contrast, OSGi makes us provide explicit, declarative dependency information. By adding headers to the JAR’s MANIFEST.MF, we can specify exactly what environments that JAR will run in:
Import-Package: org.apache.log4j
Bundle-RequiredExecutionEnvironment: J2SE-1.5
We then have a framework that is responsible for validating and loading this as an isolated module. If we install this JAR on Java 1.4, it will not “resolve”, and we will get a clear message telling us why not. If we install this JAR in an application that does not have a Log4J module already installed, then again we will get an explicit error message explaining what is missing. Because of these constraints, we can be confident that if a properly constructed bundle resolves, it can never throw a ClassNotFoundException or a NoClassDefFoundError (well, almost… there is still the issue of code that does dynamic class loading using Class.forName() or similar… there are different mechanism to handle this).
Even better, we can do all of this dependency checking statically, long before actually deploying or running an application. All of the dependency information is visible, in declarative form, right in the MANIFEST.MF. This is perfect for tools, whicht can help us to construct applications that never fail due to a missing JAR. We can also take a third-party module and immediately identify whether it will work in our application, and if not what other module we need to obtain in order to make it work.
This just scratches the surface, but I hope it at least answers Chris’ question “Why would you want or need a module system?”. Simply, without a module system, the construction and deployment of Java applications is just too error-prone. Java programmers have been coping with pain — with “JAR hell” and “classpath hell” and hidden dependencies — for more than ten years. We’ve got so used to it that we don’t consciously notice the pain any more. But you damn well notice when the pain goes away!
Incidentally, all of the above is an argument for using a module system, not necessarily for using OSGi. There are other module systems around — such as the one in NetBeans platform, and HK2 in Glassfish V1 and V2 (which is largely replaced by OSGi in V3), and JSR 277 (if it is ever built), and the old Eclipse module system (which was replaced with OSGi around four years ago). If you ask any politician whether it’s good to use your vote, they will unanimously agree that it is, and then they explain why you should use it to vote for their party. I will leave it for others (or myself in a later post) to explain why you should use OSGi rather than any of the other module systems available for Java.
Chris Aniszczyk:
I think you’re well aware of one of PDE’s mottos Neil… “managing your classpath is so 90’s”
June 6, 2008, 4:20 pmdavid:
Hi Neil,
thx for this great introduction to OSGi, after those scary results on the java.net vote (http://dlinsin.blogspot.com/2008/05/osgwhat.html) this introduction just came in time!
June 6, 2008, 4:54 pmNeil Ellis:
Nice intro Neil, btw. love your book in progress, I think another heavily overlooked value from OSGi is a means to clearly define component lifecycles in non J2EE systems. How much a pain in the bum it is to normally guarantee that a component is automatically started at application startup etc.
And then again another huge value is the registry how many times do we write the equivalent of ‘plugins’ for our systems with an adhoc form of registry.
I think these are pretty neat features for single JVM code, of course that then always leads onto the question … but what happens when we distribute
June 6, 2008, 5:42 pmmohbana:
very nice introduction, i didn’t fully understand it either or rather didn’t “get” it. Is this in your OSGI book? if not, you should include it.
June 6, 2008, 11:26 pmEmanuele Ziglioli:
Hi Neil,
spot on, as you’re always are. OSGi definitely seems great and is gaining more and more traction. But from an Eclipse perspective, what has it really added to the platform? I’ve read your excellent paper about the difference between Eclipse extensions and OSGi services. And since no one uses services in Eclipse, still the question remains: what has OSGi given to Eclipse, since no one seems to use its features? I guess, mainly a standard way of defining dependencies, and a lot of confusion, which is spread across plugin.xml and MANIFEST.MF.
June 7, 2008, 10:33 amMike Greifeneder:
One of my first impressions when I discovered OSGi was that finally we have got something which was missing a long time. A feature that some operating systems offer for ages. A wide range of installed libraries in different versions and encapsulation of each application from others. As we cannot use native OS libraries, we need this within a VM. With OSGi we get an environment, where each application doesn’t need it’s own VM and libraries anymore. It can run within the same VM in a “definied” environment of profiles, packages and services. It’s really very promising what we can build on top of OSGi.
June 7, 2008, 12:54 pmGuy Mac:
What are the cool features? Why would I care about dynamic loading of libraries if I’m not writing an application server or plugin-extensible application? If my application launch script explicitly defines the classpath to jars that it provides, why would I care about a totally different way to define static dependencies? What is the equivilant, if any, of OSGi in the c/c++ world? How does it relate to package management systems such as Debian’s apt/dpkg?
June 7, 2008, 7:32 pmTim Goeke:
The only problem is that OSGi does not solve classpath hell and introduces a lot of complexity. Just take a quick look at any OSGi forum and count the number of times people are asking about class loading errors.
Many JARs such as the WebLogic 8 client jars are almost impossible to use efficiently. If you want to create a library bundle, you have to export the whole thing and then use wild card imports which are not documented very well.
Then, there’s the whole testing issue. OSGi introduces a new layer in your code, so there’s the issue of mocking the framework and testing your code in integration with other systems. There have been hints that a new testing framework is coming, where is it? And why is testing seemingly not considered.
There are a half dozen other problems but it all boils down to this: do the benefits outweigh the problems? I think for most people, the answer is a quick “no” and they move on and just build their application with a DI framework such as Spring.
DI provides most of the best of OSGi without the crazyness. Most current applications, including Eclipse, do not even use the hot update feature, rather you restart your application.
So you have added complexity and no real benefit. Is OSGi a paper tiger?
June 8, 2008, 4:12 amNeil:
Tim,
To take your points in turn:
Class loading hell does exist in OSGi, but at least 99% of the time it is caused by “legacy” libraries that try to do internally some of what OSGi does. A classic example is Jakarta Commons Logging, which attempts to dynamically bind to a logging implementation by searching through the classpath, which does not work well under OSGi becase it makes some false assumptions about how classloaders work.
The idea of commons-logging was actually reasonable when it was invented, because Java lacked the concepts of modularity and late binding. Unfortunately many other libraries have also been forced down the path of building in such mechanisms.
These problems are inevitable as any module system gets more and more ubiquitous. People want to use every library they’ve always used, and that includes all the ones using broken hacks that only ever worked in standard Java thanks to good luck and a following wind. However I’m confident they will mostly be ironed out in the next year or two.
Regarding your WebLogic client JARs problem, I don’t have any personal experience with WebLogic so you would be better off reporting whatever problem you have via the OSGi mailing list (http://bundles.osgi.org/mailman/listinfo/osgi-dev) or to your BEA/Oracle support reps. I’m sure they can be resolved.
Testing is orthogonal to OSGi, just as it is to J2EE and other specifications in the Java space. You won’t find a chapter about testing in the Servlet specification, for example. But that doesn’t mean it hasn’t been considered. True, people do need guidance on how to do it, and I intend to include some in my book. The Spring-DM project also includes some very useful testing tools.
Of course, I believe the benefits do outweigh the costs many times over. DI solves a different set of problems from OSGi — as proven by SpringSource who are building their application server platform out of Spring and OSGi.
Eclipse does still require restarts, sadly… this is simply because it has a lot of legacy baggage from its pre-OSGi days. Again, this situation will improve with time. You can already do most updates in Eclipse without restarting, and the warning that asks you to restart is (in my opinion) excessively conservative.
Your final rhetorical question reminds me of a Fox News headline (c.f “Is Bush best president ever?” )
Regards, Neil
June 8, 2008, 10:22 pmSerge:
very nice intro. Would the .net community benefit from something like osgi or is this already in the .net framework?
June 10, 2008, 1:15 amNeil:
@Serge: many thanks. Regarding OSGi on .NET, there’s an interesting paper by Clement Escoffier, Didier Donsez and Richard Hall, entitled Developing an OSGi-like Service Platform for .NET.
Their conclusion seems to be that it’s not really possible, since .NET doesn’t offer the ability to unload assemblies from within an application domain, and inter-application domain communication has poor performance.
However, the paper is from 2006, so the situation may have changed since then. I don’t really keep up to date with .NET.
June 10, 2008, 7:33 amkeesp:
Hi Neil,
good article! Do you happen to know if if there is any working progress on equinox and JCE integration? I need to add a security provider (third-party, which is contained in a jar) in an equinox bundle and as this needs to be done in the system classloader, it is very difficult for me to see what the best way is to approach this. I prefer to avoid loading the jar file that contains the provider as a JAVA classpath entry (which would work), but this would mean that I somehow need to inform the system classloader where the provider can be found. Any best practices to achieve something like that?
Thanks
Kees
June 24, 2008, 10:08 amMuthu:
Hi Neil,
I have been reading a bit about “distributed OSGi” initiative. Was just wondering how high-availability aspect is addressed when OSGi goes distributed (inter-container communications, cross-container service invocation etc etc)? No doubt, when OSGi is poised to become a “distributed” platform availability (either through popular mechanisms like redundancy) would be what enterprise or consumer-oriented applications would look for.
Any ideas on how this aspect is addressed? Not too odd an idea that this could also be one area which your book could cover
Thanks Muthu
July 18, 2008, 7:51 amToni Menzel:
Speaking of testing for osgi - tim was asking about that - there is a young effort at OPS4J.org: PaXDrone. It is all about making it cheap to invoke framework,run some code and clean up later which is perfect for integration testing. Have a look here http://wiki.ops4j.org/confluence/display/ops4j/Pax+Drone.
cheers, Toni
July 18, 2008, 10:30 amAditya:
So why we should go for OSGi over other module system ? What r the pros n cons ?
December 18, 2008, 7:03 amToni Menzel:
@Aditya: which other module systems ? JSR 277 is dead (pro osgi now), old eclipse module system is dead (thats why its old, they use osgi now), HK2 is dead (switched to osgi). Convinced ?
January 13, 2009, 3:06 pmAditya:
Covinced !!..to be frank no..shud i go for OSGi just coz others are dead.Well there is Netbeans platform,I am not sure if there are any others.Yet like we say we should go for JAVA compared to C++ in certian situations in a similar fashion is there an answer for prefering OSGi over Netbeans.Though we had been usign OSGi ,yet I always doubt why OSGi n why not some other module system ( if there is any live )?Moreover why all these module systems became dead..may be the reason for their death might be a good answer which I would like to know.Folks please provide ur insights ?
February 18, 2009, 5:57 amNeil:
@Aditya: OSGi is the best, without a doubt.
Eclipse used to have its own module system, it converted to OSGi. Glassfish used to have its own module system, it also converted. Netbeans still has its own module system, but I guess your question will be answered when it also converts to OSGi. I believe that will happen quite soon.
February 18, 2009, 7:44 amHands-on OSGi and Modular Web Applications – Part I – Toes First » Source Allies Blog:
[...] http://neilbartlett.name/blog/2008/06/06/what-is-osgi-for/ – this one talks about the problem space [...]
October 20, 2009, 2:26 am