Java in Notes/Domino Explained: More in depth on classloaders


When you run an agent in Notes/Domino the agent class is loaded into memory by a classloader (for more information about what a classloader actually is please refer to this previous post). In the case of Notes and Domino this classloader is the AgentLoader class from the lotus.domino package (that is lotus.domino.AgentLoader).
This is also the classloader which is responsible for loading all the other classes your agent and auxillary classes may use during their execution.

You can query any class in Java about which classloader actually loaded the class into memory. You do this using the getClassLoader()-method of the class instance as shown below:

import lotus.domino.*;

public class JavaAgent extends AgentBase {

   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         // get class for the current object
         Class clazz = this.getClass();

         // get classloader that loaded this agent class
         ClassLoader cl = clazz.getClassLoader();
         System.out.println("Default ClassLoader: " + cl.getClass().getName());

      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

The output of this agent in the Java Debug Console will be:

Default ClassLoader: lotus.domino.AgentLoader

In Java every classloader may be part of a hierarchy and hence a classloader may have a parent classloader. The only classloader that never has a parent is the top classloader called the bootstrap classloader which is installed by the JVM when initialized. The bootstrap classloader cannot be substituted in the case of Notes/Domino.

If a classloader is asked to load a class and it cannot find it, the request is delegated to the parent classloader if one is present. To demonstrate this we first need an extra classloader – let’s write our own classloader that loads classes from outside the Notes/Domino system classpath. Writing ones own classloader is actually VERY simple. You simply have to extend the java.lang.ClassLoader class and override one method. The code below shows how to write a classloader that loads classes from the root of the C-drive:

import java.io.*;

public class MyClassLoader extends ClassLoader {
   public MyClassLoader() {
      super();
   }
   public MyClassLoader(ClassLoader cl) {
      super(cl);
   }

   public Class findClass(String name) throws ClassNotFoundException {
      try {
         // use utility method to load class bytes from
         // the root of the C-drive
         byte[] b = this.loadClassData(name);
         return this.defineClass(name, b, 0, b.length);
      } catch (Exception e) {
         throw new ClassNotFoundException();
      }
   }

   private byte[] loadClassData(String name) throws Exception {
      // open a the requested file and read the bytes into an array
      InputStream in = new BufferedInputStream(
                       new FileInputStream("c:\" + name + ".class"));
      byte[] raw_classdata = new byte[1024];
      int length = in.read(raw_classdata);

      // since the byte array must be exactly the
      // correct length we copy the bytes into a
      // new array
      byte[] classdata = new byte[length];
      for (int i=0; i<length; i++) {
         classdata[i] = raw_classdata[i];
      }

      // return the byte array
      return classdata;
   }
}

As you can see it is quite easy and only the imagination sets limits as to where you could load classes from in your agents. You could actually write a classloader to load classes from a central class repository using some network protocol of your choosing.

Since we need some classes to load using our new classloader I have written two very simple classes called OutsideClasspath and OutsideClasspath2 as shown below. The classes should be compiled using the javac-tool or using Eclipse and placed in the root of your C-drive. If you do not want to compile the classes yourself you can download them using the links below.

public class OutsideClasspath {
   public OutsideClasspath() {
      ClassLoader cl = this.getClass().getClassLoader();
      System.out.println("Constructor of OutsideClasspath class " +
         "(loaded by: " + cl.getClass().getName() + ")...");

      // load another class
      OutsideClasspath2 oc2 = new OutsideClasspath2();
      System.out.println(oc2.echo("Hello from OutsideClasspath"));
   }
}

public class OutsideClasspath2 {
   public OutsideClasspath2() {
      ClassLoader cl = this.getClass().getClassLoader();
      System.out.println("Constructor of OutsideClasspath2 class " +
         "(loaded by: " + cl.getClass().getName() + ")...");
   }

   public String echo(String echo) {
      return "Echo: " + echo;
   }
}

Using our new classloader is also simple. We simply instantiate it and pass the bootstrap classloader as the parent classloader in the constructor.

import lotus.domino.*;
import java.util.*;

public class JavaAgent extends AgentBase {

   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         // get bootstrap classloader
         ClassLoader cl = this.getClass().getClassLoader();
         System.out.println("Default ClassLoader: " + cl.getClass().getName());

         // get classloader of the agent and show class name
         System.out.println("Agent class loaded by: " +
                this.getClass().getClassLoader().getClass().getName());

         // create our own classloader passing the
         // bootstrap classloader in the constructor
         cl = new MyClassLoader(cl);
         Class c = cl.loadClass("OutsideClasspath");
         Object o = c.newInstance();

         // create a java.util.Date object using our own classloader
         // to demonstrate the hierarchical nature and delegation
         // model of classloaders
         Class clazz_date = cl.loadClass("java.util.Date");
         Date d = (Date)clazz_date.newInstance();
         System.out.println(d);

      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

The output of the agent is as follows:

Default ClassLoader: lotus.domino.AgentLoader
Agent class loaded by: lotus.domino.AgentLoader
Constructor of OutsideClasspath class (loaded by: MyClassLoader)...
Constructor of OutsideClasspath2 class (loaded by: MyClassLoader)...
Echo: Hello from OutsideClasspath
Sat Apr 01 22:13:49 CEST 2006

The output demonstrates a couple of things:

  • The agent class is loaded by the bootstrap classloader.
  • That we are able to load classes using our own classloader what is we can load and instantiate an instance of the OutsideClasspath class.
  • That classes use the classloader that loaded them to load any classes they need. This is demonstrated by the fact that the OutsideClasspath2 class is automatically loaded by MyClassLoader when the OutsideClasspath class creates and instance of it.
  • That classloaders delegate the loading of classes to their parent if they cannot find the class. This is demonstrated by the fact that the request to load java.util.Date is delegated to the bootstrap classloader.

I hope this post has demonstrated the power of classloaders and how you can easily write one that suits your purpose.

Please note:

  • Even if we had not supplied the bootstrap classloader in the constructor of the MyClassLoader class a ClassNotFoundException exception would not have been thrown at runtime when asked to load the java.util.Date class. This is because the bootstrap classloader is always asked as a last resort. Setting the parent classloader is only important if you build your own hierarchy of classloaders as is the case in J2EE application servers.
  • The MyClassLoader class should be part of the agent if you decide to try the code out for yourself in Domino Designer.

Java in Notes/Domino Explained: Classloading


When doing Java development you write your code in classes and hence deal with classes. Each Java class in your source code is compiled into a binary class file.

JavaAgent.java (text) -(compile)-> JavaAgent.class (binary)

The compilation is done automatically for you when you write Java in Notes/Domino or if programming in Eclipse. It does however still help to be aware of what actually goes on under the covers when your agent is run.


“The process of finding the binary classes and loading them into memory is called classloading…”

The fact that each class is compiled into a separate file means that a Java agent consisting of 4 classes will contain 4 compiled classes. Since all the code isn’t combined into a separate file (like an EXE-file on Windows) the Java Virtual Machine (JVM) needs to locate these compiled classes at runtime and load them into memory so it can use them hence the word classloading.

The default classloader in Notes/Domino (lotus.domino.AgentLoader) is capable of loading classes from the classpath of the current agent which consists of:

  • Classes that are part of the agent itself.
  • Classes and JAR-files added to the agent using the “Edit Project”-button.
  • Java Script libraries added to the agent using the “Edit Project”-button.
  • Classes present on the Notes/Domino system classpath.

See the post “Managing external Java dependencies in Domino Designer” for more information on the Notes/Domino system classpath.

The exceptions

There are three exceptional situations that can and do occur frequently when working with Java in relation to classloading. These are the java.lang.ClassNotFoundException, the java.lang.NoClassDefFoundError and the java.lang.UnsupportedClassVersionError. Below are short excerpts from their explanation in the JDK. When programming in Notes/Domino you will probably encounter the first two at some point. The third I do not wish upon you… ๐Ÿ™‚

ClassNotFoundException Thrown when an application tries to load in a class through its string name using the forName method in class Class.
NoClassDefFoundError Thrown if the Java Virtual Machine or a ClassLoader instance tries to load in the definition of a class (as part of a normal method call or as part of creating a new instance using the new expression) and no definition of the class could be found.
UnsupportedClassVersionError Thrown when the Java Virtual Machine attempts to read a class file and determines that the major and minor version numbers in the file are not supported.

The ClassNotFoundException exception is encountered when you try to dynamically instantiate a class using a string name (java.lang.Class.forName()) and the class cannot be found in the classpath. This is a checked exception you must catch when doing a Class.forName() call.

The NoClassDefFoundError is raised if you use a class in your agent which in turn uses another class that cannot be loaded at runtime. The most common scenario for this is when using third-party library consisting of multiple JAR-files. In this case you may only have included one of the JAR-files in your agent to make it compile. The agent may compile but if classes in the included JAR-file uses classes in a second JAR-file, that wasn’t included in the agent, this error will occur.

The UnsupportedClassVersionError is less likely to occur if you are on Notes/Domino 6.x or 7.x but does occur quite a lot if you are using third-party libraries on older Notes/Domino versions. The binary Java class format changes slightly over time (much like the On-Disk-Structure of Notes databases) and since the JVM cannot be forward compatible it needs a way to tell you that you are trying to load a class into the JVM it doesn’t know how to handle (the version of the class format is wrong). The way around this is to try and compile the classes using a Java 1.1.8 compiler or using “classic” mode with newer compilers.

So why is this important?

To really appreciate the concept of classloading you need to realize that:

  1. Java classes are not loaded into memory before actually used.
  2. Java supports dynamic classloading.
  3. You can write your own classloader.
  4. Classloaders in Java are hierarchical and thus can delegate classloading to other classes.

You don’t have to worry too much about item 1. I’ll explain item 2 and 3 in detail in subsequent posts to avoid making this post too long. Item 4 deals with delegation of classloading in case a specific classloader cannot find a given class. Suffice it to say that these four concepts allows the JVM to conserve memory and you to write very powerful and highly dynamic agents/classes.

All this fuss about classloading might seem overly complex but consider the fact that many of the classes that you use in your Java agent aren’t some you have written. They are part of the Java Development Kit (the java.lang packages) or part of third-party libraries that you program and execute your agents against.

If there were no classloaders you could not have dynamic classloading and all classes would need to be loaded into memory before starting an agent. This could/would take up a lot of memory and be highly inefficient.

Anyone who attended AD212 (Introduction to DXL) at Lotusphere?

While reading through the presentation from the “Introduction to DXL”-session (AD212) by Dick Annicchiarico from Lotusphere 2005 I saw the below screenshot.

Am I the only one not having the “Viewer” or “Transformer” menuitems in my “Tools…DXL” menu in Domino Designer? I only have the “Exporter” menuitem… ๐Ÿ™

Just wondered if anyone who sat in on the presentation knows whether you have to install “something extra” to see the menuitems.

Java in Notes/Domino Explained: Packages

.quote {
font-style: italic;
font-size: 1.1em;
font-weight: bold;
width: 300px;
}
.quote_left {
float: left;
padding-right: 3px;
}
.quote_right {
float: right;
padding-left: 3px;
}


In Java all classes are part of a package. This goes for the classes you write, classes from third-party libraries and the classes from the Java Development Kit (JDK). Think of a package in the words most litteral meaning – something that groups together one or more objects to make them easier to handle. The objects that we package are classes (and interfaces).

All classes for Notes/Domino are grouped together in a single package called “lotus.domino”. Since the name of a class is actually the name of the package combined with the name of the class, the “real”, fully qualified, name of the Document class (the equivalent of the NotesDocument class from LotusScript) is actually “lotus.domino.Document”. Apart from the “lotus.domino” package you will routinely use classes from the core Java development kit (JDK). The packages from the JDK all start with “java”.


“If you do not explicitly put a class in a package it will be part of what’s called the default package”

If you do not explicitly put your classes in a package they will be part of what’s called the default package. For most programming in Java in Notes/Domino there’s no need to worry about putting your classes in a package – they mostly come into play when writing classes for distribution (e.g. designing API’s).

Importing

When referring to classes in Java you need to use the fully qualified name (e.g. lotus.domino.Document). This will quickly become very cumbersome so you can tell the compiler which packages you use in your code and it will automatically insert the package names at compile time.

In Java-speak this is called “importing a package”.


“Importing packages” is telling the compiler which packages you are using in your code…

For instance by importing the lotus.domino package in Java agents you can simply refer to the Document class instead of the lotus.domino.Document class. This can save quite a lot of typing, it makes the code less verbose and easier to read.

The two agents below are exactly the same from the compilers point of view. The first example shows an agent that import the entire lotus.domino package using the asterix notation (line 1 in bold):

import lotus.domino.*;

public class JavaAgent extends AgentBase {
   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext ac = session.getAgentContext();
         Database db = ac.getCurrentDatabase();
         View view = db.getView("($All)");
         Document doc = view.getFirstDocument();
         while (null != doc) {
            // do stuff with the document...
            doc = view.getNextDocument(doc);
         }
      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

The second example shows the same agent but this time I didn’t import any packages. This means that I have to prefix each class with the name of its package. It is more verbose and harder to read.

public class JavaAgent extends lotus.domino.AgentBase {
   public void NotesMain() {

      try {
         lotus.domino.Session session = getSession();
         lotus.domino.AgentContext ac = session.getAgentContext();
         lotus.domino.Database db = ac.getCurrentDatabase();
         lotus.domino.View view = db.getView("($All)");
         lotus.domino.Document doc = view.getFirstDocument();
         while (null != doc) {
            // do stuff with the document...
            doc = view.getNextDocument(doc);
         }
      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

Please note: The java.lang package is implicitly imported – always!

Drawbacks

However great this is, it also means that you must be aware when this approach will give you problems. Consider using and importing two packages that both contain a class of the same name. In this situation the compiler can’t figure out which class to use (from which package) so it wont compile the code. The solution here is to fully qualify the offending classes with the package names to explicitly tell the compiler which class you are referring to.

Shooting with smaller bullets

Apart from importing an entire package at a time by using the asterix notation (lotus.domino.*) you can also import a single class at a time. You do this by substituting the asterix with the name of the actual class (lotus.domino.Database, lotus.domino.Document etc.). This approach is the one automatically used in Eclipse and it helps alleviate the problem of class name collisions described above. The example above would then look like this:

import lotus.domino.AgentBase;
import lotus.domino.Session;
import lotus.domino.AgentContext;
import lotus.domino.Database;
import lotus.domino.View;
import lotus.domino.Document;

public class JavaAgent extends AgentBase {
   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext ac = session.getAgentContext();
         Database db = ac.getCurrentDatabase();
         View view = db.getView("($All)");
         Document doc = view.getFirstDocument();
         while (null != doc) {
            // do stuff with the document...
            doc = view.getNextDocument(doc);
         }
      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

If you do not care about creating your own packages this is all you need to know about packages. If you do care about creating your own packages stay tuned for the next post on packages.

Article: Customizing the look and feel of Domino Web Access 6.5

While searching for the article on customizing forms etc. in Domino Web Access that was mentioned during one of the sessions at Lotusphere I found an article on customizing the look and feel of DWA. Just posting so I know where to find it…

Lotus developerWorks: Customizing the look and feel of Domino Web Access 6.5

Update (3 April 2006): I finally found the article (Manipulating data in Domino Web Access).

Loving policies – especially mail settings


Starting with Domino 7 the policy document has six fields to policy setting document. The sixth settings document called mail settings was introduced in the Domino 7 release and allows you to control many of the mail and calendar settings of your users centrally.

The sweet thing is that only the server, and hence the Domino Directory, needs to be on release 7 for you to exploit this functionality.

The below text is excerpted from “Creating a mail policy settings document” in the Domino 7 Administration help file:

"The effective mail policy for a user is stored in that user's mail database in the calendar profile. The administration process writes the mail policy to the calendar profile. The administration process checks for mail policy updates every 12 hours, and when necessary updates the user mail files according to the changes in the mail policy settings document."

As stated it is the administration process that applies the settings so it doesn’t matter that you have a mixed environment of Notes client versions. I have succesfully used this approach in a combined 6.x, 6.5.x, 7.x customer environment.

It can be a real timesaver if you need to deploy common calendar alarm settings, letterheads etc. across your organization. The settings will even be applied even though the user doesn’t log on.

Re: How are roaming id-files encrypted?

Well the message from Lotus Support on the roaming id-file question (see the original post “How are roaming id-files encrypted?” for the whole story) is that no one seems to know. The technician in charge of my PMR called me and it seems like none of the people, including the support guys in Ireland, knows what’s up with the “double encryption”. He did however say that he got in contact with a developer in Germany via IM who would take a look at the actual Notes/Domino code on Friday.

The technician from Lotus Support would get back to me on Friday or Monday so we’ll see what happens. Lets just hope the answer isn’t hard to come by because it is security by obscurity

Show ‘n Tell Thursday: Setting the Content-type from a Java agent (30 Mar 2006)


Suppose you are all into this new AJAX thing and you need to output XML from your agent. If you write the agent in LotusScript you simply output the Content-type header using a Print statement as the first output to the browser.

Print "Content-type: text/xml"

Setting the Content-type from a Java agent however seems to drive many people crazy. They seem to set the Content-type header correctly but it fails to manifest itself in the browser (the XML comes out wrapped in Domino HTML). This happens since they fail to understand what they actually need to do. The Content-type needs to go on its own line hence you need to insert a linebreak after the Content-type as the below example shows.

import lotus.domino.*;

public class JavaAgent extends AgentBase {

  public void NotesMain() {

    try {
      Session session = getSession();
      AgentContext agentContext = session.getAgentContext();

      java.io.PrintWriter pw = this.getAgentOutput();
      pw.println("Content-type: text/xml");
      pw.println("<?xml version="1.0" ?>");
      pw.println("<names>");
      pw.println("<name first="Mikkel" last="Heisterberg" />");
      pw.println("</names>");

    } catch(Exception e) {
      e.printStackTrace();
    }
  }
}

The key (highlighted in bold above) is to use the println()-method of the java.io.PrintWriter instead of simply the print()-method or the write()-method.

I said it wouldn’t be a long post…

Re: Is the lack of Java skills in the Notes/Domino developer community the Achilles’ heel of IBM?

First of all, there is no doubt that the followers of the Notes/Domino platform are very passionate about the platform, where it is and where it is going. This is a good thing. We saw evidence of this at Lotusphere as well. That being said I think that the discussion in the thread on Eds blog has probably ventured somewhat off-topic into a discussion on development platforms and a discussion on strongly vs. loosely typed programming languages. Probably a discussion best left in another forum… ๐Ÿ™‚

Worse though is the fact that some just refuse to accept the change that is inevitable in this industry. The move to Java, or another new programming language, in the future is inevitable. However no one, certainly not myself, dispute the fact that LotusScript is the de facto programming language of the Notes/Domino platform. We all know IBM makes mistakes once in a while but I certainly do not think they would saw of the branch they are sitting on by deprecating the LotusScript language. As I wrote in the original post: “LotusScript will probably be supported from now to the demise of the platform.”

I think Ed explains it very well and clearly in one of his comments (@61) to the discussion thread:

"So yes, the userbase wants to keep Lotuscript as the default language. And to be clear, Lotuscript doesn't go away, change, or deprecate in the Notes "Hannover" release. And some of the ideas here on how to extend the value of a Lotuscript skillset are interesting and make sense. But a view of the larger IT market indicates that the need to incorporate other languages -- AND development tools -- with the Notes environment of today is an even better bet."

Please note here that not deprecating LotusScript is in no way the same as extending it and having the LotusScript API provide the same level of functionality that I think the Java and JavaScript API will provide in the Hannover+ timeframe. LotusScript will survive as a Notes/Domino language but Java and JavaScript will, even though adoption seems to be painfully slow, become the new Lingua franca of the IBM World.

We, as the ever-passionate Notes/Domino community, have to overcome our fear of change and accept that application development has, and continues to move away, from the proprietary, all-in-one-NSF, single-platform development style that has been prevalent for so many years. This doesn’t mean that you have to move away from this type of development and a lot of applications will continue to benefit from the RAD (oops – I said it!) environment of Notes/Domino.

It is just that the majority of new applications being developed, or existing ones getting a tune-up, needs to look outside the Notes/Domino platform for information. Whether this be a simple lookup in a RDBMS, a call to a web service or a series of complicated calls to an ERP system doesn’t really matter. The can of worms is open and to think that the complexities of system integration will disappear in the foreseeable future is just mad. Having a non-proprietary programming language with widespread industry adoption makes this kind of system integration a lot easier and the benefits should be clear. For one we can avoid the proliferance of custom LSX modules wrapping C/C++ API code to provide access to system X, Y and Z.

The shift towards composite applications, including the kind you can build in Hannover, will be all about system integration – this is why Java adoption is a necessity. Not for the traditional kind of applications but for the new stuff.

Instead of bitching and moaning I think we, as a community, should be thrilled and instead focus on the fact that IBM has seen this shift coming, and that Notes/Domino with the Hannover client is embracing the future and providing the tooling to leverage existing investments in other platforms. Lets embrace the future not hide away from it refusing to accept the change that is coming.

The need as a developer to skill-up by embracing Java might seem unjust and a burden but the alternative would be even worse. You, and the customer you serve (whether the customer be internal or external), would be left behind with a dead, proprietary, used-to-be-great platform. Instead you are getting a heads-up while there is time.

Happy coding… ๐Ÿ™‚