Test agents in Eclipse by extending AgentBase

I continuously get questions on how I do Java agent development and I’m happy to answer them as I hope that some of the answers I provide means that more and more Domino developers see Java as a strong alternative to LotusScript for background processing. Most times the approach I recommend is a mock object approach that I wrote about waaaaaay back in 2006 (did I really write that 10 years ago?!?!?).

If / when you want to read the posts please start from part no. 1. Here’s a link to all 5 parts:

The approach doesn’t handle importing the code back into Domino Designer but it does allow you to “mock” the Notes side of the API and allows you to write, test and debug your agents in Eclipse. Then when they’re completely done you can import it back into Notes without any changes. This is the approach I would choose and that I still use when writing Java agents here 10 years later.

Hope this helps but if it doesn’t please let me know.

Java in Notes/Domino Explained: On Java Security and how it relates to Notes/Domino


I get so many questions on Java security and how it all fits together that I finally decided to blog it instead of simply answering over and over again. So here goes…

Security was built into Java from its inception and everything in a Java Virtual Machine (JVM) runs under the control of a security manager. Now the security manager may be very lax or it may be very restrictive as the one for Java applets or Notes/Domino agents is. A security manager is installed into the JVM when it’s constructed meaning either from the command line or, in the case of Notes/Domino, when a JVM is created to run agents, XPages or the like. A security manager may decide that a certain operation (method call) is “powerful” and hence will require the code executing the operation to be allowed to do so.

The way this is done in Java is by the operation itself (the method you are calling e.g. “write file” or “do stuff with reflection”) asking the security manager whether the code calling it has a given permission that it supplies to the security manager. The security manager may either decide that it does and hence return quietly or it may decide that the calling code does not have the required permission. To signal the latter it will throw a java.security.AccessControlException which you’ve seen if you’re reading this.

So how do we grant these permissions?

In Java permissions are controlled in the java.policy file located in the file system under <notes>/jvm/lib/security or <domino>/jvm/lib/security. I will not go into detail on the syntax of this file as resources are readily available on the web but suffice to say that different locations (be those file based or network based) are granted different sets of permissions (for more info see http://docs.oracle.com/javase/6/docs/technotes/guides/security/PolicyFiles.html). By default jar-files in jvm/lib/ext, the XPages runtime, OSGi bundles etc. are granted all permissions possible (java.security.AllPermission) whereas other code is granted a limited set of permissions. So if you need to run code that requires a different set of permissions you edit this file and restart the server/client. Now often times simply editing this file is not an option. Either you do not control the server, you cannot edit it for all clients or your security guidelines tell you just can’t do it. In either case editing the file is seldom the entire solution. And you need to understand why. Before showing you some code let me explain the scenario I’m going to use below first.

The scenario is that Joe Notes Developer needs to use reflection (wikipedia) to change the accessibility of a method from private to public (yes it’s possible) to override the private restriction set on the generate() method of the Test class (see below). To do this he uses a method like the one below:

public void doAccessible(Object obj) throws Exception {
   Method[] methods = obj.getClass().getMethods();
   for (int i=0; i<methods.length; i++) {
      Method m = methods[i];
      m.setAccessible(true);
   }
}

The class he needs to affect is likewise simple (he needs to make the generate() method public):

private class Test {
   private String uuid = null;

   public Test() {
      this.generate();
   }

   public String getUUID() {
      return this.uuid;
   }

   private void generate() {
      this.uuid = UUID.randomUUID().toString();
   }
}

To start of Joe Notes Developer creates a Java agent in Domino Designer and sticks the below code in NotesMain of the agent:

Test t = new Test();
System.out.println(t.getUUID());
doAccessible (t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

But oh no!! Joe gets the dreaded AccessControlExeption as the code in the Java agent may not use reflection this way.

java.security.AccessControlException: Access denied (java.lang.reflect.ReflectPermission suppressAccessChecks)
   at java.security.AccessController.checkPermission(AccessController.java:132)
   at java.lang.SecurityManager.checkPermission(SecurityManager.java:544)
   at COM.ibm.JEmpower.applet.AppletSecurity.superDotCheckPermission(AppletSecurity.java:1449)
   at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1617)
   at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1464)
   at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:118)
   at ReflectionUtil.doAccessible(Unknown Source)
   at JavaAgent.NotesMain(Unknown Source)
   at lotus.domino.AgentBase.runNotes(Unknown Source)
   at lotus.domino.NotesThread.run(Unknown Source)

Nothing is holding Joe back though so he puts the reflection code into a ReflectionUtil class which he puts in a jar-file in jvm/lib/ext as he has heard that this location allows for this kind of funky stuff. Using the ReflectionUtil instead of a class method from his agent is like this:

ReflectionUtil l = new ReflectionUtil();
Test t = new Test();
System.out.println(t.getUUID());
l.doAccessible(t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

Arrrghhhh no!! He still gets an AccessControlException. But how can this be?? He did put the code in the special jvm/lib/ext folder which is granted AllPermissions. Hmmm…

The reason is that security in Java is layered and it’s the layer or “context” in Java parlance initiating the call that decides which permissions are assigned. This is why simply making a call from a Java agent in Notes/Domino to a Java library in jvm/lib/ext that need permissions other than the default set doesn’t just work. It doesn’t work because it’s the context of the Java agent which is in effect. So wouldn’t it be great if you could change the context? Well you can… 🙂 This is done using the java.security.AccessController class (docs).
The AccessController class is a nifty little thing. It works by you supplying an instance of the java.security.PrivilegedAction interface which is an interface with a single method called run. Inside the run-method you put the code requiring special permissions. Below is an example of using the AccessController class:

AccessController.doPrivileged(new PrivilegedExceptionAction() {
   public Object run() throws Exception {
      ReflectionUtil l = new ReflectionUtil();
      Test t = new Test();
      l.doAccessible(t);
      System.out.println("Completed...");
      return null;
   }
});

But wouldn’t you know it – this also fails! Of course the reason is quite obvious – I think – we cnanot just have developers go ahead elevating their persmissions themselves. As Paul Mooney would say – developers mus be controlled and kept down. It needs to be the privileged code meaning the code in jvm/lib/etc that needs to elevate the permissions. To do this we move the AccessController call info the ReflectionUtil in a special “doAccessiblePrivileged” method as this code can then decide to elevate permissions as it runs from a location with a higher security context. The code in the agent is therefor now like this (change in bold):

ReflectionUtil l = new ReflectionUtil();
Test t = new Test();
System.out.println(t.getUUID());
l.doAccessiblePrivileged(t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

And the code in ReflectionUtil looks like this:

public void doAccessiblePrivileged(final Object obj) throws Exception {
   AccessController.doPrivileged(new PrivilegedExceptionAction() {
      @Override
      public Object run() throws Exception {
         ReflectionUtil.this.doAccessible(obj);
         return null;
      }
   });
}

Now the code runs just fine as Joe is using code that knows that it might need to elevate the permissions so it provides a call doing just that (doAccessiblePrivileged).

Of course this can make it very difficult to just use a standard library from the web in Notes as it may not provide a way to elevate the permissions as no one ever saw that as a use case. In this case a wrapper API that elevates the permissions in jvm/lib/ext may be necessary to make it work.

Hope this explains it… 🙂

The example code is available for download including the required jar-file to put in jvm/lib/ext.

Show ‘n Tell Thursday: Cross compiling sidebar plugins for Notes 8.0.x and Notes 8.5 (7 August 2008)


I haven’t done a SnTT in months (actually the last one was back on 24 May 2007) so I guess it’s about time. This week it’s about cross compiling sidebar plugins for Notes 8.5 and Notes 8.0.2. Enjoy…

Since installing Notes 8.5 I have had a lot of trouble developing in Eclipse 3.4 for Notes 8.5 and having the plugins work in Notes 8.0.x as well. The problem showed itself as a NullPointerException when trying to load the plugin Activator and a “Viewpart is null” message in the Notes UI. Looking at the log trace showed a class incompatibility message (“bad major version at offset=6”).

So far I have been screaming my lungs out, developing in Eclipse 3.4 and building the plugins in a virtual machine with Eclipse 3.2 as I couldn’t get plugins to work otherwise. Now I finally found a solution that lets me develop in Eclipse 3.4 and having the plugins work in Notes 8.0.x and Notes 8.5.

The issue is that Eclipse 3.4 configured with Notes 8.5 as a target platform is using Java 6 and that Notes 8.0.x is using Java 5 which causes class format problems. The solution is to set the required execution environment in Eclipse 3.4 which will cause Eclipse to build correctly. Setting the required execution environment (J2SE-1.5) is done in the “Overview”-tab of the MANIFEST.MF editor as shown below.

Using the GUI is of cause just a shorthand for editing the manifest manually. As an alternative you can edit your META-INF/MANIFEST.MF file directly and add the following line:

Bundle-RequiredExecutionEnvironment: J2SE-1.5

Please note that this is not the default value when creating new plugins in Eclipse 3.4 so you’ll have to pay attention and make sure it’s set correctly. This is of cause only necessary if your plugins need to work on Notes 8.0.x as well as Notes 8.5.

Thanks to Pierre Carlson from IBM for pointers on this.

Show ‘n Tell Thursday: Showing a progress bar / joining threads (24 May 2007)


A fellow developer in the developerWorks forum had an issue where he wanted to load a lot of data and needed a progress bar to indicate progress. His problem was that the main thread terminated before his data loading was done and he needed help.

The solution is to use a data loading thread and Thread.join () the thread to wait for it to complete to avoid the main thread from terminating the agent. If you do not join the thread the main thread will terminate and abort the loading. The joining is bolded in the below code.

To test it out try running the below code with and without the l.join() line below commented out and see how the progress bar never reaches the end when the thread isn’t joined.

Also note that I’m extending NotesThread to avoid dealing with threads etc. in Notes and that the code is no where near production quality!

import lotus.domino.*;
import javax.swing.JDialog;
import javax.swing.JProgressBar;

public class JavaAgent extends AgentBase {
   private static int COUNT = 30;

   public void NotesMain() {

      try {
         // initialize
         Session session = this.getSession();
         AgentContext ac = session.getAgentContext();

         // start thread to similate loading
         Loader l = new Loader();
         l.start();
         for (int i=0; i<COUNT; i++) {
            System.out.println("i=" + i + " (" + session.getUserName() + ")");
            Thread.sleep(100);
         }

         // wait for progress thread
         l.join();

      } catch(Exception e) {
         e.printStackTrace();
      }
   }

   public static class ProgressBar {
      private JDialog dialog = null;
      private JProgressBar bar = null;

      public ProgressBar() {
         dialog = new JDialog();
         bar = new JProgressBar(0, COUNT);
         dialog.getContentPane().add(bar);
         dialog.setSize(250, 40);
      }
      public void visible(boolean flag) {
         dialog.setVisible(flag);
      }
      public void update(int i) {
         this.bar.setValue(i);
         this.bar.updateUI();
      }
      public void dispose() {
         if (this.dialog.isVisible()) {
            this.visible(false);
         }
         this.dialog.dispose();
      }
   }

   public static class Loader extends NotesThread {
      private Session session = null;
      private Database db = null;
      private View view = null;

      public Loader() throws NotesException {
         session = NotesFactory.createSession();
         db = session.getDatabase(null, "names.nsf");
         view = db.getView("People");
      }

      public void runNotes() {
         try {
            ProgressBar b = new ProgressBar();
            b.visible(true);

            for (int i=1; i<=COUNT; i++) {
               Document doc = this.view.getNthDocument(i);
               if (null != doc) {
                  System.out.println("Loaded person: " +
                     doc.getItemValueString("Firstname") +
                     " " + doc.getItemValueString("Lastname"));
               }
               b.update(i);
               Thread.sleep(500);
            }
            b.dispose();

         } catch (Exception e) {
            e.printStackTrace();
         }
      }

   }
}

Java in Notes/Domino Explained: Stacktraces


Not much information on Java in Notes/Domino per say in this post but simply a pointer to a nice article on reading stack and tread dumps which is a must for all programming in Java.

Remember that as of Java 1.4 (Notes/Domino 7.x) you can use Java classes access the elements of a stacktrace. You can use this to further diagnose the trace and make a more intelligent decision on how to handle it. Look for the StackTraceElement class in the javadocs.

Link to article: Of Thread dumps and stack traces ….

Java in Notes/Domino Explained: NotesException


The documentation on NotesException is a little scarse but basically it’s simply an exception like all the others and it inherits from java.lang.Exception via org.omg.CORBA.UserException. It inherits from org.omg.CORBA.UserException since it needs to be throwable when using remote sessions. NotesException has three constructors for you:

  • NotesException()
  • NotesException(int code, String message)
  • NotesException(int code, String message, Throwable t)

The second and third ones has an integer and a string argument which is the error code and error message as you know them from the LotusScript Error statement. The third argument of the third constructor is for throwing nested exceptions. Anyways – you can throw NotesException as you do with any exception using throw new:

try {
   ...do some stuff
} catch (NotesException e) {
   // throw new NotesException();
   // throw new NotesException(99999, "My own error code...");
   // throw new NotesException(99999, "My own error code...", e);
}

What is a little strange though is that the actual error codes for the NotesExceptions throws cannot be found in the NotesException class but is found in the lotus.domino.NotesError class. This means that if you like to test for “human understandable” error codes (e.g. NOTES_ERR_INVALID_DATE) instead of the actual error code (e.g. 4045) you need to use the constants in the NotesError interface since there isn’t a way to include the error codes like you do in LotusScript unless you implement the interface.

I would consider it bad form to implement the NotesError interface simply to get the error codes. Either use it as a static constant or, if using Java 5 outside of Notes/Domino, use static imports.

As a rule of thumb I would recommend that you do not use lotus.domino.NotesException for your own program code but rather use custom exception classes (by extending java.lang.Exception) or use standard exception classes from the JVM as you see fit. That’s how I do it anyways…

Java in Notes/Domino Explained: Domino Session Tester


Testing local and remote sessions to Notes/Domino can be a hassle so I’ve created a small utility to help test the different kinds of sessions. To use the Domino Session Tester download the two listed jar-files at the end of the post to a directory, copy the notes.jar to the same directory and follow the below instructions for use.

Please note: If you get a message saying “Exception in thread “main” java.lang.NoClassDefFoundError: lotus/domino/NotesException” you haven’t copied Notes.jar to the directory…

Base syntax

The base syntax of the utility is as follows:

C:>java -jar domino_session_tester.jar

Domino Session Tester by Mikkel Heisterberg (http://lekkimworld.com)
Version: 1.0, Date: 10 July 2006

***********************************************************************************
* This utility is provided "AS IS" with no warranties, and confers no rights.     *
* Some rights reserved (disclaimer: https://lekkimworld.com/pages/disclaimer.html) *
*                                                                                 *
* This utility uses code from the Apache Jakarta Commons CLI project              *
* (http://jakarta.apache.org/commons/cli) and respects their copyrights.          *
***********************************************************************************

usage: Domino Session Tester
 -t,--type    session type (remote/local)

The above command will show the different options available to you. The only required argument is the -t argument where you specify the type of session to create (“local”/”remote”). Once you select “local” or “remote” more options will be available. Apart from this the utility should be pretty straight forward to use. I have shown some examples below.

Testing local session (using id-file from local Notes installation)

C:>java -jar domino_session_tester.jar -t local

Testing anonymous remote sessions

C:>java -jar domino_session_tester.jar -t remote -h diiop.example.com

Testing authenticated remote sessions

C:>java -jar domino_session_tester.jar -t remote -h diiop.example.com -u "John Doe" -p secret

Comments are as always welcome.

Downloadable files:

Java in Notes/Domino Explained: Different kinds of sessions


When creating sessions to Notes/Domino (lotus.domino.Session) you have to distinguish between the two different types:

  • Local sessions
  • Remote sessions

Below I’ll describe both types and more importantly what you cannot do with the different types.

Local sessions

You create a local session using the methods of lotus.domino.NotesFactory class that doesn’t take a hostname argument. The session is actually just a thin wrapper around native calls to an underlying DLL/shared library hence you need to have the binary Notes and/or Domino code installed on the machine your code runs on.

Local sessions cannot be created using a LTPA (Lightweight Third Party Autentication) token or anonymously.

To use local sessions you need to include the notes.jar file on the classpath of the code.

Remote sessions

Remote (or Corba) sessions are run over the IIOP wire protocol hence you need the DIIOP server task loaded on the Domino server. You do not need to have the HTTP server task loaded on the Domino server to use remote sessions. The reason you would normally load HTTP anyway is if you want the code to automatically discover the hostname and port to use for the Domino server. This is done via the IOR (Interoperable Object Reference) and the diiop_ior.txt on the Domino server (http://diiop.example.com/diiop_ior.txt). If you do not load the HTTP task you can provide the IOR to the NotesFactory method as a String.

You cannot create remote sessions against a Notes client installation.

Remote sessions are created using the methods of lotus.domino.NotesFactory that take a hostname as an argument. Remote sessions can be created anonymously, using cleartext username/password or using a previously established LTPA token.

To use remote sessions you need to include the ncso.jar file on the classpath of the code.

Java in Notes/Domino Explained: Test agents in Eclipse by extending AgentBase (part 5)


By now (I assume you read part 1, part 2, part 3 and part 4) we have a working AgentBase extension called EclipseAgentBase. In this part I’ll show how to use the new and improved AgentBase class to easily write, test and debug Notes/Domino Java agents in Eclipse.

Please bear in mind that setting up the runtime environment for each agent below can seem like a chore and unnecessarily complex and cumbersome. It does however have the advantage that you can test the agent code and not worry about whether the database is a known state etc. It is my experience that most times when agents fail in test it is because some required document or profile document is missing. You can stub out these requirements with the new EclipseAgentBase class so you can focus on the agent code.

Enough explaining – let’s look at some code.

Simple agents, no objects from AgentContext

In the most simple agents, which is normally never the case,me where you do not use any objects from the AgentContext you only need to override the NotesMain method and implement the main()-method boilerplate code. As you can see from the example you can use the isNotes() method if you need to know whether the code is running in Notes/Domino or in Eclipse. Notice how the code runs perfectly no matter if you run the code in Notes/Domino or in Eclipse.

import lotus.domino.NotesThread;
import lotus.domino.Session;

public class ExampleAgent1 extends EclipseAgentBase {

   public static void main(String[] args) {
      try {
         NotesThread.sinitThread();
         new ExampleAgent1().NotesMain();

      } catch (Exception e) {
         e.printStackTrace();
      } finally {
         NotesThread.stermThread();
      }
   }

   public void NotesMain() {
      try {
         Session session = this.getSession();
         if (this.isNotes()) {
            System.out.println("We're in Notes...");
         } else {
            System.out.println("We're outside Notes...");
         }
         System.out.println("Username: " + session.getUserName());
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
}

“Normal” agents, using AgentContext.getCurrentDatabase()

In what I call “normal” agents you will probably at least use some information about the current database. To do this we have to override the callbackGetCurrentDatabase() method of the agent (in bold below). This overridden method will only be used when running outside Notes/Domino. If your agent does a lot of calls to the method you might want to store the database instance and return the same instance if called multiple times.

import lotus.domino.Database;
import lotus.domino.NotesException;
import lotus.domino.NotesThread;
import lotus.domino.Session;

public class ExampleAgent2 extends EclipseAgentBase {

   public static void main(String[] args) {
      try {
         NotesThread.sinitThread();
         new ExampleAgent2().NotesMain();

      } catch (Exception e) {
         e.printStackTrace();
      } finally {
         NotesThread.stermThread();
      }
   }

   protected Database callbackGetCurrentDatabase() throws NotesException {
      return this.getSession().getDatabase(null,"names.nsf");
   }

   public void NotesMain() {
      try {
         Session session = this.getSession();
         System.out.println("Database title: " +
               session.getAgentContext().getCurrentDatabase().getTitle());
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
}

Document agents, using AgentContext.getUnprocessedDocuments()

Many agents is built to process the unprocessed documents of the database. This poses some problems when configuring the runtime environment for testing. This is purpose I have written a couple of utility classes to help set this up.

Stay tuned.

Java in Notes/Domino Explained: Can I cast a null pointer?


In short – yes you can. There is nothing that prevents you from compiling the following:

String s1 = (String)null;

I even think you’ll have to do it to make your code compile for one for the methods of lotus.domino-package. I can’t of the top of my head remember which method it is, but there is one method where the compiler will force you to cast a null to a String. You’ll also sometimes end up casting a null pointer when doing dynamic classloading.

I told you it would be a short post.