Is the Notes/Domino platform getting too complex?

Ben Poole posted a link to a rant on developerWorks (notes.net). The post addressed some of the same points that Nathan addressed some time back in his “Hourglass Community” post.

I think it is interesting that a post like this was posted to LDD not too long after Nathan posted about the same thing. I find it curiously interesting that the Notes/Domino platform turns more and more complicated and that the questions posted on LDD turn more and more basic… It could be that (real) developers simply get their information from other sources such as blogs and that those left in the forum really are the newbies. It could also be an indication that the platform getting increasingly complex is revealing to real problem for IBM Lotus.

How do we fix it? How do we get Notes/Domino developers to get smarter? It is really our, non-IBM’ers, job? Something to ponder over the weekend…

DIIOP_IOR_HOST

Today is slow on new topics so I thought I would share a nice (D)IIOP notes.ini related setting for the Domino server. When you run the Domino IIOP server behind a NAT’ed connection you will need to change the IP address that the server publishes through its diiop_ior.txt file. If you don’t clients will try to connect to the un-NAT’ed IP address instead of the NAT’ed one (since that’s the one Domino binds to).

The solution is to set the DIIOP_IOR_HOST notes.ini setting to the NAT’ed address and restart the DIIOP task. This will make the diiop_ior.txt contain the NAT’ed address.

Please note that the setting is called DIIOPIORHOST from Domino 7 and onwards but DIIOP_IOR_HOST is still supported as stated in the notes.ini documentation on developerWorks.

Multiple SMTP relay hosts

Had a strange issue at a customer yesterday. The customer is running a set of clustered SMTP devices for inbound and outbound SMTP relay and we had added both IP addresses of these clustered devices to the relay server field on the Configuration document of both their clustered Domino servers. Yesterday one of the clustered SMTP devices went down but unfortunately Domino didn’t immediately switch to the other SMTP device. Doing a “tell router status” indicated that the router task was waiting for the device to come back up. As it was it looked as though it was unaware of the second entry for the SMTP relay server.

The Domino Administrator help file states under “Configuring multiple relay hosts”:

To enable greater control over outbound message routing, you can configure multiple relay hosts. Using multiple relay hosts enables Domino to route mail addressed to certain Internet domains to certain relay hosts, without first performing a DNS lookup. For example, you can split external SMTP mail routing so that Domino routes all outbound Internet mail along one path, except mail addressed to a specific domain, such as *.acmepartner.com, which it sends through a specific SMTP server.

To configure multiple relay hosts, create a Foreign SMTP Domain document for each set of destinations, and then create SMTP connection documents to match these foreign SMTP domain documents. For example, using the previous example, you would create one Foreign SMTP Domain document for *.* and another for *acmepartner.com.

From the description I gather that Domino only can use one relay server and it you have multiple relay hosts you can only distribute the traffic among the servers based on domain name matches e.g. all .dk and .net traffic to one server and the rest to the other. But that’s load balancing and not failover.

Fortunately the customer have some intelligent network switches where you can define a logical interface (which is the one you would specify on the Configuration document) and have the switch distribute the traffic to the multiple devices based on which devices are up (the switch senses which of the devices respond to SMTP). Hence the switch does the failover and not Domino. This approach works but I was expecting Domino to provide this kind of functionality.

I wouldn’t want to start doing Notes now…

Well actually I would – I love the product but this morning after seeing a developerWorks article titled “Using and understanding Reader Names fields in IBM Lotus Notes and Domino” I caught myself thinking: “Who actually reads something as basic as this?”

I only just finished the thought before realizing that although I have been with the product for a LONG time and consider most of the features well known, many new developers are coming to Notes each year. I wouldn’t want to start now because where do you start?

When I started with Notes I think there were about 3 different courses available with nested @If’s and understanding the difference between @DbColumn abd @DbLookup being some of the most advanced topics. I haven’t counted but my guess is that there are a hell of a lot more courses now! πŸ™‚ Where do new developers begin? Is the scope of the product now such that the developers who will become the “jack-off-all-Domino-trades” are gone? Will new developers have to specialize to actually grasp the product and be productive? How long will it take before they are actually able to contribute to a decent size project? With quality design elements that is… Not just forms with a couple of fields or views, designed in a way that would make Nathan Freeman and Chris Blatnick scream! Quality stuff ready for prime time…

It’s a whole different ball game now. New technologies. Different platforms. Different languages. Different vocabulary; Notes, Domino, Eclipse, RCP, Expeditor, Form, View, Script Library, PostOpen, WebQuerySave, Design Synopsis, Principal field, Amgr task, Composite application, web service…. Considering that I already used Notes and Domino interchangeably above, although the words mean totally different things, doesn’t make it easier.

When were the last time you started a new junior developer off in Notes/Domino development? How long did it take before he/she was productive on his own? My guess is that it took a long time. I’m not saying that it is impossible, but I find it very hard to get new developers going in Notes. There is just so much stuff you simply have to know before you can produce anything that is off value to the customer (whether he be internal or external to the company).

I guess you have to start with the basics and just accept that it takes time.

Simply having to know a lot of stuff and starting with the basics concludes the rant and takes me back to the article on reader/author fields. Good thing these articles are still being written – where would we be without them… πŸ™‚

Is the security of the Notes/Domino Java implementation questionable? (security vulnerability in the Notes/Domino Java API)

DISCLAIMER
The information below is provided as-is and I cannot be held liable for any damages, direct or indirect, caused by the information in this post or based on the below findings. The information here is offered in the interest of full disclosure. I have been working with IBM Lotus to diagnose and pinpoint the exact consequences of the below findings since May 2006.

Description

As you might know a central security measure in the Notes/Domino security infrastructure is the difference between restricted and unrestricted operations. Only users granted unrestricted access may perform sensitive operations such as disk I/O and manipulating the system clock. The implementation flaw I found in the Java API of Notes/Domino allows me to circumvent these restrictions and hence circumvent the security settings of the Domino server.

As such the guidelines given in this post could also be used to fully replace the Java API and perform additional operations without the knowledge of the owner of the Domino server or Notes client.

Prerequisites

  • Disk access to the Domino server or Notes client or be able to write an agent or other piece of code that may accomplish the task for you.

Steps to reproduce

Below I describe the steps necessary to circumvent the SecurityManager and/or hide malicious code.

  1. Obtain a copy of the Notes.jar file from the Domino server and copy it to a local workstation.
  2. Unpack the archive using the jar-command.
  3. Decompile the code (I used the JODE version 1.1.2-pre2 decompiler from http://jode.sourceforge.net)
  4. Using Eclipse, or similar, edit the code in the constructor of the lotus.notes.AgentSecurityContext class as shown below:
    public AgentSecurityContext(ThreadGroup threadgroup, boolean bool) {
      m_restricted = bool;
      m_file_read = true;
      m_file_write = true;
      m_net_access = true;
      m_class_loader = true;
      m_extern_exec = true;
      m_native_link = true;
      m_system_props = true;
    
      try {
        AgentSecurityManager agentsecuritymanager = (AgentSecurityManager) System
          .getSecurityManager();
        if (agentsecuritymanager != null)
        agentsecuritymanager.newSecurityContext(this, threadgroup);
       } catch (ClassCastException classcastexception) {
         /* empty */
       }
    }
    
  5. Compile the class and replace the version from the unpacked Notes.jar
  6. Create a new Notes.jar with the manipulated code and replace the Notes.jar on the server. You might have to shutdown the server/client to be able to replace the file.

Using a Domino server in a virtual machine I created a text file called readme.txt in the root of the c-drive on the server and ran the below agent as scheduled on the server. The agent tries to read data from the readme.txt file in the root of the c-drive on the local server (Windows 2000 Server). As expected the JVM throws a java.lang.SecurityException using the Notes.jar supplied with the Domino installation. If I replace the Notes.jar supplied by IBM with my manipulated Notes.jar the agent runs to completion without any incident thus circumventing the security measures put in place by the Domino server.

import lotus.domino.*;
import java.io.*;

public class JavaAgent extends AgentBase {
   public void NotesMain() {
      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         System.out.println("Starting to run agent...");
         FileReader r = new FileReader("c:\readme.txt");
         StringBuffer buffer = new StringBuffer();
         char[] data = new char[128];
         r.read(data, 0, 127);
         buffer.append(data);
         System.out.println("File data: " + buffer.toString());
         r.close();

      } catch(Exception e) {
         e.printStackTrace();
      }

      System.out.println("Done running agent...");
   }
}

Consequences

One thing is being able to circumvent the restricted/unrestricted security measure of the Domino server. Another thing is that this can be done without the administrator or users knowing about it.

As mentioned above you might even be able to use the steps to replace some of the core classes (such as the class implementing the Document interface). By doing this you could have the manipulated class send you a copy of all e-mails generated using the Document.send() method or to add a specific user to all reader/author fields being written to documents.

This should be possible since all the Domino API types are interfaces and as such are open for re-implementation. It does however also mean that you have to manipulate the factory methods of the API.

I must stress that I haven’t tried this myself – yet…

Suggestions

Issues like this could be avoided by digitally signing the Notes.jar file provided by IBM and have the Domino server and Notes client verify the signature of the jar-file before loading classes from it. Since a lock is placed on the jar-file by operating system once read (at least on Windows), the impact on performance should be minimal since the jar-file only needs to be checked once.

As an aside I can mention that some of the jar-files provided with the Domino server/Notes client are digitally signed by IBM already:

  • ibmjcefips.jar
  • ibmjceprovider.jar
  • ibmpkcs11.jar
  • ibmpkcs11impl.jar

Resources

Re: Still a Domino Developer

There’s quite a heated discussion going on at the moment at edbrill.com and at codestore.net. I’m a bit late to chime in but here’s my 5 cents…

I enjoy working in Notes/Domino and overall I think that the platform really kicks a…! Does the product have its shortcomings? Sure – but that is true for most Swiss-army knife products. Could things be done better? Sure – but so it’s true for many other applications and environments. Do I always agree with the decisions of Lotus? Nope – but I hardly ever agree with anybody… πŸ™‚

Being serious for a moment… My number one grievance with Lotus is not about feature XYZ but rather that that the bug/feature submission process for many IBM products, incl. Lotus Notes, is opaque. Once the bug/feature request is submitted it is very difficult to get the *real* status on the issue unless you know someone on the inside.

Would it be possible to make bug/feature request submission more transparent? What if the bug/feature request database was available on the web for all to peruse? Bug tracking systems like Atlassian JIRA and Bugzilla have functionality for voting on bugs. In this way all customers and business partners could see what’s going on and what’s actively being developed. It could also help Lotus to gauge what’s really important to developers and what’s not.

While I know that a process is probably not always possible I think it would go a long way to give developers an outlet for issues and ideas. It could be that I as an user wouldn’t be allowed to create issues in the database myself but it would give me a chance to keep track of my issues.

Just a thought…

Force Notes to use mail.box when multiple mail.boxes are enabled

Although IBM states that putting e-mails directly in the mail.box isn’t a supported option, and Julie Kadashevich has been touting against it for years on Notes.net, apparently there is a setting for the notes.ini ensuring that Domino will continue to use the mail.box even though multiple mail.boxes has been configured. Only goes to show that it doesn’t help trying to enforce not using a hack that developers use because it is just easier…

Can Notes be forced to use mail.box when multiple mail.boxes are enabled via the Lotus Support RSS feed.

Spot the Domino Administrator bug!

I’m diagnosing some performance problems for a customer and I’m using the realtime statistics on the Server/Performance tab of Domino Administrator. After spending a fair amount of time I saw a strange bug in the display. Can you see it?

Need a hint? Click here to get it…