Learning something new every day – a reserved LotusScript comment

As described previously I have been looking into supporting Domino 7 web services written in LotusScript in LotusScript.doc. The code for web services is not accessible using DXL. Torber Bang helped me out by providing some C API code that will return the code as a string. The returned code looks like the code you get if you try to access the code directly from the NotesDocument objects using the LotusScript API.

Looking at the returned code it would seem that the comment staring with “++” has to be a reserved comment since it is used to separate event blocks in LotusScript. For fun I tried it in the Domino Designer and just as I thought – the agent would not save.

'++LotusScript Development Environment

How’s that for a certification question?

Give an example of an illegal comment in LotusScript.

Non-capturing parenthesis’ in regex to save the day

My love of regular expressions keeps expanding… Luckily for those mainly programming LotusScript on Windows I recently found out that the Microsoft VBScript RegEx object supports non-capturing parenthesis’ which is great when working with sub-matches.

For more information on sub-matches and some working LotusScript agent code to use when testing it refer to my sub-matches post.

For the Java programming / non-Windows users
If you need server-side agents on platforms other than Windows you can still use regex. You will need to use Java though. The options:

  • Notes / Domino 5.x: Use Apache Jakarta ORO compiled to JVM 1.1 (you will need to recompile from source to make the class format compatible).
  • Notes / Domino 6.x: Use the default binary Apache Jakarta ORO distribution.
  • Notes / Domino 7.x: Use the Java 1.4 built in regex classes found in the java.util.regex-package.

Non-capturing parenthesis’ 101 (some knowledge of regex is required)
When working with regular expressions everything you put in parenthesis’ will be available for you to reference after a match has been found. These matches are then accessible using an index into a Match-collection (the result of the execute() function). The purpose of non-capturing parenthesis’ is to be able to use parenthesis’ but without having them capture anything and therefore being added to your Match-collection.

I needed this non-capturing functionality just the other day when writing an agent to extract parts of the body text from incoming e-mail using a “When mail arrives”-agent. The incoming e-mails looks like the text below.

Direct link to issue: http://www.example.com/somedir/someurl?id=1234

Praesent hendrerit, duis ad ut enim consequat sed consectetuer nulla.
Status: Open
Responsible John Doe
Next followup: 24-12-2005

In the above example we would like to get all the the text after the initial URL and having the text available in a single match in the Matches-collection. Without non-capturing parenthesis’ this just isn’t possible.

Let us look at an example – it’s a little bit long but bear with me…

The easy way out would be to use a regex like this:

?id=d+s+(.*)

This basically means:

  1. find the string ?id= followed by at least one digit (?id=d+)
  2. after this make sure there is at least one line break (s+)
  3. get all the text following the found line break and put the text in the matches-collection ((.*))
Result:
Praesent hendrerit, duis ad ut enim consequat sed consectetuer nulla.

The reason this doesn’t work (we do not get the entire text and we do not get the trailing Status, Responsible and Next followup lines) is because the period-character matches everything but not line breaks.

OK so we throw in some s sequences to make sure we match the line breaks as well.

?id=d+s+(.*s*.*)
Result:
Praesent hendrerit, duis ad ut enim consequat sed consectetuer nulla.
Status: Open

It is getting better but we still don’t get all the lines after the text – only the first one. The problem is that we only match “some text, a possible line break and some text”. This grouping occurs a couple of times so we need to tell the regex to match this continuously.

?id=d+s+((.*s*.*)*)
Result:
Praesent hendrerit, duis ad ut enim consequat sed consectetuer nulla.
Status: Open
Responsible John Doe
Next followup: 24-12-2005

While the result looks right it doesn’t live up to the initial requirement which was only one entry in our Match-collection. With the above regex we will have two matches where the second one is blank (two sets of parenthesis’).

The reason is because all the parenthesis’ are capturing information, but only one set of parenthesis’ has some text to capture. Non-capturing parenthesis’ to the rescue – the only change from above is highlighted in bold and blue).

?id=d+s+((?:.*s*.*)*)

Adding ?: just inside the second set of parenthesis’ makes them non-capturing and will make the regex do what we want.

Why not do a Mid$-statement?
You might be asking yourself why go through all this trouble? Why didn’t I just do this using normal string operations and a traditional Instr/Mid combination:

Dim i As Integer
Dim result As String
i = Instr(1, text, "?id=")
result = Mid$(text, i+4)

Agreed – I could have done that and the result would have been the same, but what if there was a change in the received e-mail and some text was added after the “Next followup line” which we didn’t want to include? What if the format of the e-mail was changed so the URL was at the bottom? What if you needed to support multiple formats of e-mails using the same agent?

The power of regular expression really shines here since your agent doesn’t change – just the regex.

However I am as lazy as the next person and the reason I did it using regex was another. I needed to be able to let users specify the what to do on specific incoming e-mail patterns – mail rules on steroids. Regex is the only way to do this since users will continuously add patterns which would leed to constant reprogramming of the agent. Using regex the patterns can be specified using configuration documents in the database and tested by the user before being moved into production.

Try doing that with an agent and Instr/Mid!! 🙂

Domino domains are for mail routing…

Enough introduction – what is the point I want to make?

Alan writes: “Similar to how you need a Passport when travelling between two countries, two Domains need to be “cross-certified” in other to trust each other.”

That statement isn’t correct. Domains in Domino is, strictly speaking, a way to plan and manage mail routing. Nothing else. There is an one-to-one relationship between a Domino domain and a Domino Directory, that is a Domino Directory holds all the servers and users in the domain. Servers and users from different domains may belong to the same organization(s) and/or organizational units.

I agree there is a certain security aspect in play when deciding whether to divide your Domino infrastructure into domains but the main issue has to do with mail routing. Another has to do with deviding responsibility for managing different parts of the installation to regional IT-departments.

Cross-certification on the other hand is a cryptology term and is used for establishing trust between organizations, servers and users that does not share a common certificate. It does therefore not make sense to speak of cross-certification for Domino domains since a domain isn’t a cryptology “thing”. It is like comparing apples and oranges.

That’s all… 🙂

If you think I am wrong please let me know.

Rumours of the demise of Notes keeps circulating – now customers are calling us asking whether it is true!!

At the office we have received two calls from different customers asking to the rumours that are circulating about the future demise of Lotus Notes. Apparently the customers have been contacted by Microsoft shops asking them whether they really want to bet on a loosing platform that IBM plans to stop investing in after 2007!!

Who starts these rumours? Well I guess we know but how can they get away with it?

I really hope IBM will rise to the challenge and with BIG WORDS in the printed press clearly make the point that Lotus Notes is here to stay!

Creating a download tracker for Domino

function displayServletCode() {
var e = document.getElementById(‘servlet_code’);
e.style.visibility=’visible’;
e.style.display=’block’;
}

When doing a download tracker for binary content there is some issues that must be addressed – one is the MIME type of the file being downloaded. The MIME type determines how the calling browser determines what to do with the response sent by the download tracker. Since the main content being downloaded is going to be images I need to change the MIME type since the browser would otherwise simply display the image. The easy solution is to set the MIME type to application/octet-stream since it will cause the browser to ask where to save the file.

Another obstacle when handling binary data from an agent in Domino is that the default agent output you can obtain from the AgentBase class in Domino is a java.io.PrintWriter. PrintWriter is for character data and therefore not of much use to me.

You obtain the agent output writer by using the getAgentOutput() method of the AgentBase class from which all agents inherit:

import lotus.domino.*;
import java.io.PrintWriter;

public class JavaAgent extends AgentBase {

   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         PrintWriter pw = this.getAgentOutput();

      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

The getAgentOutput()-method is the only method to agent output which is discussed in the Domino 6.5.x help database. As mentioned above a PrintWriter isn’t of much use to me – what I really needed was a way to obtain a java.io.OutputStream implementation. Although not mentioned in the documentation I thought Lotus might have added a way to get a such so I wrote an agent to peek at the available methods as reported by the JVM using reflection:

import lotus.domino.*;
import java.io.PrintWriter;
import java.lang.reflect.*;

public class JavaAgent extends AgentBase {

   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();

         Method[] methods = this.getClass().getMethods();
         for (int i=0; i<methods.length; i++) {
            // get info about the method
            Class[] param = methods[i].getParameterTypes();
            Class rt = methods[i].getReturnType();
            Class[] exceptions = methods[i].getExceptionTypes();
            int mod = methods[i].getModifiers();

            System.out.print(Modifier.toString(mod));
            if (rt.equals(Void.class)) {
               System.out.print(" void ");
            } else {
               System.out.print(" " + rt.getName() + " ");
            }
            System.out.print(methods[i].getName());
            System.out.print("(");
            for (int j=0; j<param.length; j++) {
               if (j>0) System.out.print(", ");
               System.out.print(param[j].getName());
            }
            System.out.print(")");
            for (int j=0; j<exceptions.length; j++) {
               if (j>0) System.out.print(", ");
               System.out.print(exceptions[j].getName());
            }
            System.out.println(";");

         }

      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

As one might have guessed there is a method called getAgentOutputStream() (highlighted in bold) that returns an OutputStream as shown in the snippet below:

public void NotesMain();
public final void startup(lotus.domino.AgentInfo);
public final void runNotes()lotus.domino.NotesException;
public lotus.domino.Session getSession();
public static lotus.domino.Session getAgentSession();
public boolean isRestricted();
public java.io.PrintWriter getAgentOutput();
public java.io.OutputStream getAgentOutputStream();
public void setDebug(boolean);
public void setTrace(boolean);
...
...

Well this was great so I went ahead and coded the actual agent. Getting at the correct document and attachment based on supplied parameters is straight forward using the Domino Java classes. Once I had a hold on the correct attachment as an EmbeddedObject object I used the getInputStream() method to get an InputStream for the attachment. Until this point all is well and good.

From here on it should be quite simple to set the MIME type via the Content-Type HTTP header and read the bytes from the InputStream and writing them to the OutputStream. Or so I thought…

import lotus.domino.*;
import java.io.*;

public class JavaAgent extends AgentBase {

   public void NotesMain() {

      try {
         Session session = getSession();
         AgentContext agentContext = session.getAgentContext();
         EmbeddedObject o = null;

         // code to get at the correct attachment as an EmbeddedObject
         // object has been left out

         // get output stream and set mime type
         OutputStream agent_out = this.getAgentOutputStream();
         agent_out.write("Content-Type: application/octet-streamn".getBytes());

         // get an input stream for the attachment
         InputStream o_in = o.getInputStream();

         // write bytes from input stream to output stream
         byte[] bytes = new byte[1];
         while (o_in.read(bytes) > -1) {
            agent_out.write(bytes);
         }

         // flush and close
         agent_out.flush();
         agent_out.close();

      } catch(Exception e) {
         e.printStackTrace();
      }
   }
}

While the above code compiles and run just fine there is a problem. For some reason I can’t figure out not all the bytes actually reach the browser. Substituting the OutputStream from Domino (agent_out) with a FileOutputStream I can write the attachment to disk just fine so reading and writing is well. The problem must lie with the Domino OutputStream.

Well I’ll save you more ramblings about my debugging attempts but after a LOT of troubleshooting I ended up rewriting the code as a servlet. Once I ported the above code to a servlet it works like a charm when running it from Tomcat (click here to display the code).

This just strengthened my observation that the problem was with the Domino OutputStream implementation. Again using reflection I could find out exactly which kind of OutputStream Lotus was using:

...
// get output stream and set mime type
OutputStream agent_out = this.getAgentOutputStream();
System.out.println(agent_out.getClass().getName());
Class parent = agent_out.getClass().getSuperclass();
while (null != parent) {
   System.out.println(parent.getName());
   parent = parent.getSuperclass();
}
...

The result is a PrintOutputStream as shown by the result below:

java.io.PrintStream
java.io.FilterOutputStream
java.io.OutputStream
java.lang.Object

A java.io.PrintStream will use the default encoding when printing characters so it might be because of this my output data gets corrupted.

Conclusion
Well while the solution using a servlet works it isn’t perfect since it is so much easier to configure an agent than a servlet. Looking at the bright side a servlet provided superiour performance so it isn’t so bad I guess.

To summarize I was happy when I found the getAgentOutputStream() method but was equally disapointed when I found out that it didn’t work. To be fair I don’t know whether it is my code or whether it is something in the obtained OutputStream that teases me. I just hope the above will save someone else from spending precious time fiddeling around with binary data from a Domino web agent and that they will go directly to the servlet.

Firefox 1.5 RC2 – I took the plunge…

Well the overall design haven’t changed much – actually the only thing I have noticed so far is that the layout of the “Options…” dialog now has the selectors on top instead of on the right. The display of web pages seems to render faster than in Firefox 1.0.7. Apart from this there are some smaller changes to the available preferences. Check out the release notes for information on all changes.

On the downside there are some extensions that doesn’t work with Firefox 1.5. Of the extensions I have installed the following doesn’t work with 1.5RC2:

  • Live HTTP Headers
  • Greasemonkey
  • JavaScript debugger

The fact that the Live HTTP Headers extension doesn’t work could make me switch back since I use that one a lot in my daily work. As for the JavaScript debugger and Greasemonkey I might be able to live without them for the time being.

Apart from the extensions I have had no bad experiences with it so far so happy days…