Show ‘n Tell Thursday: Using XSLT on DXL documents (8 June 2006)


Using XSLT on XML is a great way to filter the data before parsing the result document. With DXL this is even more true to the VERY verbose nature of DXL. An advantage of using XSLT to filter the DXL document before parsing is that it is much easier to filter based on XPath queries rather than walk the DOM tree or using SAX parsing and writing and handling all the events that would arise from parsing a DXL document.

If you are not familiar with XPath or XSLT I suggest you read up on the technologies. Although I haven’t used the tutorials myself the tutorials provided on w3cschools.com tends to be quite good. At the site you’ll find tutorials on both XPath and XSLT.

I would also like to plug my upcoming article in the May/June issue of THE VIEW where I also discuss this subject and how I used the techniques in LotusScript.doc… 🙂

Well basically using XSLT with DXL is quite simple if you know how (isn’t everything). The only caveat is that you have to remember that the DXL documents exported from Lotus Notes use the dxl-namespace so you’ll have to import the namespace and use the namespace in all XPath queries. Normally a XSLT stylesheet starts like this:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

When using XSLT on DXL documents you’ll have to import the dxl-namespace as well (change in bold):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:dxl="http://www.lotus.com/dxl" version="1.0">

Once you have done this you can use XPath as normally and make your templates match their target. The below example will create a bullet list of the script library names from a complete export of a Notes database (notice the use of the dxl-namespace in bold):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:dxl="http://www.lotus.com/dxl"
     version="1.0">
     <xsl:output method="html" version="1.0" encoding="iso-8859-1" />

     <xsl:template match="/">
       <ul>
         <xsl:apply-templates select="/dxl:database/dxl:scriptlibrary" />
       </ul>
     </xsl:template>

     <xsl:template match="//dxl:scriptlibrary">
       <li>
         <xsl:value-of select="@name" />
       </li>
     </xsl:template>

</xsl:stylesheet>

Applying the above style sheet to an export of the LotusScript.doc database will produce a bullet list like the one below:

  • CLASS: Script
  • CLASS: ScriptParser
  • CLASS: ScriptElement
  • CLASS: Collections
  • CLASS: InputStream
  • CLASS: ScriptFactory
  • CLASS: OutputStream
  • CLASS: DocumentationWriter
  • CLASS: Index
  • CLASS: OutputStreamFactory
  • CLASS: ClassHierarchyIndex
  • CLASS: ScriptSource
  • DBDesign
  • CLASS: DocumentationBuilder
  • CLASS: LSDocConstants
  • CLASS: Comment
  • CLASS: DecisionMaker
  • CLASS: ConfigDocuments
  • CLASS: TemplateVersion

An easy way to test XSLT style sheets if you do not have an IDE capable of helping you is to associate the style sheet with the DXL document and then preview using a browser. To see how see my post called Apply XSLT style sheet to an XML document for browser friendly display.

Hope the above helps someone…

Show ‘n Tell Thursday: SSH tunneling – a must have tool for any travelling consultant (18 May 2006)


With the use of firewalls that also restrict the outbound ports accessible from the corporate networks it is getting harder and harder to access resources at the office while at a customer location. Luckily there is a solution that works most of the time and that all travelling consultants should know about.

SSH tunneling is the solution to so many questions. For some reason many networks allows you to create outbound SSH connections which makes it possible to tunnel Notes (port 1352) and Sametime (port 1533) traffic to a SSH server and from there to the destination. Don’t worry it is much simpler than it sounds.

This tutorial shows you how to set it up. I recommend PuTTY as the SSH client to use. The below screenshot shows an example of tunneling Notes and Sametime traffic to sametime.example.com and notes.example.com respectively.

Tunnelling traffic via the internet beats the hell out of replicating via GSM dial-up or via a fax/PSTN line.

Show ‘n Tell Thursday: Allow deletion of reservations in the RnR database (6 April 2006)


Some users prefer to create reservations directly in the RnR database instead of from their calendar. If this is the case they also sometimes need to delete a reservation. This is however not easily possible since the Querydocumentdelete event in the database script prevents this. Pressing delete will only result in the user being shown following message: “You must use the Delete Reservation action provided in the view to delete selected reservation(s).”

The Delete Reservation action is however not visible to normal users having been given the standard access of Author as recommended in the About document. The action is only shown to Editors and above.

The solution is to give the users Editor access or customize the RnR template to control the visibility of the action based on a role instead.

Show ‘n Tell Thursday: Setting the Content-type from a Java agent (30 Mar 2006)


Suppose you are all into this new AJAX thing and you need to output XML from your agent. If you write the agent in LotusScript you simply output the Content-type header using a Print statement as the first output to the browser.

Print "Content-type: text/xml"

Setting the Content-type from a Java agent however seems to drive many people crazy. They seem to set the Content-type header correctly but it fails to manifest itself in the browser (the XML comes out wrapped in Domino HTML). This happens since they fail to understand what they actually need to do. The Content-type needs to go on its own line hence you need to insert a linebreak after the Content-type as the below example shows.

import lotus.domino.*;

public class JavaAgent extends AgentBase {

  public void NotesMain() {

    try {
      Session session = getSession();
      AgentContext agentContext = session.getAgentContext();

      java.io.PrintWriter pw = this.getAgentOutput();
      pw.println("Content-type: text/xml");
      pw.println("<?xml version="1.0" ?>");
      pw.println("<names>");
      pw.println("<name first="Mikkel" last="Heisterberg" />");
      pw.println("</names>");

    } catch(Exception e) {
      e.printStackTrace();
    }
  }
}

The key (highlighted in bold above) is to use the println()-method of the java.io.PrintWriter instead of simply the print()-method or the write()-method.

I said it wouldn’t be a long post…

Show ‘n Tell Thursday: Validating XML when parsing with NotesSAXParser (23 Mar 2006)

When using and parsing XML for any purpose other than purely educational you really want to be using a DTD (Document Type Definition) to make sure the document is “valid” in the eyes of the application. This is possible both when using the DOM and the SAX parser, though there are differences as to when you might discover a violation of the DTD.

Using DOM a violation will be discovered before you actually start walking the DOM tree since the entire document is parsed and validated before control is given back to you as the programmer. Using SAX a violation is not reported until encountered which means that you might have parsed and processed most of the XML document so beware… When using SAX with a DTD you really shouldn’t consider any data from the XML as being valid until the SAX_EndDocument event has been raised (in the case of LotusScript).

To use a DTD with the SAX parser you really only need to be aware of the fact that the path to the DTD should be specified using the file:// protocol as shown in the example XML document below (notice the DTD import in bold):

<?xml version="1.0" encoding="iso-8859-1" ?>
<!DOCTYPE names SYSTEM "file:///c:/names.dtd">
<names>
  <name>
    <first>Mikkel</first>
    <last>Heisterberg</last>
  </name>
</names>

In this simple example the DTD is very simple and simply states that:

  • there is an element called “names”
  • the “names”-element can have 0 to many children elements called “name”
  • the “name”-element must have two children called “first” and “last” in that order
  • the contents of the “first”-element is parsable character data (#PCDATA)
  • the contents of the “last”-element is parsable character data (#PCDATA)

A caveat that most forget to mention is that the name of the top-level element should be named in the DTD import (<!DOCTYPE names SYSTEM…).

<!ELEMENT names (name*)>
<!ELEMENT name (first,last)>
<!ELEMENT first (#PCDATA)>
<!ELEMENT last (#PCDATA)>

Left is only to tell the NotesSAXParser to validate the parsed XML document against the specified DTD using the InputValidationOption-property (see code in bold).

Sub Initialize
  Dim session As New NotesSession
  Dim stream_xml As NotesStream
  Dim parser As NotesSAXParser

  Set stream_xml = session.CreateStream()
  Call stream_xml.Open("c:names.xml")

  'create validating parser
  Set parser = session.CreateSAXParser(stream_xml)
  parser.InputValidationOption = VALIDATE_ALWAYS

  'declare events
  On Event SAX_StartElement From parser Call SAXStartElement

  'start parsing
  On Error Goto catch
  Call parser.Parse()
  Exit Sub
catch:
  Msgbox parser.Log
  Resume finally
finally:

End Sub

Public Sub SAXStartElement(Source As NotesSAXParser, Byval ElementName As String, Attributes As NotesSAXAttributeList)
   Print "Encountered start of element: " + ElementName
End Sub

The above code can be copied into an agent in Domino Designer if you want to play around with it. To play around with it change the XML (e.g. by adding multiple “name”-elements and having the last one be invalid) to see how the violation isn’t reported until encoutered. Both the XML and the DTD goes in the root of the C-drive as the code is written.

Happy parsing…

Show ‘n Tell Thursday: Loving DNS blacklists (16 Mar 2006)

Before I continue however, let me tell you that the real definitive voice on SPAM prevention in Domino is really Chris Linfoot whose site contains a wealth of information on how to prevent SPAM in Domino. Julian and Bruce did two “Taking Notes” podcasts with Chris on SPAM that you might want to check out as well.

Anyways – for most companies SPAM is a really big issue and something that turn the hair of the IT department gray. I have heard estimates to the fact that 66-75% of the e-mail traffic on the web is generated by spammers. Something to be quite concerned about.

As an enterprise there are a couple of measures you can take to protect yourself from receiving massive amounts of SPAM:

  • Route your e-mail through a third-party that handles the SPAM control for you.
  • Purchase a hardware device such as a Baracuda.
  • Run an internal SPAM prevention server such as TrendMicro or another SMTP host with some kind of baysian filtering.
  • Use DNS blacklists.

In addition to the above a new approach called “graylisting” is starting to gain acceptance. In my mind graylisting sounds very promising for the time being but isn’t yet supported by Domino though.

Common for all the above solutions are that they are either quite expensive or time consuming to setup and configure. While all of the above solutions work you can get a lot of bang for your buck using a free approach called DNS blacklisting. While the concept of blacklisting is probably well known to most, the concept of DSN blacklisting may not be. Using DNS blacklist(s) the Domino server will query the blacklist(s) for the IP address of the sending SMTP server on each transaction. If the IP address of the sending server is found in a configured blacklist the server can log the occurance or simply refuse to accept the e-mail.

Configuring DNS blacklists in Domino is easy. Simply edit the configuration document for your incoming SMTP server and go to the “Router/SMTP” –> “Restrictions and Controls…” –> “SMTP Inbound Controls” tab. Toggle the “DNS Blacklist filters” to enabled and add a couple of blacklist servers and select the appropriate action to perform when a sending server is found in the blacklist. Please not that all servers are queried – not just until a match is found so you really shouldn’t just add all the DNS blacklists in the World…

There are numerous lists of there on the web with varying quality and different levels of strictness. We normally use the below list for customer installations without any issues (again Chris Linfoot has more on the different lists):

Apart from the above lists you might want to utilize some lists that deny e-mail from know spammer nations if you do not normally receive e-mails from those countries. At the office we deny e-mails from Taiwan, China and Russia:

  • ch.countries.nerd.dk
  • ru.countries.nerd.dk
  • tw.countries.nerd.dk

A complete list of country lists and information on how to use these lists are available at countries.nerd.dk.

If you are running Domino 7 you also really should instruct the Domino server to only accept e-mails to the recipients that exists in the Domino Directory. This is done on the same tab as above in the “Inbound Intended Recipients Controls” section:

To see how many connections are rejected using the blacklists you can use a Domino server command at the console:

sh stat smtp

This will produce output like the below:

  SMTP.Command.DATA = 1357
  SMTP.Command.EHLO = 33
  SMTP.Command.HELO = 5548
  SMTP.Command.Invalid = 3955
  SMTP.Command.MAIL = 5693
  SMTP.Command.NOOP = 3
  SMTP.Command.QUIT = 1603
  SMTP.Command.RCPT = 2065
  SMTP.Command.RSET = 155
  SMTP.DNSBL.bl.spamcop.net.Hits = 1115
  SMTP.DNSBL.ch.countries.nerd.dk.Hits = 1
  SMTP.DNSBL.dnsbl.sorbs.net.Hits = 2532
  SMTP.DNSBL.list.dsbl.org.Hits = 163
  SMTP.DNSBL.ru.countries.nerd.dk.Hits = 5
  SMTP.DNSBL.TotalHits = 3867
  SMTP.DNSBL.tw.countries.nerd.dk.Hits = 51
  ...
  ...

Above you can see some SMTP statistics from a server at my office. You can see a line each of the blacklists we employ starting with “SMTP.DNSBL” telling you how many hits each list have had. Please note that the statistic for a blacklist is not visible until the blacklist have had at least one hit. As you can see we deny quite a lot of SPAM just using this simple approach.

Happy blacklisting…

Show ‘n Tell Thursday: Migrating Outlook Express e-mail to Lotus Notes (2 Mar 2006)

Introduction
Please note that this post does not address the complexities of migrating entire Microsoft Exchange installations. For this there are IBM tools as well as some 3rd party solutions that I really prefer. After reading through the below example you’ll see that this approach really doesn’t work for larger migrations but might work well for migrating individual users coming into your organization with an existing Outlook Express installation.

Migrate contacts from the address book by exporting them as vCards and then import these into the personal NAB of the user.

Requirements
Well for starters make sure to install the migration tools as part of the Notes Client install package. There are a couple of different mail systems you can migrate from but here I’ll focus on Microsoft Outlook Express since it is the most common scenario for single user migrations. It is also required that you run the IMAP server task on the Domino server.

From here on I’ll assume:

  • That you succesfully installed the Notes client, including the migration tools, on the machine where the Outlook Express client is installed.
  • That you already created the user using Domino Administrator and configured the Notes client accordingly.
  • That the user mail database is using the Extended Mail Template (or above).
  • That the mail server is running the IMAP server task.

This post covers Notes 6.5.x/7.x (but I assume Notes 6.0.x will work as well).

Steps to migrate Outlook Express
Open a command prompt and navigate to your Notes binary directory (e.g. C:Program FilesLotusNotes). Running the nupgrade.exe tool with the “/?” command line option will show the list of available options.

C:Program FilesLotusNotes>nupgrade.exe /?

Notes 7.x Notes.6.5.x

As you can see from the screenshot on the left the command line option “4” means Outlook Express so we go ahead and run the command again using that option. As you can see from the screenshot on the right, the prompt from Notes 6.5.x (and probably also Notes 6.0.x) is wrong. Although the command line option “4” isn’t listed it is still available.

C:Program FilesLotusNotes>nupgrade.exe 4

After being prompted for the Notes password the following dialog box is presented. Make sure the information on the right is correct.

  • User Name
    It is pretty common having to change is the specified username. The username should be a username valid for logging in using the IMAP/HTTP protocol.
  • Account name
    This can be anything and is only used to create an account in Outlook Express as shown later.
  • IMAP server
    Specify a DNS resolvable hostname to the Domino server containing the mail database of the specified username.

Click OK and (very) quickly hereafter the process is complete.

A common error message at this point is the “Internet Address Is Missing” dialog box. To resolve this refer to technote 1091375. The message basically means that the user doesn’t have an e-mail address specified in his Person document on the server.

Now when you open the Outlook Express client you should see something like the screenshot below. Aside from the existing account a new account has been added at the bottom (in this case called “Domino Administrator”).

Select the account on the left and click the “IMAP Folders…” button in the right pane as shown below.

You will be presented with a dialog box showing the existing folders in the Notes mail database. Select the individual folders you would like to make visible and click the “Show”-button. A little icon will indicate that the folder is visible. New folders can be created as normally in Outlook Express.

Back in Outlook Express simply drag the e-mails from the existing Outlook Express folders to the folders added from the mail database or to newly created folders. You can select multiple e-mails at once.

Once you migrating the e-mails you should see the e-mails appearing in the Notes mail database as shown below.

Technorati tag(s):

Show ‘n Tell Thursday: Export directly from a web agent to an Excel spreadsheet in landscape format (23 Feb 2006)

A lot of web applications has a requirement to export some data to Excel spreadsheets. Whether you write the code (normally an agent) to export the data in LotusScript or Java you normally have two options:

  • Use a HTML page
  • Use a XML document

I’m going to concentrate here on using the HTML option since it is by far the easiest and the one that produces the shortest agent. However using the XML option affords you a number of other advanced options such as creating named cells and creating multiple worksheets in a workbook. I really suggest looking into this option if you need more specialized formatting.

To do this the easiest approach is creating the spreadsheet in Excel, saving is as XML and then using the generated XML document as a “template”.

Anyways – on with the export. To export from the web to Excel using a HTML document is mind numbingly easy. Simply create a HTML document with a HTML table and set the Content-type to application/vnd.ms-excel. Exporting a static spreadsheet with two columns and two rows is as simple as the following:

Sub Initialize
   Dim session As New NotesSession
   Dim docContext As NotesDocument
   Set docContext = session.DocumentContext

   'set content-type
   Print "Content-type: application/vnd.ms-excel"

   'start table
   Print |<HTML>|
   Print |<body><table border="1">|

   'output table
   Print |<tr><td>|
   Print |1-1|
   Print |</td><td>|
   Print |1-2|
   Print |</td></tr>|
   Print |<tr><td>|
   Print |2-1|
   Print |</td><td>|
   Print |2-2|
   Print |</td></tr>|

   'stop table and document
   Print |</table></body></html>|

End Sub

The trick is however to set the page title and page orientation for print. To do this you use can use some Microsoft proprietary CSS extensions. The changes from the above code is highlighted in bold.

Sub Initialize
   Dim session As New NotesSession
   Dim docContext As NotesDocument
   Set docContext = session.DocumentContext

   'set content-type
   Print "Content-type: application/vnd.ms-excel"

   'start table
   Print |<html xmlns:x="urn:schemas-microsoft-com:office:excel">|
   Print |<head>|
   Print |
<style>
  <!--table
  @page
   {mso-header-data:"&CMultiplication Table00ADate: &D00APage &P";
   mso-page-orientation:landscape;}
   br
     {mso-data-placement:same-cell;}

  -->
</style>
  <!--[if gte mso 9]><xml>
   <x:ExcelWorkbook>
    <x:ExcelWorksheets>
     <x:ExcelWorksheet>
      <x:Name>Sample Workbook</x:Name>
      <x:WorksheetOptions>
       <x:Print>
        <x:ValidPrinterInfo/>
       </x:Print>
      </x:WorksheetOptions>
     </x:ExcelWorksheet>
    </x:ExcelWorksheets>
   </x:ExcelWorkbook>
  </xml><![endif]-->|

   'start body and table
   Print |</head><body><table border="1">|

   'output table
   Print |<tr><td>|
   Print |1-1|
   Print |</td><td>|
   Print |1-2|
   Print |</td></tr>|
   Print |<tr><td>|
   Print |2-1|
   Print |</td><td>|
   Print |2-2|
   Print |</td></tr>|

   'stop table and document
   Print |</table></body></html>|

End Sub

It is as simple as that. The trick is of cause in the CSS extensions to control the page setup. The ones used here should be pretty self-explainatory. There are numerous other CSS extensions you can use – see the link at the bottom for more information.

Hope it helps someone. Happy Thursday.

Further reading:

Technorati tags: ,

Show ‘n Tell Thursday: Fixing the sorting in the “Recent Messages” (16 Feb 2006)

If you are like me (and most of the organizations I do consulting for) you like the newest e-mails to be on top in the inbox which is the opposite of the default sorting in the inbox. Previously, Notes 5 and earlier, you had to customize the $Inbox-folder in the mail template to achieve this. Since Notes 6 and the addition of persistant sorting of view/folder columns you don’t need to do this anymore since the users can do it themselves.

This is all well and good but if you use the “My Work” bookmarks page you will see that the newest e-mails are at the bottom of the “Recent Messages” list on the Today-page no matter how to sort your e-mails. This annoyed me for a while until I decided to track down why the sorting of the $Inbox-folder wasn’t honored.

As you can see below I found the offending option and fixed it.

To fix the sorting you need to open the bookmark.nsf database in the Designer and go to the list of forms. Find the form called “ng-today_mail” and open it for editing.

Now select the embedded view in the content pane and bring up the preference box. Go to the second tab and deselect the “Show recent first” option. Save and close the design element.

That should do it.

Technorati tags: