Rediscovering a lost gem – Stopwatch

Got a call from a customer reporting that the news on frontpage on their intranet was appearing very slowly. The news are displayed using an AJAX approach for asynchroneous loading using an agent as the data source for the XML. So far I haven’t been aware of any performance problems but “suddenly” the functionality is performing very poorly. Time to debug.

While pondering how to troubleshoot the performance problems I remembered the Stopwatch LotusScript class. I have used the class previously in other projects and it works like a charm. The class is originally from a THE VIEW article called “Performance Testing LotusScript Code Using Object-Oriented Design Techniques” by Burke LaShell (January/February 2000).

Using the class is a simple as these steps:

  1. Download the stopwatchclass.nsf database from the article abstract page.
  2. Copy the Stopwatch script library from the database to the database you would like to use it in.
  3. Use the Stopwatch class using its very simple API.
  4. After each run the class will produce a nice looking e-mail report.

Below is an example of using the class. The code should be self-explainatory but notice the use of the Stopwatch class and its Start and Stop methods.

Sub Initialize
   'declarations
   Dim session As New NotesSession
   Dim docContext As NotesDocument
   Dim subsite As String
   Dim fetcher As NewsFetcher
   Dim xml_conv As XmlConverter
   Dim repositories As Vector
   Dim xml As String
   Dim query_string As String

   'create stopwatch
   Dim stopwatch As New Stopwatch

   'error
   On Error Goto catch

   'get document context
   Set docContext = session.DocumentContext

   'get QueryString
   query_string = docContext.QUERY_STRING(0)

   'get the requested sub-site key if any (i.e. sdainfo, scaninfo)
   subsite = Strrightback(query_string, "&subsite=")

   'create a NewsFetcher
   Call stopwatch.Start(|New NewsFetcher|)
   Set fetcher = New NewsFetcher(subsite)
   Call stopwatch.Stop(|New NewsFetcher|)

   'create a XMLConverter
   Call stopwatch.Start(|New XmlConverter|)
   Set xml_conv = New XmlConverter()
   Call stopwatch.Stop(|New XmlConverter|)

   'get the news documents
   Call stopwatch.Start(|fetcher.GetRepositories()|)
   Set repositories = fetcher.GetRepositories()
   Call stopwatch.Stop(|fetcher.GetRepositories()|)

   'convert the documents to XML
   Call stopwatch.Start(|xml_conv.Convert()|)
   xml = xml_conv.Convert(repositories)
   Call stopwatch.Stop(|xml_conv.Convert()|)

   'write the XML back to the browser
   Call stopwatch.Start(|Print XML|)
   Print "Content-type: text/xml"
   Print xml
   Call stopwatch.Stop(|Print XML|)

   'exit
   Goto finally

catch:
   Print Error & ", line: " & Erl
   Resume finally
finally:

   'send
   Call stopwatch.MailAllWatchValues(|jdoe@example.com|, |Agent Debug - "(WEB - NewsFeed)"|)

   'exit
   Exit Sub
End Sub

The above code will produced a report like the one shown below for each run (the report is e-mailed to jdoe@example.com). As you can see the total run time was 8,8 seconds (!!!) with the fetcher.GetRepositories() call taking up all of the time.

seconds     %        calls     secs/call   event
===========================================================================
00008,812   100,0%   0000001   00008,812   Total run time
00008,812   100,0%   0000001   00008,812   fetcher.GetRepositories()
00008,812   100,0%   0000001   00008,812   NotesRepo.GetDocs() - get news docs
00000,000   000,0%   0000001   00000,000   New NewsFetcher
00000,000   000,0%   0000001   00000,000   New XmlConverter
00000,000   000,0%   0000001   00000,000   NotesRepo.GetDocs() - get view
00000,000   000,0%   0000001   00000,000   xml_conv.Convert()
00000,000   000,0%   0000001   00000,000   Print XML

Using this data I was able to diagnose and solve the problem. Below is performance data from the agent after solving the problem (a view index refresh problem) – notice how the runtime is now a mere 0,015 seconds…

seconds     %        calls     secs/call   event
===========================================================================
00000,015   100,0%   0000001   00000,015   Total run time
00000,015   100,0%   0000001   00000,015   xml_conv.Convert()
00000,000   000,0%   0000001   00000,000   New NewsFetcher
00000,000   000,0%   0000001   00000,000   New XmlConverter
00000,000   000,0%   0000001   00000,000   fetcher.GetRepo()
00000,000   000,0%   0000001   00000,000   NotesRepo.GetDocs() - get view
00000,000   000,0%   0000001   00000,000   NotesRepo.GetDocs() - get first document
00000,000   000,0%   0000001   00000,000   NotesRepo.GetDocs() - loop documents
00000,000   000,0%   0000001   00000,000   Print XML

There are two caveats to using the Stopwatch class:

  • The agent has to allow restricted operations since the script library uses a function from kernel32.dll.
  • Since the class uses kernel32.dll the script library only works on Windows.

Firefox and external XML entity references

I’m currently doing some work with XML and representing it as HTML in browsers using XSLT by including a reference to a style sheet. To reduce dublicating too much information across the different XML files I produce and use external entity references.

For those that doesn’t know what an external entity reference is let me give a short explanation. Bascially XML allows you to define an entity using the DOCTYPE declaration that may be referenced from within the XML document. A reference to an XML fragment as an entity may be internal (in the document itself) or external (in another file). The below code snippet shows how to reference an external file as an entity:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE file_contents [
   <!ENTITY external_info SYSTEM "external_info.xml">
   <!ENTITY internal_info "Some internal information">
]>
<file_contents>
   &external_info;
   &internal_info;
</file_contents>

In the above example the &external_info; will be replaced with the contents of the external_info.xml file and &internal_info; will be replaced with “Some internal information” when the file is parsed.

I found one limitation though when using external entity references in that Firefox doesn’t include XML from these external references in the source document for the XSLT transformation (or when previewing the document). This isn’t an issue in Internet Explorer at all so for this particular project it’s nice that the users are on Internet Explorer.

After researching the issue on Google I found an article on developerWorks (XML in Firefox 1.5, Part 2: Basic XML processing) that confirms my findings:

"As far as reading external files, Firefox does not read any external entities at all, whether parameter entities (such as DTDs and DTD fragments) or general entities (external, well-formed XML fragments). <snip /> Support of such external entities does have possible security implications, and possible performance implications, but both have workarounds, and I hope Firefox addresses these limitations soon."

I’m using Firefox 1.5.0.3 on Windows XP Prof. SP2.

Notes/Domino 6 and 7 Forum: Java Thread Synchronization in HTTP JVM

Saw a very interesting post on Java thread synchronization in the Notes/Domino 6 and 7 Forum over at Lotus developerWorks. I see that Julian already posted a reply but he also admits he’s not exactly a subject matter expert. Anyone else who has some information on the subject that they dare to share?

Link to post: Java Thread Synchronization in HTTP JVM

Firefox problem with multiple startup pages

Don’t know if you are aware of this but in Firefox you can open multiple tabs automatically when the browser starts – kind of like homepage on steoroids. You do this by separating the URLs with the pipe character (|) in the list of homepages in the options (on the General tab in ToolsOptions…). I use this a lot if I found some information I do not care bookmarking but I’ll need the next day.

There seems to be a problem with this feature however. When reopening the browser the tabs will not expand and show the page titles (see screenshot below) until I close one tab. After researching a bit it turned out to be caused by the otherwise excellent Tab Mix Plus extension. Disabling the extension removes the issue. Hmm….

The issue is on Firefox 1.5.0.3 and Tab Mix Plus 0.3.0.5 on Windows XP SP2.

Don’t plan!!

Received a nice quote by John Preston of Boston University from my big brother:

“The nicest thing about not planning is that failure comes as a complete surprise and not preceded by a period of worry and depression.”

Critical RealVNC vulnerability discovered

If you are running RealVNC for remote access you should make sure to upgrade the server software due to a recently discovered critical vulnerability. This is especially true if the VNC machine is available on the standard port and without VPN requirements.

“Using the following method, it is trivial to gain access to any RealVNC server without knowing the password. This allows full control of the target machine, with privilege levels equalling that of the user under which the RealVNC server runs – often full Administrator access on Windows desktops.”

Full article on securityfocus.com: RealVNC 4.1.1 Remote Compromise