Running Mulesoft AnyPoint Studio on MacOS with homebrew

I’m using homebrew to manage many of the addons I use on my Mac. I had openjdk 12.0 installed previously but needed OpenJDK 1.8 to run Mulesoft AnyPoint Studio 7.4. Making that work I had to install OpenJDK 1.8 from adoptopenjdk (https://adoptopenjdk.net/) using homebrew and then tweak the ini-file controlling the AnyPoint Studio which is Eclipse based.

I started by downloading and installing Mulesoft AnyPoint Studio from Mulesoft. Then I installed openjdk 1.8 using homebrew.

brew tap AdoptOpenJDK/openjdk
brew cask install adoptopenjdk8

OpenJDK 1.8 was installed to /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk and the Java Runtime Environment could be found in /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre. Making sure it works is always a good idea.

/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre/bin/java -version

Now edit the AnypointStudio.ini for and specify the Java Runtime to use using the -vm switch (in bold).

-startup
../Eclipse/plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar
--launcher.library
../Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.551.v20171108-1834
-vm 
/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
-vmargs
--add-modules=ALL-SYSTEM
-Xms512m
-Xmx1024m
-XX:MaxPermSize=512m
-Dosgi.instance.area.default=@user.home/AnypointStudio/studio-workspace
-Dhttps.protocols=TLSv1.2,TLSv1.1,TLSv1
-Dsun.zip.disableMemoryMapping=true
-Dequinox.resolver.revision.batch.size=1
-Dmule.testingMode=true
-Dorg.mule.tooling.runtime.args=-XX:-UseBiasedLocking,-Dfile.encoding=UTF-8
-Dorg.mule.tooling.runtime.proxyVmArgs=-Dcom.ning.http.client.AsyncHttpClientConfig.useProxyProperties=true
-Djdk.http.auth.tunneling.disabledSchemes=
-XX:ErrorFile=./studio_crash_report.log
-Dorg.mule.tooling.client.usecache=true
-Dtooling.concurrent.local.repository.enabled=false
-Dtooling.client.configuration.filter.parameters.reserved.names=false
-Dfile.encoding=UTF-8
-Djava.awt.headless=true
-XstartOnFirstThread
-Dorg.eclipse.swt.internal.carbon.smallFonts

Simple speedtest app deployed using heroku

My parents mobile broadband connection in their summer house seemed a little flaky and my father asked me how we could monitor it. Easy I thought. I would simply grab a Raspberry pi and one of the available tools so I googled and found a nice tutorial. Getting it to work was easy enough but after having this run for a few days the results were weird and didn’t match what we saw while there. So what do any self-respecting programmer do? Write his/her own of course… 🙂

So I broke out my language of choice (Java) and wrote a simple servlet I could use using curl. The servlet responds to GET and POST requests and hence allows me to measure download and upload speed. Using curl it was very easy to capture the connection time and the actual time to do the operation. Putting it all together using cron allowed me to run it on schedule and pipe the result to CSV files for later analysis.

Now I needed a place to run the app – the obvious choice was a free dyno on Heroku. To deploy I simply created an app, set two configuration parameters and then pushed using Git which in turn built, deployed and ran the application. So easy. And all using command line.

Next steps? Send the results to a Google spreadsheet using IFTTT to allow my dad to get the data himself and analyse to his hearts content…

The application is available at github.com/lekkimworld/speedtest and the README explains the whole heroku deployment process.

Developing plugins for IBM Notes on Mac

I’ve been developing plugins for IBM Notes on Mac for years now but never really got around to sharing the steps on the blog. The below steps – in very crude form – works with Java 8 on Mac OS El Capitan (v. 10.11) using IBM Notes 9.0.1. The below sections are additions to the regular steps on creating a target platform documented otherwise on this blog

Main-tab

Run a product: com.ibm.notes.branding.notes

Execution Envionment: JavaSE-1.6

Arguments-tab

Program arguments:

-personality com.ibm.rcp.platform.personality
-debug
-console
-ws cocoa

VM Arguments:

-Declipse.registry.nulltoken=true
-Djava.util.logging.config.class=com.ibm.rcp.core.internal.logger.boot.LoggerConfig
-Dcom.ibm.pvc.webcontainer.port=8080
-Declipse.pluginCustomization="/Applications/IBM Notes.app/Contents/MacOS/rcp/plugin_customization.ini"
-Djava.protocol.handler.pkgs=com.ibm.net.ssl.www.protocol
-Dosgi.hook.configurators.exclude=org.eclipse.core.runtime.internal.adaptor.EclipseLogHook
-Dosgi.framework.extensions=com.ibm.rcp.core.logger.frameworkhook
-Xbootclasspath/a:"/Applications/IBM Notes.app/Contents/MacOS/rcp/eclipse/plugins/com.ibm.rcp.base_${rcp.base_version}/rcpbootcp.jar"
-XstartOnFirstThread

Environment-tab

  • DYLD_LIBRARY_PATH=/Applications/IBM Notes.app/Contents/MacOS
  • NOTESBIN=/Applications/IBM Notes.app/Contents/MacOS

Developing code using IBM Notes in Eclipse on Mac OS

I’m cleaning out my drafts folder and stumbled unto this one I never posted. The steps has changed slightly after IBM Notes 9.0.1 for Mac was released as that release works fine with the never JVM’s for the Mac. Actually there are very few steps you need to do to make the code work. To complete a standard console app that prints the username from the current session to stdout do the following:

  1. Write the code – could be something like this:
    import lotus.domino.NotesFactory;
    import lotus.domino.Session;
    import lotus.notes.NotesThread;
    
    public class Main {
       public static void main(String[] args) throws Exception {
          NotesThread.sinitThread();
          Session session = NotesFactory.createSession();
          System.out.println(session.getUserName());
          NotesThread.stermThread();
       }
    }
    
  2. Run the code as a Java Application. This will fail as the JVM cannot load the required libraries.
  3. Edit the Run Configuration (click the dropdown next to the green run button and choose “Run Configurations…”)
  4. Switch to the “Environment”-tab
  5. Add an environment variable called DYLD_LIBRARY_PATH with a value of “/Applications/IBM Notes.app/Contents/MacOS” (without the quotes)
  6. Click Apply and Run

This was done in Eclipse neon on Mac OS El Capitan (10.11) using Java 8.

Simple tool to save certificate chain certificates as PEM files

It’s been increasingly frustrating to support our OnTime Group Calendar for Microsoft customers with on-prem Exchange as they usually use a self-signed certificate for TLS resulting in Java throwing a fit. Getting the certificate chain using a browser or OpenSSL is easy enough but for some customers that still prove too difficult. I couldn’t find a tool to automate the export so I wrote a small tool in Java. The tool simply takes the address of the site to contact and saves the certificate chain as individual PEM files ready for import into the Java keystore. Now there is no fingerprint check so use at your own risk. Using the tool is like so:

java Main http://www.ibm.com

The code is available on Github and doubles as an example of how to accept all certificates using a custom TrustManager and HostNameVerifier. I even threw in some Java 8 to make Rene happy 😦

YMMV…

Test agents in Eclipse by extending AgentBase

I continuously get questions on how I do Java agent development and I’m happy to answer them as I hope that some of the answers I provide means that more and more Domino developers see Java as a strong alternative to LotusScript for background processing. Most times the approach I recommend is a mock object approach that I wrote about waaaaaay back in 2006 (did I really write that 10 years ago?!?!?).

If / when you want to read the posts please start from part no. 1. Here’s a link to all 5 parts:

The approach doesn’t handle importing the code back into Domino Designer but it does allow you to “mock” the Notes side of the API and allows you to write, test and debug your agents in Eclipse. Then when they’re completely done you can import it back into Notes without any changes. This is the approach I would choose and that I still use when writing Java agents here 10 years later.

Hope this helps but if it doesn’t please let me know.

Mac Yosemite, Java, IBM Notes and OnTime Group Calendar

After upgrading my Mac to OS X Yosemite (10.10) I had to reinstall Java to make IBM Notes startup just like Rene describes. To install go to the download page for Java on apple.com, download and install. It takes around 5 minutes and you are ready to go. Once installed the Java runtime makes IBM Notes fly again and I can confirm that the OnTime Group Calendar UI’s run just fine on OS X Yosemite.

Java in Notes/Domino Explained: On Java Security and how it relates to Notes/Domino


I get so many questions on Java security and how it all fits together that I finally decided to blog it instead of simply answering over and over again. So here goes…

Security was built into Java from its inception and everything in a Java Virtual Machine (JVM) runs under the control of a security manager. Now the security manager may be very lax or it may be very restrictive as the one for Java applets or Notes/Domino agents is. A security manager is installed into the JVM when it’s constructed meaning either from the command line or, in the case of Notes/Domino, when a JVM is created to run agents, XPages or the like. A security manager may decide that a certain operation (method call) is “powerful” and hence will require the code executing the operation to be allowed to do so.

The way this is done in Java is by the operation itself (the method you are calling e.g. “write file” or “do stuff with reflection”) asking the security manager whether the code calling it has a given permission that it supplies to the security manager. The security manager may either decide that it does and hence return quietly or it may decide that the calling code does not have the required permission. To signal the latter it will throw a java.security.AccessControlException which you’ve seen if you’re reading this.

So how do we grant these permissions?

In Java permissions are controlled in the java.policy file located in the file system under <notes>/jvm/lib/security or <domino>/jvm/lib/security. I will not go into detail on the syntax of this file as resources are readily available on the web but suffice to say that different locations (be those file based or network based) are granted different sets of permissions (for more info see http://docs.oracle.com/javase/6/docs/technotes/guides/security/PolicyFiles.html). By default jar-files in jvm/lib/ext, the XPages runtime, OSGi bundles etc. are granted all permissions possible (java.security.AllPermission) whereas other code is granted a limited set of permissions. So if you need to run code that requires a different set of permissions you edit this file and restart the server/client. Now often times simply editing this file is not an option. Either you do not control the server, you cannot edit it for all clients or your security guidelines tell you just can’t do it. In either case editing the file is seldom the entire solution. And you need to understand why. Before showing you some code let me explain the scenario I’m going to use below first.

The scenario is that Joe Notes Developer needs to use reflection (wikipedia) to change the accessibility of a method from private to public (yes it’s possible) to override the private restriction set on the generate() method of the Test class (see below). To do this he uses a method like the one below:

public void doAccessible(Object obj) throws Exception {
   Method[] methods = obj.getClass().getMethods();
   for (int i=0; i<methods.length; i++) {
      Method m = methods[i];
      m.setAccessible(true);
   }
}

The class he needs to affect is likewise simple (he needs to make the generate() method public):

private class Test {
   private String uuid = null;

   public Test() {
      this.generate();
   }

   public String getUUID() {
      return this.uuid;
   }

   private void generate() {
      this.uuid = UUID.randomUUID().toString();
   }
}

To start of Joe Notes Developer creates a Java agent in Domino Designer and sticks the below code in NotesMain of the agent:

Test t = new Test();
System.out.println(t.getUUID());
doAccessible (t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

But oh no!! Joe gets the dreaded AccessControlExeption as the code in the Java agent may not use reflection this way.

java.security.AccessControlException: Access denied (java.lang.reflect.ReflectPermission suppressAccessChecks)
   at java.security.AccessController.checkPermission(AccessController.java:132)
   at java.lang.SecurityManager.checkPermission(SecurityManager.java:544)
   at COM.ibm.JEmpower.applet.AppletSecurity.superDotCheckPermission(AppletSecurity.java:1449)
   at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1617)
   at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1464)
   at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:118)
   at ReflectionUtil.doAccessible(Unknown Source)
   at JavaAgent.NotesMain(Unknown Source)
   at lotus.domino.AgentBase.runNotes(Unknown Source)
   at lotus.domino.NotesThread.run(Unknown Source)

Nothing is holding Joe back though so he puts the reflection code into a ReflectionUtil class which he puts in a jar-file in jvm/lib/ext as he has heard that this location allows for this kind of funky stuff. Using the ReflectionUtil instead of a class method from his agent is like this:

ReflectionUtil l = new ReflectionUtil();
Test t = new Test();
System.out.println(t.getUUID());
l.doAccessible(t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

Arrrghhhh no!! He still gets an AccessControlException. But how can this be?? He did put the code in the special jvm/lib/ext folder which is granted AllPermissions. Hmmm…

The reason is that security in Java is layered and it’s the layer or “context” in Java parlance initiating the call that decides which permissions are assigned. This is why simply making a call from a Java agent in Notes/Domino to a Java library in jvm/lib/ext that need permissions other than the default set doesn’t just work. It doesn’t work because it’s the context of the Java agent which is in effect. So wouldn’t it be great if you could change the context? Well you can… 🙂 This is done using the java.security.AccessController class (docs).
The AccessController class is a nifty little thing. It works by you supplying an instance of the java.security.PrivilegedAction interface which is an interface with a single method called run. Inside the run-method you put the code requiring special permissions. Below is an example of using the AccessController class:

AccessController.doPrivileged(new PrivilegedExceptionAction() {
   public Object run() throws Exception {
      ReflectionUtil l = new ReflectionUtil();
      Test t = new Test();
      l.doAccessible(t);
      System.out.println("Completed...");
      return null;
   }
});

But wouldn’t you know it – this also fails! Of course the reason is quite obvious – I think – we cnanot just have developers go ahead elevating their persmissions themselves. As Paul Mooney would say – developers mus be controlled and kept down. It needs to be the privileged code meaning the code in jvm/lib/etc that needs to elevate the permissions. To do this we move the AccessController call info the ReflectionUtil in a special “doAccessiblePrivileged” method as this code can then decide to elevate permissions as it runs from a location with a higher security context. The code in the agent is therefor now like this (change in bold):

ReflectionUtil l = new ReflectionUtil();
Test t = new Test();
System.out.println(t.getUUID());
l.doAccessiblePrivileged(t);
t.generate();
System.out.println(t.getUUID());
System.out.println("Completed...");

And the code in ReflectionUtil looks like this:

public void doAccessiblePrivileged(final Object obj) throws Exception {
   AccessController.doPrivileged(new PrivilegedExceptionAction() {
      @Override
      public Object run() throws Exception {
         ReflectionUtil.this.doAccessible(obj);
         return null;
      }
   });
}

Now the code runs just fine as Joe is using code that knows that it might need to elevate the permissions so it provides a call doing just that (doAccessiblePrivileged).

Of course this can make it very difficult to just use a standard library from the web in Notes as it may not provide a way to elevate the permissions as no one ever saw that as a use case. In this case a wrapper API that elevates the permissions in jvm/lib/ext may be necessary to make it work.

Hope this explains it… 🙂

The example code is available for download including the required jar-file to put in jvm/lib/ext.

Java Maps and load factors

When writing Java code be that for standalone Java, Domino agents or even XPages you often need a data structure to map keys to values. In Java this is a map (java.util.Map) and it works much like a List in LotusScript. Maps are used a lot but since Map is an interface you need an implementation whih is most often a java.util.HashMap.

The way to construct a HashMap is using one of the constructors and if you are like me a couple of months ago you just use the default constructor that is the one that doesn’t take any arguments. Now this is well and fine if you know what you’re doing. Most times the use of the default constructor is the wrong choice and can / will lead to memory being wasted and even worse is worse performance when adding more than 12 elements.

Lets take each in turn. The wasted memory comes from the classes used internally inside a HashMap to keep track of keys and values. For this post knowing that not filling the map (e.g. creating a HashMap using the default constructor and only stuffing 5 elements in it) will waste memory. A most excellent introduction is From Java Code to Java Heap. On a related note a key take-away from that article is to consider LinkedList instead of ArrayList if you not know the ultimate number of elements and hence cannot size correctly at object construction.

This memory waste can be avoided by always using the constructor that takes an initial size and hence size the Map correctly. Now it turns out that that’s only part of the story as a HashMap also has what’s called a “load factor”. The load factor is a decimal number – i.e. a percentage – specifying how much the map may be filled before an extension and hence a full rehash of the keys take place. This means that with the default load factor of 0.75 you may only put 12 elements in a HashMap before it is extended and hence you take a performance hit. Again size your map correctly from the get go taking the load factor into consideration. The last point is that the size should always a power of 2 meaning that if you specify a size of 20 it will actually be 32.

A most nice discussion that caused me to look deeper into this can be hear on episode 391 of the Java Posse podcast (Java Posse #391 – Newscast for August 10th 2012) from 0:58 to 1:05.

The easy way to loop an XPage managed bean

I got harassed by Nathan (hmmm his blog appears to be down just now) yesterday for not contributing to the XPages dicussions out there so I thought I’d better remedy that. A quick tip is to make sure your Managed Bean implements Iterable if it contains a list of “things” and the obvious behaviour would be to loop over these things. If you implement Iterable you simple loop on the object using the “new” for-loop (when in Java) instead of first returning a List or array and then looping that with a for- or while loop.

Below is an example of a Managed Bean (just a Java-element from 8.5.3) that implements Iterable and hence it implements the iterator() method. That method should return an Iterator for the data you want to expose. Here I just return an iterator to the underlying list.

package com.example;

import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;

import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.Session;
import lotus.domino.View;

import com.ibm.domino.xsp.module.nsf.NotesContext;

public class PeopleLister implements Iterable {
  // declarations
  private List people = new LinkedList();

  public PeopleLister() {
    NotesContext ctx = NotesContext.getCurrent();
    Session session = ctx.getCurrentSession();
    try {
      Database db = session.getDatabase(null, "names.nsf");
      View view = db.getView("People");
      Document doc = view.getFirstDocument();
      while (null != doc) {
        this.people.add(doc.getItemValue("FullName")
          .elementAt(0).toString());
        Document docTemp = view.getNextDocument(doc);
        doc.recycle();
        doc = docTemp;
      }
    } catch (Throwable t) {

    }
  }

  public Iterator iterator() {
    return this.people.iterator();
  }

  public String toString() {
    StringBuilder b =
      new StringBuilder(this.people.size() * 25);
    for (String p : this) {
      if (b.length() > 0) b.append(',').append(' ');
      b.append(p);
    }
    return b.toString();
  }

  public String[] getList() {
    return (String[]) people
      .toArray(new String[people.size()]);
  }
}

With that in mind I can just use the bean as follows from SSJS code (e.g. a Label):

var people = new com.example.PeopleLister();
var result = "";
for (p in people) {
  if (result.length() > 0) result += ", ";
  result += p;
}
return result;

Happy coding!

https://lekkimworld.files.wordpress.com/prettyprint/prettify.js

prettyPrint();