Missing passthru_nonProxyHosts for IBM Connections widget proxy

When writing widgets for IBM Connections (version 1, 2, 3 and 4) and you have an iWidget (judgement still out on the OpenSocial gadget support) that needs to talk to other network resources than the IBM Connections server you need to change the proxy-config.tpl to work around the same origin policy restrictions imposed on JavaScript running in a browser. The change is easy enough and well documented but what do you do if you need to use a HTTP proxy for requests leaving your network? Well you use the passthrough proxy setting for the proxy (see ) to make sure that traffic passes through your HTTP proxy. The problem however is if you need to use a HTTP proxy to access resources external to your network but not to access internal resources since the setting is global and applies to all rules. So be warned and plan your network accordingly.

For the record there is a setting in Mashup Center to work around this (passthru_nonProxyHosts) but that setting hasn’t been implemented for IBM Connections unfortunately.

An IBM webcast you do not want to miss

This is a webcast not to miss. IBM will discuss the next version of Notes and also discuss the open beta that will start soon!! And as one who has been using the next Notes version for a while I can honestly say that you want to see this. Below is the announcement from IBM.

“Join IBM on November 13th at 10am Eastern Time US for an important series of announcements and previews. Learn how IBM Lotus Notes and IBM Lotus Domino, better than any other platform investment, positions you to create a more effective workforce. During this special broadcast, we will introduce our new IBM Notes and Domino Social Edition beta program, and updated product roadmaps, that truly demonstrate IBM’s strong commitment to the Lotus Notes and Domino platform. Speakers include Ed Brill, Director for Social Business and Collaboration Solutions, IBM; Scott Souder, Program Director for Notes, iNotes and IBM Connections Mail and Daniel Lieber, Innovative Ideas Unlimited. And, stay afterwards for a special edition of the IBM Collaboration Solutions monthly community call to discuss what you just heard. “

Speakers:
Ed Brill, Director for Social Business and Collaboration Solutions, IBM
Scott Souder, Program Director for Notes, iNotes and IBM Connections Mail
Daniel Lieber, Innovative Ideas Unlimited, Inc.

Date: Tuesday, Nov. 13, 2012
Time: 7:00 am PT / 10:00 am ET / 4:00 pm CET

Register to join

Domino rules as an API application server platform

Over the last 1.5 years we at OnTime have been making a serious investment into rewriting our backend for the OnTime Group Calendar product. The most fundamental change has been the change from a traditional Notes/Domino application (if it ever was a such) to be a server/API first approach. This means that there’s now a layer of separation between the business logic on the server and the business login in the clients namely the API. The API is now king!! It means that most, if not all, knowledge about the server operation and data storage has been removed from the clients and server and clients may evolve indepently of one another.

Another key benefit of an “API first” approach is that new exciting UI’s and uses of the OnTime data has started to pop up as it’s now easy to get going. As an application developer you do not have to worry about the OnTime backend or even worry about Domino. As long as you have HTTP access and may use JSON you’re all set.

As an example of where you’ve seen the power of the API approach – maybe without thinking about it – is in Mail and Calendar in Notes 8+. Think about how adding Java views to the “mail and calendar experience” in Notes 8+ has allowed the “mail and calendar experience” to leapfrog the rest of the client. This is because there’s a layer of separation between the data and the views and it allows them to evolve separately and at different paces.

Redesigning OnTime to an “API first approach” hasn’t been a smooth ride all along and sure there has been things we’ve changed along and way and the API backend has been rewritten more than once. The key is however that the interface to the API has only changed once which really wasn’t so much a change as a move to a new, additional, data format for the response messages. This has been key as it allowed the clients to stay constant while the API changed.

The server side API has been written in Java all along which has been a very good choice for us. The request and response messages started out being in plain text in our own proprietary format which worked out well in the beginning but it has become clear that the end user (programmer) still needed a fair amount of knowledge about OnTime to use the API. Over the last 7-9 months we’ve made a transition from first plain text in/plain text out, over plain text in/JSON out to a place where we’re now completely JSON in/JSON out. Taking it slow has been important as not to break any clients meaning our own clients or any customer API programs there might be out there now since we opened up the API for purchase.

One thing we’ve learned however is that API versions in the “real world” needs to coexist and we really couldn’t rip and replace. We’ve now move to a model where the user of the API sends an expected API version number along with the requests to signal what API version he/she expects to be returned and the API will honor that. This means that we’ve managed to move along while still maintaining backwards compatibility. Furthermore we are quite safe that we will be able to futher evolve the API while shielding customers and clients.

The real beauty about the API is the Domino application server though.

We have implemented and evolved on the API from day one using nothing more than a standard Domino server – running the HTTP task if HTTP access to the API is required – using standard Java agents. No fancy OSGi plugins here (yet!) – everything is plain ol’ Domino. Keeping everything on Domino has also allowed us to easily leverage the API in an integration problem we’re doing at a Danish customer at the moment (more on that in a later post). How? Using the web services framework available in Domino of course. Nothing fancy there. Just add a web services endpoint to respond to a SOA server and we were golden – it too scales extremely well.

Builing on Domino hasn’t limited us in any way yet – quite contrary. It has allowed us to add additional application endpoints using the easy of development approach of Domino and it has allowed us to scale well beyond thousands of users (think 15.000+ users) without problems. All this and still allowing us to seamlessly service clients from Domino 7 and upwards. So sweet.

Below is the API stack we’re using as you can see Domino feature prominently at the bottom of the stack.



(click image for a larger version)

So start building you own API’s for your Domino apps. It will be very much needed to get the full potential from Notes 8.5.4 Social Edition and the embedded experiences support in there. It’s all nothing if there is no API to read from. If only there were a Domino oAuth provider… Oh well – I’m sure it will all be there in good time.

Back from loads of travel – thoughts on the Leadership Alliance and Tokyo

So I’m back from a 16 day travel which had me home for 50 odd hours after the 5 days away. First Boston for the Leadership Alliance in Boston which of course – as seen other places on the interwebs is very hush hush – and then Tokyo for 8 days to meet up with our OnTime partner there. The trip was far easier from a jetlag perspective than what I had expected but then again I usually cope with jetlag very well. I did come back with pneumonia though 🙁

As mentioned what happens at the Leadership Alliance stays at the Leadership Alliance but suffice to say that it was indeed VERY interesting. If you are wondering whether IBM Connect 2013 will be worth attending it would be a resounding YES!!!! from my point of view. The message is better than it has been for a long while if you ask me. Of course IBM still needs to execute on it but man it’s looking good!

In Tokyo we joined our partner at the booth at the annual ITPro Expo. I’ll do a separate blog post on our adventures in Japan and how the event went as it really deserves it. Also it should be a lesson to many IBM Lotus Notes/Domino markets out there as it clearly shows why Japan is a thriving and vibrant Notes/Domino market. They do so much stuff right marketingwise and leaves so many markets in the dust. Including Denmark. Really. It was very interesting to see.

Now once I’m off this pneumonia thing I will be back in full swing. Stay tuned.

Social Connections IV in Amsterdam

The agenda for the upcoming Social Connections IV event in Amsterdam is now public and I’m happy to say that it includes yours truly. I will do a 30 minute introduction to the joys of widget development for IBM Connections. I’m pretty sure that there are still slots available for the event so if you’re in Europe on 30 November and you’re working with IBM Connections you really should join us. Oh – and bring your customers…

LCUSER.DUAL is an undefined name doing IBM Connections 3.0.1 side-by-side migration

In the process of moving our internal IBM Connections 3.0.1 server to IBM Connections 4.0 we need to upgrade our DB2 to be 64 bit which is giving us some problems. After talking to IBM they convinced us to first do a side-by-side migration of our 3.0.1 DB2 databases to another 3.0.1 instance before upgrading the databases to 4.0. However in the process we discovered that the documentation for this process is inacurate so I wanted to post the solution here in case others needed it. Referring to the documentation (Migrating 3.0.1 data side-by-side) one of the steps is to record the sequence numbers for 4 DB2 sequences used for the draft tables. The progress of these sequences should be discoverable by using the following SQL but it fails.

(Profiles only.) Run the following commands to update the
database sequence for DB2 or Oracle target databases:

DB2
Run the following commands on the 3.0.1 source database:
SELECT EMPINST.EMPINST.EXT_DRAFT_SEQ.NEXTVAL AS
   EXT_DRAFT_SEQ FROM DUAL;
SELECT EMPINST.EMPINST.EMP_DRAFT_SEQ.NEXTVAL AS
   EMP_DRAFT_SEQ FROM DUAL;
SELECT EMPINST.CHG_EMP_DRAFT_SEQ1.NEXTVAL AS
   CHG_EMP_DRAFT_SEQ1 FROM DUAL;
SELECT EMPINST.CHG_EMP_DRAFT_SEQ2.NEXTVAL AS
   CHG_EMP_DRAFT_SEQ2 FROM DUAL;

"SQL0204N "LCUSER.DUAL" is an undefined name. SQLSTATE=42704".

The problem is that the documented SQL references a non-existing table called “DUAL” and some table columns which then doesn’t make any sense either. What you actually want to look at are the actual sequences (see “SELECT SEQNAME FROM SYSCAT.SEQUENCES”). Once you know that it’s pretty easy to get the starting values.

Please note that if you’ve never used the draft functionality and DSML to sync changes back to LDAP you can happily omit these steps altogether.

Latest pet project

My latest pet project has been working with the OnTime Group Calendar API and try to look into new product ideas or use-cases for calendar data. One thing that came up really quickly is a way to expose all the calendars in OnTime in iCal format and using the API and ical4j it took all about 10 minutes to do. So now we have the option to expose each user in OnTime Group Calendar as an iCal feed honoring the access rights of the logged in user. Now that the data is available as iCal as well it makes remixing and reusing the data in custom scenarios extremely easy and the number of clients grew from our 6 to basically unlimited.

Below is an screenshot example of my adding OnTime calendars as subscribed calendars on my iPad (click the image for a larger version).



(click on image for larger version)

Now using multiple overlays on my iPad (or my iPhone) is just as horrible for as using multiple Lotus Notes overlays (or other clients) which only justifies why we develop client UI’s tailor made for the purpose of group calendaring. But… The overlay option is very nice and allows me to quickly discover what a colleague is doing while not in another OnTime client. It even allowed me to expose my business calendar very easily to my wife. No more manual weekend calendar sync – win!!

I’m not saying this will ever become a part of OnTime Group Calendar but it sure is nice.

The code is written as an agent so it was very easy to deploy and using a Internet Site rewrite rule I was able to do a nice URL per user. So now we have a url like http://example.com/ical/jdoe@example.com for each user.

Now off to look for more ways to use the API.

IBM Connections 4.0 is 64 bit only

This weekend we’ve been upgrading to IBM Connections 4.0 on a number of systems and for one we are having problems. The problem is that it’s a small demo environment which is running on 32 bit Windows. The new release is only supported on 64 bits and the DBWizard will actually not run on 32 bit as the JVM supplied with the wizard is 64 bits only. Whether you can run the wizard on a 64 bit machine and connect to a 32 bit DB2 instance remains to be seen. So now you know…

IBM Connections 4.0 Detailed System Requirements for Windows

UKLUG is about to start

It’s Monday morning in Cardiff, UK, and I’m getting ready to head to breakfast and then to the venue for UKLUG. I’m really looking forward to speaking at this event (today at 11.45am – be there or be square) which is always one of the highlights of the European LUG’s. You will also see the top of the Pop of LUG/Lotusphere speakers here and the agenda looks extremely good.

For the speakers the event actually started last night where we were treated to a very nice private tour of the Cardiff Castle and following that dinner in the old (like 2000 years old) castle undercroft. Very nice. Thanks to TC-Soft and Tim Clark for providing guide books for everyone and of course the entire UKLUG led by Warren and Kitty for organizing and making it all possible.

Java Maps and load factors

When writing Java code be that for standalone Java, Domino agents or even XPages you often need a data structure to map keys to values. In Java this is a map (java.util.Map) and it works much like a List in LotusScript. Maps are used a lot but since Map is an interface you need an implementation whih is most often a java.util.HashMap.

The way to construct a HashMap is using one of the constructors and if you are like me a couple of months ago you just use the default constructor that is the one that doesn’t take any arguments. Now this is well and fine if you know what you’re doing. Most times the use of the default constructor is the wrong choice and can / will lead to memory being wasted and even worse is worse performance when adding more than 12 elements.

Lets take each in turn. The wasted memory comes from the classes used internally inside a HashMap to keep track of keys and values. For this post knowing that not filling the map (e.g. creating a HashMap using the default constructor and only stuffing 5 elements in it) will waste memory. A most excellent introduction is From Java Code to Java Heap. On a related note a key take-away from that article is to consider LinkedList instead of ArrayList if you not know the ultimate number of elements and hence cannot size correctly at object construction.

This memory waste can be avoided by always using the constructor that takes an initial size and hence size the Map correctly. Now it turns out that that’s only part of the story as a HashMap also has what’s called a “load factor”. The load factor is a decimal number – i.e. a percentage – specifying how much the map may be filled before an extension and hence a full rehash of the keys take place. This means that with the default load factor of 0.75 you may only put 12 elements in a HashMap before it is extended and hence you take a performance hit. Again size your map correctly from the get go taking the load factor into consideration. The last point is that the size should always a power of 2 meaning that if you specify a size of 20 it will actually be 32.

A most nice discussion that caused me to look deeper into this can be hear on episode 391 of the Java Posse podcast (Java Posse #391 – Newscast for August 10th 2012) from 0:58 to 1:05.