Reserved characters in WebSphere Application Server passwords… Really!?

Had somewhat of a surprise today when IBM Support informed us that the issue our customer was experiencing could be due to unsupported characters in the password of the user mapped to the connectionsAdmin J2C alias. Say what!? But apparently there are restrictions on the different characters one can use. The password we were using had exclamation point (!) in it which is a no no. The customer is currently on WebSphere Application Server 8.5.5.6 and support suggested we try and upgrade to 8.5.5.7. Funny thing is that the customer has been using that password for years so it must have worked previously.

IBM Connections wiki: Special characters in password

WebSphere Application Server 8.5.5 InfoCenter: Characters that are valid for user IDs and passwords

Hiding widget on my own profile

Was looking through widgets-config.xml for IBM Connections v.5 to diagnose a customer issue and stumbled on the below widgetDef attribute. Now I haven’t tried it yet but it looks like you can use it to hide a certain widget on your own profile e.g. only show when viewing other peoples profile. Could be useful.

hideWidgetForMyProfile=”true”

Loading widget data in IBM Connections 5 by the aggregator

One of the areas that changed fundamentally in IBM Connections 5 is how widget resources (JavaScript and CSS) is loaded by the browser. In prior versions the resources were loaded by the end-user browser through the AJAX proxy in IBM Connections Profiles or Communities depending on the feature in use. Starting with IBM Connections 5 the resources are aggregated and loaded by the Common feature that now also caches the resources. For end users this is great as speed and performance improves but for developers and admins it can be hard to diagnose what’s going on.

In Profiles it’s pretty easy – once you know how – to see what the aggregator is aggregating for the current user. The below video shows how to see this is Profiles. I’m still trying to fully understand it in Communities and will post the info once I have it.

Fixing an IBM Connections Social Mail CPU spike problem

The other day we did a test upgrade of our internal IBM Connections 4.5 environment from CR3 to CR4 before doing the real upgrade. After the upgrade the CPU of the WebSphere Application Server node (we are in a single node architecture) would spike to a 100%. After some digging and perusing of log files we narrowed the problem down to IBM Social Mail and that component being loaded. Actually even more specifically to the Discovery Servlet which is used to discover the mail service for a particular user. The issue appeared to be a hung thread as indicated by the below stacktrace. See highlight in bold.

[4/30/14 13:39:51:534 CEST] 00000040 ThreadMonitor W WSVR0605W: Thread "WebContainer : 5" (0000014b) has been
active for 770854 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.getBundle(DefaultClassLoader.java:273)
at org.apache.aries.jndi.Utils.getBundleContext(Utils.java:111)
at org.apache.aries.jndi.Utils.doGetBundleContext(Utils.java:99)
at org.apache.aries.jndi.Utils.access$100(Utils.java:43)
at org.apache.aries.jndi.Utils$1.run(Utils.java:68)
at org.apache.aries.jndi.Utils$1.run(Utils.java:66)
at java.security.AccessController.doPrivileged(AccessController.java:229)
at org.apache.aries.jndi.Utils.getBundleContext(Utils.java:66)
at org.apache.aries.jndi.OSGiInitialContextFactoryBuilder.getInitialContext(OSGiInitialContextFactoryBuilder.java:44)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:232)
at javax.naming.InitialContext.initializeDefaultInitCtx(InitialContext.java:318)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:348)
at javax.naming.InitialContext.internalInit(InitialContext.java:286)
at javax.naming.InitialContext.(InitialContext.java:211)
at javax.naming.directory.InitialDirContext.(InitialDirContext.java:91)
at com.ibm.social.pim.discovery.ldap.domino.DominoLDAPConnector.connect(DominoLDAPConnector.java:68)
at com.ibm.social.pim.discovery.services.domino.LDAPPersonData.findPerson(LDAPPersonData.java:43)
at com.ibm.social.pim.discovery.services.domino.LDAPPersonData.findPerson(LDAPPersonData.java:69)
at com.ibm.social.pim.discovery.services.domino.DominoMailServiceLDAPConnector.connect(DominoMailServiceLDAPConnector.java:69)
at com.ibm.social.pim.discovery.services.domino.DominoMailServiceLDAPConnector.connect(DominoMailServiceLDAPConnector.java:61)
at com.ibm.social.pim.discovery.DiscoveryServiceManager.findUserByEmail(DiscoveryServiceManager.java:163)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.doDiscovery(DiscoveryServlet.java:229)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.processRequest(DiscoveryServlet.java:198)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.doGet(DiscoveryServlet.java:139)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)

We dug around a little without success so I reached out to a friend at IBM and the answer came back. This is an issue that has been seen before and is solved by fixpack 8 of IBM WebSphere Application Server so we upgraded to 8.0.0.8 and sure enough we are back up and running. Apparently fixpack 8 is now supported and actually reading through the detailed system requirements lists that (“IBM Connections 4.5 CR4 and above recommends WAS 8.0.0.8. WAS 8.0.0.6 with required fixes is still supported (see the detailed report for CR3) .”)

Thanks to friends at IBM.

On the IBM Connections SPI’s

Earlier this week I had a requirement to interact with the IBM Connections user storage from a servlet running within WebSphere Application Server meaning that I needed to obtain the currently logged in users email address from the username (i.e. the principal name in JEE speak). As I saw it there were three options – 1) reproduce the entire Federated Repositories configuration in the servlet config, 2) use an IBM Connections API if available or 3) try and figure out if there was a WebSphere API for doing it. Option 1 was a clear no-go so I started persuing option 2. Looking into the wiki for IBM Connections I quickly found the User SPI (IBM Connections SPIs) and the description sounded promising: “You can use the IBM® Connections User SPIs to access information about the users in your IBM Connections deployment.”

Wow! Exactly what I needed so I located the lc.spi.jar and added it to my classpath and started coding. An excellent API and easy to understand documentation made it very easy and in a matter of minutes I had the code done. I packaged everything up and deployed the code to WAS and ran the code. Bummer!! All I saw was an Error 500:

Error 500: java.lang.NoClassDefFoundError: com.ibm.ventura.internal.config.exception.VenturaConfigException

Hmmm. Some kind of unsatisfied dependency but which? And an internal one at that… As you know IBM Connections is made up of about a gazillion different JARs but I digged around a bit but after wasting a lot of time on that and being frustrated I gave up I emailed a friend on the IBM Connections product management team. As it turns out the SPI’s (there are multiple: Event, Seedlist, Service and User) is only meant to be used from event handlers where they run in the context of IBM Connections. Bummer. I will try and get that into the documentation as soon as possible to avoid others wasting time on this.

I did find another way which I’ll share later today or tomorrow.

Elusive IBM Connections 4.5 CR2 upgrade issue solved

We’ve had a lot of problems trying to upgrade one of our internal test environments
from IBM Connections 4.5 CR1 to CR2. The only symptom was the we were unable to move
past the screen in the update wizard (updateWizard.sh / updateWizard.bat) where we select
to install or remove fixes. The UI would simply refuse to move past this screen –
no message, no log, no nothing. To try “something else” I got the idea to try and run
the update installer in silent mode (updateSilent.bat / updateSilent.sh) and here we got
something – a stacktrace on the console (see at the end of the post). The stacktrace referenced
“org.xml.sax.SAXParseException: Premature end of file.” so it had to be something to do
with parsing an XML file.

The stacktrace also came when just trying to list the list of installed fixes.

Looking at the stacktrace there was a reference to trying to parse the history of the
Websphere installation so I started to look into the IBM Connections installation and
found some interesting difference between the installed applications. All the applications
(e.g. profiles) has a config/includes directory (profiles/profiles/profiles/config/includes)
which holds information about installed fixes. For some applications there were not mention
of CR1 being installed. Hmmmm… The stacktrace also mentioned “premature end of file” so
I started looking for empty files (e.g. files of 0 bytes) and there I found it.

In the version/history directory (C:/IBM/Connections/version/history) I found a file called
event.history of 0 bytes. Looking at another install I saw this was indeed, or should be,
be an XML file with information about installed fixes. On this system the file was empty. Maybe
the update process when installing CR1 failed to update or add it. Simply making it an
empty “event history” file solved it! (see below for template)

<!DOCTYPE event-history SYSTEM "eventHistory.dtd">
<event-history>

</event-history>

Now when we ran the updateSilent.bat to list the installed fixes we got the expected result
back (CR1). Now running the update wizard worked and we could install CR2.

I guess the lesson really is that if you parse a file that may be empty then treat the
absence of the file (or an empty file of 0 bytes) to mean the same.

C:IBMConnectionsupdateInstaller>updateSilent.bat -fix -installDir "c:ibmConnections"
Start of [ updateSilent.bat ]
Build 20130909_1034

Attempting to locate setupCmdLine.bat.

Setting permissions on c:IBMCONNEC~1version
Launch: com.ibm.websphere.update.silent.UpdateInstaller
  [1]: -fix
  [2]: -installDir
  [3]: c:ibmConnections
System file encoding: Cp1252
Set encoding: console
Console encoding set; checking file encoding.
File encoding is Cp1252; updating to Cp850.
Updating encoding
Locating target class: com.ibm.websphere.update.silent.UpdateInstaller
Locating main method
Building proper arguments
Invoking main method
Der er angivet en rettelsesopdatering
Målbibliotek for produkt: c:ibmConnections
Copyright (c) IBM Corporation 2002, 2013; All rights reserved.
IBM Connections
Version af installationsprogram til opdateringer: 4.5.0.0, dato: 09-09-13

Der er opstået en fejl under visning af installerede rettelser:
org.xml.sax.SAXParseException: Premature end of file.
   at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
   at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
   at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
   at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
   at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
   at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
   at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
   at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
   at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
   at com.ibm.websphere.product.xml.BaseFactory.load(Unknown Source)
   at com.ibm.websphere.product.xml.BaseFactory.load(Unknown Source)
   at com.ibm.websphere.product.history.WASHistory.loadHistory(Unknown Source)
   at com.ibm.websphere.product.history.WPHistory.initialize(Unknown Source)
   at com.ibm.websphere.product.history.WASHistory.<init>(Unknown Source)
   at com.ibm.websphere.product.history.WPHistory.<init>(Unknown Source)
   at com.ibm.websphere.update.silent.BaseInstaller.commonInitializeProduct(Unknown Source)
   at com.ibm.websphere.update.silent.EFixInstaller.initializeProduct(Unknown Source)
   at com.ibm.websphere.update.silent.BaseInstaller.commonInitialize(Unknown Source)
   at com.ibm.websphere.update.silent.EFixInstaller.initialize(Unknown Source)
   at com.ibm.websphere.update.silent.EFixInstaller.listInstalled(Unknown Source)
   at com.ibm.websphere.update.silent.EFixInstaller.doListInstalled(Unknown Source)
   at com.ibm.websphere.update.silent.UpdateInstaller.process(Unknown Source)
   at com.ibm.websphere.update.silent.UpdateInstaller.main(Unknown Source)
   at com.ibm.websphere.update.silent.UpdateInstaller.main(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
   at java.lang.reflect.Method.invoke(Method.java:611)
   at com.ibm.websphere.update.launch.Launcher.invokeMain(Unknown Source)
   at com.ibm.websphere.update.launch.Launcher.main(Unknown Source)
UpdateInstaller.puiReturnCode is 9

End of [ updateSilent.bat ]