Minimal web.xml for authenticated webapp on WAS 8.5.5.x

I was doing a simple servlet based web application that should on WebSphere Application Server 8.5.5.6 the other day. The servlet should require authentication. I really wanted to avoid using web.xml and go annotation based but it turned out that it wasn’t possible – at least for me. Servlets are secured using the @ServletSecurity and you specify required role(s) and HTTP constraints e.g. is HTTPS required etc.

I added the following annotations:

@WebServlet(urlPatterns={"/"}, initParams={@WebInitParam(name="foo", value="bar")})
@ServletSecurity(@HttpConstraint(rolesAllowed={"users"}))

The “users” role turned up just fine in WAS ISC but I couldn’t make the authentication kick in when I accessed the resources. Changing settings and values for the @ServletSecurity annotation e.g. explicitly mentioning GET didn’t do anything for me. For some reason the annotation wasn’t enough. To make the authentication kick in I had to add the following web.xml which is pretty much a standard web.xml you would do without annotations. You might be able to get away with a little less but at least I got it working… Oh well…

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="xmlns.jcp.org/xml/ns/javaee" xmlns:jsp="http://java.sun.com/xml/ns/javaee/jsp"
  xmlns:web="http://xmlns.jcp.org/xml/ns/javaee"
  xsi:schemaLocation="xmlns.jcp.org/xml/ns/javaee xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd
    http://xmlns.jcp.org/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  id="WebApp_ID" version="3.1">
  <display-name>MyApp</display-name>

  <security-constraint>
    <web-resource-collection>
      <web-resource-name>Protected Area</web-resource-name>
      <url-pattern>/*</url-pattern>
      <http-method>GET</http-method>
    </web-resource-collection>
    <auth-constraint>
      <role-name>users</role-name>
    </auth-constraint>
  </security-constraint>

  <security-role>
  	<role-name>users</role-name>
  </security-role>

  <login-config>
    <auth-method>BASIC</auth-method>
  </login-config>
</web-app>

Reserved characters in WebSphere Application Server passwords… Really!?

Had somewhat of a surprise today when IBM Support informed us that the issue our customer was experiencing could be due to unsupported characters in the password of the user mapped to the connectionsAdmin J2C alias. Say what!? But apparently there are restrictions on the different characters one can use. The password we were using had exclamation point (!) in it which is a no no. The customer is currently on WebSphere Application Server 8.5.5.6 and support suggested we try and upgrade to 8.5.5.7. Funny thing is that the customer has been using that password for years so it must have worked previously.

IBM Connections wiki: Special characters in password

WebSphere Application Server 8.5.5 InfoCenter: Characters that are valid for user IDs and passwords

Premaster RSA secret error with 4096-bit encryption in WAS ISC

Had a customer the other day that couldn’t import their SSL certificate into the WebSphere Application Server (WAS) Integrated Solutions Console (ISC) due to a “RSA premaster secret” error being shown when attempting the import. A PMR with IBM Support confirmed my suspicion that export restrictions was in play. Here is the response from IBM Support.

The premaster RSA secret error with 4096-bit encryption is usually due to the unrestricted JCE policy requirement.

Please, try to install the unrestricted policy files as follow:

  • Take existing jar file backup from /usr/WebSphere/AppServer/java/jre/lib/security
  • Go to the following website: http://www.ibm.com/developerworks/java/jdk/security/index.html.
    • Click Java SE 6
    • Click IBM SDK Policy files. The Unrestricted JCE Policy files for the SDK website is displayed.
    • Click Sign in and provide your IBM ID and password or register with IBM to download the files.
    • Select Unrestricted JCE Policy files for SDK for all newer versions (version 1.4.2 and higher) and click Continue.
    • View the license agreement and then click I Agree.
    • Click Download Now.
    • Install the files. Extract the file: unrestricted.zip into a directory of your choice. Copy the .jar files from the extraction directory to following
      directoriy: /usr/WebSphere/AppServer/java/jre/lib/security
    • Restart the server.

Fixing an IBM Connections Social Mail CPU spike problem

The other day we did a test upgrade of our internal IBM Connections 4.5 environment from CR3 to CR4 before doing the real upgrade. After the upgrade the CPU of the WebSphere Application Server node (we are in a single node architecture) would spike to a 100%. After some digging and perusing of log files we narrowed the problem down to IBM Social Mail and that component being loaded. Actually even more specifically to the Discovery Servlet which is used to discover the mail service for a particular user. The issue appeared to be a hung thread as indicated by the below stacktrace. See highlight in bold.

[4/30/14 13:39:51:534 CEST] 00000040 ThreadMonitor W WSVR0605W: Thread "WebContainer : 5" (0000014b) has been
active for 770854 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.getBundle(DefaultClassLoader.java:273)
at org.apache.aries.jndi.Utils.getBundleContext(Utils.java:111)
at org.apache.aries.jndi.Utils.doGetBundleContext(Utils.java:99)
at org.apache.aries.jndi.Utils.access$100(Utils.java:43)
at org.apache.aries.jndi.Utils$1.run(Utils.java:68)
at org.apache.aries.jndi.Utils$1.run(Utils.java:66)
at java.security.AccessController.doPrivileged(AccessController.java:229)
at org.apache.aries.jndi.Utils.getBundleContext(Utils.java:66)
at org.apache.aries.jndi.OSGiInitialContextFactoryBuilder.getInitialContext(OSGiInitialContextFactoryBuilder.java:44)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:232)
at javax.naming.InitialContext.initializeDefaultInitCtx(InitialContext.java:318)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:348)
at javax.naming.InitialContext.internalInit(InitialContext.java:286)
at javax.naming.InitialContext.(InitialContext.java:211)
at javax.naming.directory.InitialDirContext.(InitialDirContext.java:91)
at com.ibm.social.pim.discovery.ldap.domino.DominoLDAPConnector.connect(DominoLDAPConnector.java:68)
at com.ibm.social.pim.discovery.services.domino.LDAPPersonData.findPerson(LDAPPersonData.java:43)
at com.ibm.social.pim.discovery.services.domino.LDAPPersonData.findPerson(LDAPPersonData.java:69)
at com.ibm.social.pim.discovery.services.domino.DominoMailServiceLDAPConnector.connect(DominoMailServiceLDAPConnector.java:69)
at com.ibm.social.pim.discovery.services.domino.DominoMailServiceLDAPConnector.connect(DominoMailServiceLDAPConnector.java:61)
at com.ibm.social.pim.discovery.DiscoveryServiceManager.findUserByEmail(DiscoveryServiceManager.java:163)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.doDiscovery(DiscoveryServlet.java:229)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.processRequest(DiscoveryServlet.java:198)
at com.ibm.social.pim.discovery.servlet.DiscoveryServlet.doGet(DiscoveryServlet.java:139)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)

We dug around a little without success so I reached out to a friend at IBM and the answer came back. This is an issue that has been seen before and is solved by fixpack 8 of IBM WebSphere Application Server so we upgraded to 8.0.0.8 and sure enough we are back up and running. Apparently fixpack 8 is now supported and actually reading through the detailed system requirements lists that (“IBM Connections 4.5 CR4 and above recommends WAS 8.0.0.8. WAS 8.0.0.6 with required fixes is still supported (see the detailed report for CR3) .”)

Thanks to friends at IBM.

Terminology is the most important thing to know when when starting out with WebSphere Application Server

Over the last few weeks I’ve done a fair amount of consulting on IBM Connections – not so much the install and technical stuff but more simply talking to customers about WebSphere Application Server (WAS) and how it works. The single thing that people new to WAS seems to struggle the most with is the terminology and getting the overall architecture in place. Once that’s done most people actually like the platform and find it nice to work with. A while back I linked to a PDF containing a nice graphics on slide 4 (Overview of IBM WebSphere Application Server Concepts for IBM Lotus Connections Administrators) but it’s hard to find so I’m reproducing it below.

The first thing and single most important piece to understand about WAS is that a “server” is usually not what you think! 🙂 Or at least not what it meant to be being namely a physical thing. With the advent of virtual machines the WAS definition is getting easier to understand I think. Below is an attempt to explain the WebSphere terminology to someone starting to deal with IBM Connections.

I find that it makes the most sense to start from the physical/operating system layer (Windows server / Linux server). Each machine/operating system instance with an IP address or hostname is a “node“. The list of nodes in your WebSphere environment may be seen in the ISC under “System Administration/Nodes”. On a node there may be one or multiple servers. A server is a Java Virtual Machine (JVM) process meaning that it may be controlled individually and it may have memory assigned to it and have specific JVM parameters set if required. Please note that having multiple servers on a single node is very common in WebSphere Application Server.

When a server is created in WebSphere Application Server it is created using a profile which basically is a template for the server. There are two main profiles to worry about – “default” which is an server to run applications and “dmgr” which is the deployment manager. In an architecture with multiple nodes the deployment manager is the administration server that controls the nodes it knows about including making sure that the configuration is synchronized between the nodes. The nodes belonging to a single deployment manager belongs to the same cell. In other words there is a single deployment manager per cell.

Looking back at the nodes a node may be managed or unmanaged. A managed node is a WebSphere node that the deployment manager can talk to and send commands. An unmanaged node is the opposite. WebSphere knows about the node but cannot communicate with it on a WebSphere protocol. Something like an IBM HTTP Server (IHS) is therefore an unmanaged node. If a node is managed it will have a nodeagent installed. The nodeagent is a separate Java process which is started and stopped individually and separate from any server. The nodeagent can be used to synchronize the configuration to the node and to start and stop “stuff” on the node. The nodeagent may be configured to start all the servers running on it when ever it starts. On servers the administrator may install applications which run on the server. The applications may be started and stopped individually but usually all applications start and stop with the server.

For high availability you may choose to create a cluster of servers. Servers in a cluster runs the same applications and can take over from one another. Note that you cluster servers and not nodes so you could have a cluster of three servers on a single node.

So…

If you have an IBM Connections installation where everything is installed on the same Linux or Windows box and you chose the “small deployment” which puts all applications into the same server you will have the following:

  • A single cell
  • A single deployment manager controlling your cell
  • Three (3) nodes. One for the deployment manager, one for the server running the IBM Connections applications (managed node) and one for the IBM HTTP Server (IHS) (unmanaged node).
  • A single nodeagent
  • A single server running all the applications
  • A single cluster with the server
  • Three (3) JVM processes running – one for the deployment manager, one for the nodeagent and one for the server

Trusting certificates in WebSphere Application Server

If you make SSL connections from a WebSphere Application Server based application the server (or rather the cell) needs to trust the certificate of the server you are connecting to. This is very easy to do in WAS and is easily done using the Integrated Solutions Console (ISC). The way to establish the trust is as follows:

  1. Log into the WebSphere Application Server Integrated Solutions Console (ISC)
  2. From the lefthand navigator select Security/SSL certificate and key management
  3. In the list of related items on the right click “Key stores and certificates”
  4. Click “CellDefaultTrustStore”
  5. In the list of “Additional properties” on the right click “Signer certificates”
  6. Click “Retrieve from port”
  7. Fill out the form with the hostname of the server and the SSL port (usually 443) of the you want WAS to trust. Also supply an alias to know the trust by in the list of trusted certificates.
  8. Click the “Retrieve signer information” button to validate the input and retrieve and trust the certificate
  9. Click OK and then save the changes to the master configuration.

Websphere Application Server WIM LDAP adapter log trace

When debugging LDAP login issues for Websphere Application Server (WAS) you’re actually debugging the WIM (Websphere Identity Manager) part of WAS. The actual login piece is part of the adapters (database, ldap, file) which is the repository specific piece that WIM delegate the actual authentication to. The best debug string to use is “com.ibm.ws.wim.adapter.ldap.*=finest” as it limits the debugging to the LDAP piece of WIM.

Setting up LDAP failover for Websphere Application Server

As you may know LDAP is crucial to Websphere Application Server (WAS) when using it for IBM Connections so it makes good sense to configure failover for LDAP. If the LDAP server becomes unavailable you can no longer log in (actually you can’t even log into ISC – see Websphere Application Server Security – make sure file based auth continues if federated repository is unavailable) and WAS can have a hard time reconnecting to the LDAP. Failover is set up using either the ISC Federated Security UI or by editing wimconfig.xml directly (or using wsadmin commands). Using wimconfig.xml have some advantages as you can set some additional parameters. The screenshot below shows a secondary LDAP server added to the ISC.

Editing wimconfig.xml (see the wim/config-subdirectory of the cell configuration directory e.g. c:wasprofilesdmgrconfigcellsLCCell01wimconfigwimconfig.xml) is easy as well. You simply add an additional LDAP server to the config:ldapServers tag as shown below. The parameters in bold can be used to make sure that WAS return to the primary LDAP server (first listed) and optionally what the poll time should be (in minutes).

<config:ldapServerConfiguration primaryServerQueryTimeInterval="15"
   returnToPrimaryServer="true"
   sslConfiguration="">
   <config:ldapServers authentication="simple" bindDN="cn=LDAP User,o=Example"
      bindPassword="{xor}removed :)" connectionPool="false" connectTimeout="0"
      derefAliases="always" referal="ignore" sslEnabled="false">
      <config:connections host="cph001.intravision.dk" port="389"/>
      <config:connections host="cph002.intravision.dk" port="389"/>
  </config:ldapServers>
</config:ldapServerConfiguration>

Full info in the info center under Primary and secondary LDAP server failover.

Looking up a datasource from an IBM Connections event handler

For a customer project I’m working on these days I’m writing an event handler for IBM Connections Profiles to integrate two profile systems in real-time using the IBM Connections 4.0 Event SPI. Pretty powerful stuff in case you’ve never looked into it.

In IBM Connections an event handler is basically just a Java bean which you register in events-config.xml to be called when certain events occur such as a profile being updated, the photo set, the photo removed etc. In this event handler I needed to contact the Profiles database which should be easy as it’s registered in JNDI in Websphere Application Server. I couldn’t however use the usual java:comp/env/jdbc/profiles resource reference as there’s no J2EE deployment description for the event handler and hence the naming context hasn’t been mapped for me.

But with Websphere Application Server being the all-purpose application server that it is, I was able to find a way to make it work anyway. It turns out that all resources are mapped into a JNDI namespace using their cell and cluster prefix as well (I was able to deduce it from the “Example: Looking up an EJB home or business interface with JNDI” page).

So to look up the jdbc/profiles data source from the Cluster1-cluster scope I simply use the following. Sweet.

try {
   InitialContext ctx = new InitialContext();
   DataSource ds = (DataSource)ctx.lookup("cell/clusters/Cluster1/jdbc/profiles");
} catch (NamingException e) {
   // unable to lookup data source
}

Hiding the Social Mail username and password from socialmail-discovery-config.xml

It’s been bothering me a while that the username and password for our LDAP user was visible in clear text in our socialmail-discovery-config.xml. After going looking for a solution by using very specific searching I found a solution where you can hide the username and password and – stupid as I am – it’s actually right there in the install docs. Stupid is as stupid does. The solution is to remove the authentication data from the socialmail-discovery-config.xml and replace the <DirectoryUser> and <DirectoryPW> tags with a single <DirectoryAuthAlias> tag. This tag should reference a J2C alias configured in the Websphere ISC. Simple and effective.

The above solution is for Domino only – if you’re using Exchange mail you need two additional tags. For complete instructions on the tags refer to the wiki at Enabling the discovery service for IBM Connections Mail.