Monday, November 29, 2010

Interrogating the Java Model in an Eclipse JDT Plugin to find a method

Previously I noted how to setup an Eclipse Plugin that can access JDT, noting some nomenclature and setup steps that are non-intuitive to an Eclipse noob. In trying to wire up an in-Eclipse version of the JSP checker (see post), one of the first problems encountered is that finding methods using java.lang.reflect doesn't work; we need to use the JDT model to ask questions about the users Java code instead.

As with most noob problems this proves fairly simple once done, but initially the Eclipse APIs provide a nice baffling wall. Our code is still running in a org.eclipse.core.resources.IncrementalProjectBuilder derived class and thus has a reference to the IResource for the file we are interested in (in my specific example, for a JSP file that in turn has a qualified class name in it). To implement the JSP check we'll need to then search for a user class by qualified type name, then check if the class has a specific method.

Our first step is to check if the type exists. Initially we only want to look in the current project, so we'll need a reference to the IJavaElement for the project. It turns out we can get a reference to this from JavaCore by passing in the IProject, which we can get from our IResource for the file. This means checking if a type exists can be done similar to the following (given that we have IResource resource as the resource our builder is working on and String qualifiedTypeName containing the type name of interest):
IJavaProject javaProject = JavaCore.create(resource.getProject());
IType type;
//normally findType returns null if there is no such type but theoretically it can throw JavaModelException also
try {
  type = javaProject.findType(qualifiedTypeName);
} catch (JavaModelException jme) {
  throw new RuntimeException("Error trying to find type \"" + typeName + "\"");
boolean typeExists = null != type;
Wonderous! Now to see if we can look for a method. The docs seem to suggest we can use the SearchEngine to do look for stuff in Java code, passing a suitable SearchPattern in. We also have to provide a SearchRequestor implementation to trap our results, though in our case all we want to do is count them. For the time being we'll just look for 0 or >0 though eventually we'll have to determine whats going on if there are 2+ matches. We'll wrap all this into a couple of classes to help us simplify coding against the Java model as most of our code really only cares if something is a valid type and whether a JavaBeans-style property exists, not how we figure that out. Here is draft 1 of the Eclipse class to aid us in figuring out if typenames and properties are valid:
package com.example.validator;

import org.eclipse.core.resources.IProject;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.jdt.core.IJavaElement;
import org.eclipse.jdt.core.IJavaProject;
import org.eclipse.jdt.core.IType;
import org.eclipse.jdt.core.JavaCore;
import org.eclipse.jdt.core.JavaModelException;


public class JavaClassInfo {
 private IJavaProject javaProject;
 private IType type;
 public JavaClassInfo(IProject project, String typeName) {
  javaProject = JavaCore.create(project);  
  try {
   type = javaProject.findType(typeName);
  } catch (JavaModelException jme) {
   throw new RuntimeException("Error trying to find type \"" + typeName + "\"");

 public boolean hasProperty(String propertyName) {
  return hasMethod("get" + StringUtils.capitalize(propertyName))
   || hasMethod("is" + StringUtils.capitalize(propertyName));
 private boolean hasMethod(String methodName) {
  //Guard: must be a valid type to have methods
  if (!isValidTypename()) {
   return false;
  SearchPattern pattern = SearchPattern.createPattern(
  IJavaSearchScope scope = SearchEngine.createJavaSearchScope(new IJavaElement[] { type });
  CountingSearchRequestor matchCounter = new CountingSearchRequestor();
  SearchEngine search = new SearchEngine();
  try {, 
    new SearchParticipant[] { SearchEngine.getDefaultSearchParticipant() }, 
  } catch (CoreException ce) {
   System.out.println("Couldn't find type: " + ce.getMessage());
  if (matchCounter.getNumMatch() > 1) {
   System.out.println("Bit weird; " + matchCounter.getNumMatch() + " matches for " + methodName);
  return matchCounter.getNumMatch() > 0;

 public boolean isValidTypename() {
  return type != null;  
Note that the following implementation of SearchRequestor is used to count results:
package com.example.validator;

import org.eclipse.core.runtime.CoreException;

public class CountingSearchRequestor extends SearchRequestor {
 private int numMatch;
 public void acceptSearchMatch(SearchMatch match) throws CoreException {

 public int getNumMatch() {
  return numMatch;

This will allow us to write code similar to the following to check on type/method existance:
JavaClassInfo classInfo = new JavaClassInfo(resource.getProject(), qualifiedTypeName);
if (!classInfo.hasProperty(propertyName)) {
   //add a Marker discussing how dumb you are to have used a property that doesn't exist!
One step closer to having our in-Eclipse JSP checker.

Monday, November 22, 2010

How to setup an Eclipse Plugin that can access JDT

In aid of remembering for next time, a few notes on setting up an Eclipse project that can play with the Java model of a project follow.

The first step is figuring out which version of Eclipse to use for plugin development. It was not obvious to me that "Eclipse for RCP and RAP Developers" was what I wanted on the Eclipse download page. After some Googling I puzzled it out but a name that clearly indicated to a noob it was a bundle suitable for Eclipse extension or plugin development would have helped!

Once you have Eclipse for RCP/RAP creating a plugin project is gloriously simple. Just click through some wizard pages:

This gives you a superb little project, all setup for debug and everything! However, said project doesn't have the JDT libraries on classpath so when you try to do anything interesting using the JDT it doesn't compile. You can clearly see that the "Plug-in Dependencies" don't include any of the JDT stuff ... but how to add it?

To add the JDT bits so we can do cruel and unusual things to Java projects double-click on META-INF/MANIFEST.MF in your plugin project, choose the MANIFEST.MF tab at the bottom of the editor, and add the org.eclipse.jdt, org.eclipse.jdt.core to the Require-Bundle block. On save this will add jars to your "Plug-in Dependencies", as shown below:

All easy enough once you know; however as an Eclipse noob this was rather non-obvious. Perhaps next we'll find time to make the JSP property access checker publish to the Eclipse problems view "live".

For using the JDT the documentation is pretty good, as is the JDT_fundamentals.ppt someone put together for eclipsecon 2008 (thus perhaps dated info but still helpful).

Scanning For Invalid Property Access in JSP EL Expressions V1

In applications that use JSP we often find ourselves using an object instance and accessing properties in EL expressions. For example:
<input value="${address.line1}" ... />
The problem is that if address doesn't have a getLine1() method this fails at runtime. If someone decides to rename a method our JSP may break without us knowing until someone runs the app and sees an error, particularly if the renaming was done in a library project. More commonly, someone renames something and doesn't quite find all the places it was referenced in our gloriously unchecked JSPs. Accordingly it would be really nice if we could find a way to be automatically notified of property file misuse.

A fancier version will ultimately be built but our for our first proof-of-concept version we want to crank out a quick and dirty JUnit test that will fail if we use a non-existent property. Our first problem to overcome is that even if we scan for EL expressions that look like object field accesses we can't tell what type is being used. Our super-simple solution will be to simply have this explicitly expressed in our JSPs: we'll use a server-side comment that provides this information to our scanner.
<%-- TYPECHECK(myVar,com.example.package.MyType) --%>
The first draft of the scanner will simply search our source tree for TYPECHECK blocks using a regular expression. To find the TYPECHECK blocks we can use an expression similar to:
private static final Pattern checkDecl = Pattern.compile("(?m)^\\s*<%--\\s+TYPECHECK\\s*\\(\\s*(\\w+)\\s*,([\\w.]+)\\s*\\)\\s*--%>\\s*$");
For the record, Java-escaped regular expressions make babies weep and programmers wish for a verbatim-string mechanism, ala C#. Note that we establish capturing groups for varname (group(1)) and type (group(2)) to help pull values out of our Matcher.

In the source tree we want to scan we happen to know that we can find our source dir offset from the / resource. We can then scan *.jsp for TYPECHECK declarations and scan the files using TYPECHECK for accesses to varname that reference non-existent properties. The following listing does just that:

package com.example.validate.jsp;

import static org.junit.Assert.assertEquals;

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.lang.StringUtils;
import org.junit.BeforeClass;
import org.junit.Test;

import ...internal package....ReflectionHelper;

public class JspTypeCheckTestCase {
 private static final Pattern checkDecl = Pattern.compile("(?m)^\\s*<%--\\s+TYPECHECK\\s*\\(\\s*(\\w+)\\s*,([\\w.]+)\\s*\\)\\s*--%>\\s*$");
 private static final Pattern elExpr = Pattern.compile("\\$\\{([^}])+\\}");
 private static final String useExprTemplate = "%1$s\\.(\\w+)";
 private static File srcDir;
 public static void beforeClass() throws Exception {
  srcDir = new File(new File(JspTypeCheckTestCase.class.getResource("/").toURI()), "../../src/main/webapp").getCanonicalFile();
  * Look for JSP files declaring <%-- TYPECHECK(varname,qualifiedJavaType) --%>; for all found validate that for all 
  * expressions of the form varname.something something exists on qualifiedJavaType as a JavaBean's style accessor 
  * (eg a public T isSomething() or public T getSomething() method exists) 
  * @throws Exception
 public void doTypeCheck() throws Exception {
  int numFailures = 0;
  //examine src/main/webapp/**/*.jsp files
  Collection<File> filesToExamine = (Collection<File>) FileUtils.listFiles(srcDir, new String[] { "jsp" }, true);
  for (File fileToExamine : filesToExamine) {
   String content = FileUtils.readFileToString(fileToExamine);
   Matcher matcher = checkDecl.matcher(content);
   while (matcher.find()) {
    String fullTypeCheck =;    
    String varName =;
    String typeName =;
    Class<?> type = null;
    try {
     type = Class.forName(typeName);
    } catch (Exception e) {
     System.out.println("Bad typecheck in " + fileToExamine + ": " + fullTypeCheck
       + " (" + e.getClass().getName() + " - " + e.getMessage() + ")");
    List<String> expressions = new ArrayList<String>();
    Matcher exprMatcher = elExpr.matcher(content);
    while (exprMatcher.find()) {
    if (null != type) {
     for (String expression : expressions) {
      numFailures += checkUseOfType(fileToExamine, type, varName, expression);
  assertEquals(numFailures + " typecheck failures in JSP files", 0, numFailures);  

 private int checkUseOfType(File file, Class<?> type, String varName, String expression) {
  int numFailures = 0;
  Pattern usagePattern = Pattern.compile(String.format(useExprTemplate, varName));
  Matcher m = usagePattern.matcher(expression);
  while (m.find()) {
   String pptyName =;
   String methodName = StringUtils.capitalize(pptyName);
   if (null == ReflectionHelper.findMethod(type, "^(is|get)" + methodName + "$", new Class<?>[] {}, null)) {
    numFailures ++;
    System.out.println(file.getName() + ": " + type.getName() + " has no accessor for " + methodName
     + "; bad expression: " + expression);
  return numFailures;

A couple of notes about the source:

  • It's very much a quick/dirty/non-optimized test to see if this idea is viable and more importantly if we find having such a scan helpful day to day. It sounded good on paper but lots of things meet that criteria ;)
  • ReflectionHelper.findMethod is an internal API that looks for a method with a given name, arguments, and return type. In our case we are simply scanning for an instance method named getBlah or isBlah if we find use of varname.blah in our jsp
  • If the test fails it will print the offending expressions to stdout similar to: myfile.jsp: com.example.package.MyType has no accessor for A; bad expression ${myvar.A or myvar.B}
  • The test will fail if there are > 0 bad accesses with an assertion error similar to java.lang.AssertionError: 2 typecheck failures in JSP files expected:<0> but was:<2>
  • The test is typically run on Java 1.6, with commons-lang 2.4 and commons-io 1.4 on classpath.

    If the scan proves helpful we speculate that it may evolve to be run directly in Eclipse using a custom builder. This would give us "live" reports of bad accesses in JSPs in the problems view. Heck, maybe we could even run the EL parser on our expressions and see if our EL is valid too...?

    Saturday, November 13, 2010

    Erlang: so awesome it makes my brain bleed

    I have read several times Erlang is a pretty cool functional language. I even bought the book. The book has a lot of cool examples and I enjoyed skimming the first chapters, being blown away by some of the way-easier-than-sockets-in-C examples and so on. I felt like I was getting all this list stuff even though I'm neither a LISP programmer nor really a pure functional programmer. Unfortunately the moment I decided to try puzzling out exactly how some of the early samples worked it turned out I didn't understand as well as I thought. Luckily I think I figured it out, as detailed below.

    First step, setting up to compile & run code. I downloaded Erlang R14B (from here), ran it, entered some expressions into the interactive shell, and all seemed well until I put some code into a .erl file in a directory, booted werl.exe (I'm on Windows for this) from that directory, and asked it to compile:

    53> c(myTestModule).
    ./myTestModule.erl:none: no such file or directory

    Awesome. It turns out that looking in the current directory isn't on by default. A little poking through the part of the book devoted to basic setup revealed that I needed to create a .erlang file in my user home directory that added current directory to the list of places to look. On Win7 this means C:\Users\MyUserName\.erlang with content as follows:

    io:format("Running Erlang~n").

    With that set I get a spiffy printout ('Running Erlang') when I boot the shell and now it looks for files in my current directory.

    I also found using for syntax highlighting and live reporting of broken code as I typed it was helpful. Unfortunately running and debugging from Eclipse operated without error and without any apparent output so I couldn't actually use Erlide to run programs and view output :(

    Anyway, so far so good, next to try a simple program. List comprehensions were something very new given my background but the definition the book gave seemed clear enough:

    "The most general form of a list comprehension is an expression of the following form:
    [X || Qualifier1, Qualifier2, ...]
    X is an arbitrary expression, and each qualifier is either a generator or a filter.
    • Generators are written as Pattern <- ListExpr where ListExpr must be an expression that evaluates to a list of terms.
    • Filters are either predicates (functions that return true or false) or boolean expressions."
    (Programming Erlang, "3.6 List Comprehensions")

    I can totally handle that! Not quite sure how to actually use a filter but the generator part seems dead simple. So on to the first little program, a for-each function:
    %% the version of each/1 to run for an empty list
    each([]) ->
    %% the version of each/1 to run for a list made up of 
    %% head H (first item in list) and tail T (rest of list). 
    %% Print the arguments then run each again on the tail. 
    %% If we did something with H it'd be largely the same as
    %%    for (Type h : someList { /* do stuff with h here */ }
    each([H|T]) ->
     io:format("each(H=~w, T=~w])~n", [H, T]),
    In the shell:
    54> c(myModule), myModule:each([a, b, c]). 
    each(H=a, T=[b,c]])
    each(H=b, T=[c]])
    each(H=c, T=[]])
    The last value (ok) is the return from the function. This is simply the L-value of the last statement. In this case we're getting ok as the io:format return value.

    Our each function works as expected; this is all seeming simple enough! And then we hit the books function to generate all permutations of the items in list. And somehow it didn't just click for me.
    perms([]) -> [[]];
    perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])].
    In the shell:
    57> c(myModule), myModule:perms([a, b, c]).
    It works. And I have to admit to almost hoping it didn't! But ... how did it work? It initially looked to me like it was saying take each item from the list (H<-L) and build a list of that plus the permutations of the list with the head taken off (T <- perms(L--[H])). I'm picturing equivalent pseudo-Java'ish code similar to:
    for each item in input list
      add to output that item plus the permutations of the list from the next item onwards
    This wouldn't work at all! Item b would only pair it with c; we'd never generate combinations like [b, a, c]. Except clearly we do. OK, so what the heck happens when you have multiple generators in a list comprehension? It seems like it must be doing "for each item in each generator run the expression X". Or to put it another way, if we had two generators that each pulled items from a list our expression would run for every possible pair of list items. The second generator seems to be able to use the output of the first so basically we're talking about nested loops.

    That seems ... kind of reasonable actually. Perhaps there is a way we could confirm that is what happens? First we'll need a way to print what is happening in the expression part of a list comprehension and still accumulate the result:
    %% print B using format A and return C. 
    %% This means we can display arbitrary output 
    %% and return something in the X part of a list comprehension.
    printABReturnC(A, B, C) ->
     io:format(A, B),
    We should be able to use this to simply print each step in running through a list comprehension of a simple list, and luckily it turns out it works as expected:
    lc2([]) ->
    lc2(L) ->
     [printABReturnC("lc2(L=~w) H=~w~n", [L, H], H) || H <- L].
    printABReturnC(A, B, C) ->
     io:format(A, B),
    In the shell:
    59> c(myModule), myModule:lc2([a, b, c]).
    lc2(L=[a,b,c]) H=a
    lc2(L=[a,b,c]) H=b
    lc2(L=[a,b,c]) H=c

    Next we should be able to run a list comprehension that has two generators taking items from the same list. We'd expect that for [a,b,c] we'd get each possible pairing, [[a,a],[a,b],[a,c],[b,a],[b,b],[b,c],[c,a],[c,b],[c,c]].
    lc3([]) ->
    lc3(L) ->
     [printABReturnC("lc3 H=~w, K=~w~n", [H, K], [H,K]) || H <- L, K <- L].
    printABReturnC(A, B, C) ->
     io:format(A, B),
    In the shell:
    60> c(myModule), myModule:lc3([a, b, c]).
    lc3 H=a, K=a
    lc3 H=a, K=b
    lc3 H=a, K=c
    lc3 H=b, K=a
    lc3 H=b, K=b
    lc3 H=b, K=c
    lc3 H=c, K=a
    lc3 H=c, K=b
    lc3 H=c, K=c
    This is all slowly starting to make sense! So if we wanted to produce all unique pairs from a list we could do something like this:
    lc4([]) ->
    lc4(L) ->
     [printABReturnC("lc3 H=~w, K=~w~n", [H, K], [H,K]) || H <- L, K <- L--[H]].
    printABReturnC(A, B, C) ->
     io:format(A, B),
    In the shell:
    61> c(myModule), myModule:lc4([a, b, c]).
    lc3 H=a, K=b
    lc3 H=a, K=c
    lc3 H=b, K=a
    lc3 H=b, K=c
    lc3 H=c, K=a
    lc3 H=c, K=b

    If we go back to our original permutations example the ugly bit is perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])]. We can break this down to see we have:

    • One generator that simply takes each item from the list: H <- L
    • One generator that requests all permutations of the original input list except for the element HT <- perms(L--[H])
      • That is, pair item A with all permutations of [B, C]. Pair item B with all permutations of [A, C].
    • Our first generator produces a series of simple values. Our second produces a series of lists. Our expression [H|T] takes each item from each generator and pairs them up as a list
      • That is, for each item in the input list, make a list where that item is the head and the tail is all permutations of the list without that item

    That seems reasonable. We should be able to use our spriffy print function to get output clearly showing it in action though:
    lc5() ->
    lc5([]) ->
    lc5(L) ->
     io:format("lc5(~w)~n", [L]),
     [printABReturnC("H=~w, T=~w~n", [H,T], [H|T]) || H <- L, T <- lc5(L--[H])].
    printABReturnC(A, B, C) ->
     io:format(A, B),
    And finally we can clearly see in the shell that what we thought was happening is indeed happening:
    62> c(myModule), myModule:lc5([a, b, c]).
    H=c, T=[]
    H=b, T=[c]
    H=b, T=[]
    H=c, T=[b]
    H=a, T=[b,c]
    H=a, T=[c,b]
    H=c, T=[]
    H=a, T=[c]
    H=a, T=[]
    H=c, T=[a]
    H=b, T=[a,c]
    H=b, T=[c,a]
    H=b, T=[]
    H=a, T=[b]
    H=a, T=[]
    H=b, T=[a]
    H=c, T=[a,b]
    H=c, T=[b,a]
    In Java it seems that perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])]. would be equivalent to:
    import java.util.ArrayList;
    import java.util.Arrays;
    import java.util.List;
    public class Perms {
     public static void main(String[] argv) {
      System.out.println(perms(Arrays.asList("A", "B", "C")));
     public static List<List<String>> perms(List<String> input) {
      List<List<String>> output = new ArrayList<List<String>>();
      for (String H : input) {
       List<String> LminusH = new ArrayList<String>(input);
       if (LminusH.isEmpty()) {
        //[H|T] when T is empty
        output.add(new ArrayList<String>(Arrays.asList(H)));
       } else {
        for (List<String> T : perms(LminusH)) {    
         //a list made up of [H|T]
         List<String> HT = new ArrayList<String>();
      return output;
    Note that the second generator can reference the output of the first, else the L--[H] wouldn't work, hence the equivalence to nested loops seems correct. Also note this isn't meant to be the best way to accomplish this in Java, rather it is the way most similar to what I think perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])]. does in Erlang.

    That was kind of a lot harder that Hello, World was!!

    Oh and for the record I'm using the Erlang brush for SyntaxHighlighter from

    Monday, November 8, 2010

    CXF backwards compatibility: adding an attribute on the server causes an UnmarshalException on the client

    When updating CXF (and therefore JAXB) from cxf-bundle 2.2.3 to 2.2.10 we ran into a problem where our client and server became more coupled than we had in mind. Specifically, adding an attribute to a server entity would cause an exception if the object was returned to a client that wasn't expecting that attribute (eg was using the previous definition of the server interface): Unmarshalling Error: unexpected element (uri:"", local:"myNewProperty"). Expected elements are <...list of existing properties...> 
     at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(
      ...excitingly lengthy callstack...
    Caused by: javax.xml.bind.UnmarshalException
     - with linked exception:
    [javax.xml.bind.UnmarshalException: unexpected element (uri:"", local:"myNewProperty"). Expected elements are <...list of existing properties...>]
     at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(
     at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(
     at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(
     at org.apache.cxf.jaxb.JAXBEncoderDecoder.unmarshall(
     at org.apache.cxf.jaxb.JAXBEncoderDecoder.unmarshall(
    ...excitingly lengthy callstack...

    This makes it a bit hard to update our server without updating the client in lockstep. We specifically wanted to be able to add attributes to entities without breaking our clients to avoid tying teams together too closely. Luckily it turns out that this problem can be ignored by providing a custom ValidationEventHandler as of CXF 2.2.4 using new properties added for CXF-2455. Unfortunately the event is not very distinct so we have to either follow worst-practices and examine the message to decide if this is the specific error we want to suppress or simply suppress all errors:
    import javax.xml.bind.ValidationEvent;
    import javax.xml.bind.ValidationEventHandler;
    public class IgnoreUnexpectedElementsHandler implements ValidationEventHandler {
     public boolean handleEvent(ValidationEvent event) {
      //true: keep going. In this case we only want to continue for the error we're trying to hide.
      return event.getMessage().startsWith("unexpected element (");
    Note that for this example we're assuming our handler goes into the default package which isn't a particularly good idea in reality. Once we have created our validation event handler we need to tell our client to use it. In our case in the Spring configuration of the client:
    <jaxws:client address="${}" id="myServiceClient" serviceclass="com.something.myService">
       <entry key="jaxb-validation-event-handler" value-ref="ignoreUnexpectedElementsHandler">
    Kind of lame to have to do really.