Tuesday, June 14, 2011

Beginning Scala: Building a project with Maven and Eclipse

Let us suppose that after many years languishing in the dark corridors of Java, frantically scrambling to avoid the various framework ghouls, one finally emerged into the light and saw a Scala frolicking in the sun on a much greener grassy field. The terse little Scala looks inviting to the verbose, ragged, framework beridden, somewhat aged Java developer ... but how to start? Well luckily there are a few nice introductions (Scala for Java Refugees, The busy Java developer's guide to Scala) out there and the staircase book (by the language creator no less) is a must if one is serious about learning the language.

Introductions are all well and good but as a Java developer I want some first-class tools and I want to run some code! Until recently first-class tools were seriously lacking. Luckily in recent months even the languages creator has acknowledged the tooling was lacking and has taken aggressive steps to correct the issue.

At the time of writing I use Eclipse 3.6.2 Helios (Java Developer edition) with Scala-IDE 2.0.0.somethingOrOther-beta. For the sake of this example I will assume we wish to build the project using Maven. First step, download the tools:
  1. Eclipse Helios
    1. At the time of writing I favor the Java developer edition as it is somewhat less bloated than the EE version; http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/heliossr2
  2. Scala IDE 2
    1. The 2.0 release is a major modification and the first solid Eclipse IDE plugin for Scala; the 1.x version was prone to all sorts of "exciting" behaviors
    2. Add to Eclipse using the update site from link on the front page of http://www.scala-ide.org/
  3. m2eclipse
    1. Add to Eclipse using the update site from http://m2eclipse.sonatype.org/installing-m2eclipse.html
  4. Maven 3
    1. Download from http://maven.apache.org/download.html
    2. Make sure if you run mvn -v in command prompt it prints the expected version
Now we have the tools, lets setup a project! First up, create a directory for your project and in that directory create a file called pom.xml similar to:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.blogspot.whileonefork</groupId>
  <artifactId>mvn-scala-trial1</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>Sample Project for Blog</name>
  <url>http://com.blogspot.whileonefork</url>
</project>

Next, create a directory named src containing a directory named main containing a directory named scala. Your project should now contain the following:
mvn-scala-project
|--pom.xml
`--src
   `--main
      `--scala

In the src/main/scala directory, create a new file HelloWorldJustLikeJava.scala with the simplest hello world implementation possible:
object HelloWorldJustLikeJava {
  def main(args:Array[String]) = {
    println("Hello, World");
  }
}

This, while arguably not the absolute simplest Scala hello world, is the definition closest to the one you would write in Java. Now to get it compiling in Maven. If you open a command prompt in the mvn-scala-project directory and run mvn clean install it will succeed but you will get a warning that no content was marked for inclusion; this occurs because by default Maven doesn't include the plugin necessary to build Scala code.

To compile Scala files we must update our pom.xml; we'll tell Maven what plugin to use to build Scala files and we'll advise it of the existence of a URL where it can find the plugin in question. Our updated pom.xml should look like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.blogspot.whileonefork</groupId>
  <artifactId>mvn-scala-trial1</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>Sample Project for Blog</name>
  <url>http://com.blogspot.whileonefork</url>
  
  <!-- Notify Maven it can download things from here -->
  <repositories>
    <repository>
      <id>scala-tools.org</id>
      <name>Scala-tools Maven2 Repository</name>
      <url>http://scala-tools.org/repo-releases</url>
    </repository>
  </repositories>  
  
  <dependencies>
    <!-- Scala version is very important. Luckily the plugin warns you if you don't specify: 
        [WARNING] you don't define org.scala-lang:scala-library as a dependency of the project -->
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>2.9.0-1</version>
    </dependency>
  </dependencies>
  
  <build>  
    <!-- add the maven-scala-plugin to the toolchain -->
    <plugins>
      <plugin>
        <groupId>org.scala-tools</groupId>
        <artifactId>maven-scala-plugin</artifactId>
        <version>2.14.2</version>
      </plugin>
    </plugins>
  </build>
</project>

If you run mvn scala:compile in the mvn-scala-project directory the project should build successfully and in /target/classes you should find two .class files: HelloWorldJustLikeJava$.class and HelloWorldJustLikeJava.class. Scala (usually) compiles the .class files to run on the JVM, just like Java.

All the usual Java tools can be used on these .class files; for example if we run javap on HelloWorldJustLikeJava$ we see the Scala source was compiled to Java; in this case fairly predictable Java:
public final class HelloWorldJustLikeJava$ extends java.lang.Object implements scala.ScalaObject{
    public static final HelloWorldJustLikeJava$ MODULE$;
    public static {};
    public void main(java.lang.String[]);
}
Digging a little deeper, using javap -verbose, we can see that our HelloWorldJustLikeJava class invokes HelloWorldJustLikeJava$:
public static final void main(java.lang.String[]);
  Code:
   Stack=2, Locals=1, Args_size=1
   0:   getstatic       #11; //Field HelloWorldJustLikeJava$.MODULE$:LHelloWorldJustLikeJava$;
   3:   aload_0
   4:   invokevirtual   #13; //Method HelloWorldJustLikeJava$.main:([Ljava/lang/String;)V
   7:   return
HelloWorldJustLikeJava$ provides a simple implementation of main and some other odds and ends:

public static final HelloWorldJustLikeJava$ MODULE$;

public static {};
  Code:
   Stack=1, Locals=0, Args_size=0
   0:   new     #9; //class HelloWorldJustLikeJava$
   3:   invokespecial   #12; //Method "":()V
   6:   return

public void main(java.lang.String[]);
  Code:
   Stack=2, Locals=2, Args_size=2
   0:   getstatic       #19; //Field scala/Predef$.MODULE$:Lscala/Predef$;
   3:   ldc     #22; //String Hello, World
   5:   invokevirtual   #26; //Method scala/Predef$.println:(Ljava/lang/Object;)V
   8:   return
  LineNumberTable:
   line 3: 0

  LocalVariableTable:
   Start  Length  Slot  Name   Signature
   0      9      0    this       LHelloWorldJustLikeJava$;
   0      9      1    args       [Ljava/lang/String;

The practice of outputting additional classes beyond what you directly coded is VERY common in Scala. A class using closures and other features of Scala will often output an exciting wack of $whatever classes.

Anyway, at this point the objective of making our project compile with Maven is achieved; we can now move on to getting it setup to edit in Eclipse. In the mvn-scala-project directory (the same one as pom.xml), create a .classpath file and a .project file:

.classpath
<?xml version="1.0" encoding="UTF-8"?>
<classpath>
  <classpathentry kind="src" path="src/main/scala"/>
 <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
 <classpathentry kind="con" path="org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER"/>
 <classpathentry kind="con" path="org.scala-ide.sdt.launching.SCALA_CONTAINER"/>
 <classpathentry kind="output" path="target/classes"/>
</classpath>


.project
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
 <name>mvn-scala-project</name>
 <comment></comment>
 <projects>
 </projects>
 <buildSpec>
  <buildCommand>
   <name>org.scala-ide.sdt.core.scalabuilder</name>
   <arguments>
   </arguments>
  </buildCommand>
  <buildCommand>
   <name>org.maven.ide.eclipse.maven2Builder</name>
   <arguments>
   </arguments>
  </buildCommand>
 </buildSpec>
 <natures>
  <nature>org.maven.ide.eclipse.maven2Nature</nature>
  <nature>org.scala-ide.sdt.core.scalanature</nature>
  <nature>org.eclipse.jdt.core.javanature</nature>
 </natures>
</projectDescription>

At this point your project should contain the following:
mvn-scala-project
|--.classpath
|--.project
|--pom.xml
`--src
   `--main
      `--scala
         |--HelloWorldJustLikeJava.scala
`target
   `--classes
      |--HelloWorldJustLikeJava$.class
      |--HelloWorldJustLikeJava.class
If you happened to run mvn clean the target directory may be absent.

In Package Explorer in Eclipse your project should look similar to:


And now we can edit our Scala in Eclipse and compile with Maven, and can therefore easily load our project into just about any continuous integration server. Hooray!

EDIT: Assuming you are using m2eclipse, you will want to make sure your Eclipse IDE boots using a JDK VM; this is accomplished by editing eclipse.ini and adding a -vm argument. Bumping memory up a bit usually helps too; the file I am currently using on a Windows dev box follows:
-startup
plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.2.R36x_v20101222
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-vm
C:/Program Files/Java/jdk1.6.0_07/bin/javaw.exe
--launcher.defaultAction
openFile
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx768m
EDIT2: If you want the project to build a bit more gracefully in Eclipse consider binding Scala compile to run by default (eg when you run mvn clean install instead of the somewhat novel mvn scala:compile). Modify the plugin element to add the execution:
<build>  
    <!-- add the maven-scala-plugin to the toolchain -->
    <plugins>
      <plugin>
        <groupId>org.scala-tools</groupId>
        <artifactId>maven-scala-plugin</artifactId>
        <version>2.15.2</version>
        <executions>
            <execution>
                <goals><goal>compile</goal></goals>
            </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

Friday, June 3, 2011

When the UI designer doesn't use the UI

Suppose you wanted to provide a basic interface where users could set training goals, stuff like "Run 50km in June" perhaps. This seems semi-reasonable right? - well, at least as long as nobody ever wants to set a goal for two weeks.


So surely if I want a goal for one month I can just set Calendar month, pick a day in the month (or maybe it'll even flip to be a month dropdown?), and go, right?

Not so fast bucko, UI has got to be validated and you jackasses aren't going to start putting goals for the past into our system!
Not to worry though, the "calendar month" relative to June 3rd is a slightly weird way to put it but it does make sense. Or does it?
So um ... I can't enter the first because it's in the past, and I must pick the first day of a calendar month ... so it's impossible to specify a goal for June? And you don't even give me an arbitrary start/end date option so I could at least set June 3rd-30th?

Clearly the designer of this never had to actually use it! Arglefarg.

Tuesday, May 31, 2011

The magic of getOrElse

For the last month or so I've been coding in Scala. From time to time I use Option's; they seem pretty handy; it's much nicer to get an Option[T] back instead of a null. I semi-often find myself writing things like:

//where f is a function returning Option[Something]
   val t = f(args) getOrElse 3

Recently I realized (classic lightbulb moment :) that getOrElse actually takes a function as the argument:

def getOrElse [B >: A] (default: ⇒ B): B
  //Returns the option's value if the option is nonempty, otherwise return the result of evaluating default.

Why does this matter? It means that the code passed to getOrElse doesn't run at all (default is not evaluated) if the Option is defined! For example, this code:

object GetOrElseTest {
 def main(args : Array[String]) : Unit = {
  def a() = Some(3)
  def b() = None
  
  val aV = a getOrElse { println("nothing from a :("); 1 }
  val bV = b getOrElse { println("nothing from b :("); 1 }
  
  println("aV=%d, bV=%d".format(aV, bV)) 
  
  0
 }
}

Will print this output (note that the println for getting a, which is defined, never runs):

nothing from b :(
aV=3, bV=1

That's awesome and really hard to accomplish in Java. Imagine if log4j could do this; suddenly we wouldn't have any issues with things log.debug("a" + objectThatIsExpensiveToToString.toString()) having a runtime cost even when debug isn't enabled because we wouldn't evaluate the function to create a message at all if debug is off.

The new uniforms are pretty snappy eh first officer!

Monday, May 30, 2011

How to print Ivy's cachepath one artifiact per line with Ant

Many moons ago one Andrew Beacock shared how to use to dump out a classpath in a human-readable fashion (see http://blog.andrewbeacock.com/2005/08/pretty-printing-java-classpaths-using.html). The amount of Ant pain this gem saved me back in the day is ... well a lot.

Fast forward to 2011 and we're using Ant with Ivy as our dependency manager. We use Ivy's retrieve Ant task to get our artifacts but how do we print out a simple list of what Ivy got for us? Well, it turns out cachepath can get this for us, optionally filtered if we see fit:

<!-- after a resolve; in our case this target depends on our resolving target -->
    <ivy:cachepath pathid="ivy.cachepath" settingsRef="ivy.settings" />
    <pathconvert pathsep="${line.separator}  "
                 property="dependency.list"
                 refid="ivy.cachepath"/>
    <echo>DEPENDENCY LIST</echo>
    <echo>  ${dependency.list}</echo>

This will print something along the lines of:

[echo]   C:\...\.ivy2\cache\org.slf4j\slf4j-api\jars\slf4j-api-1.6.1.jar
     [echo]   C:\...\.ivy2\cache\org.slf4j\jcl-over-slf4j\jars\jcl-over-slf4j-1.6.1.jar
     [echo]   C:\...\.ivy2\cache\org.slf4j\log4j-over-slf4j\jars\log4j-over-slf4j-1.6.1.jar
     [echo]   C:\...\.ivy2\cache\ch.qos.logback\logback-classic\jars\logback-classic-0.9.28.jar
     [echo]   C:\...\.ivy2\cache\ch.qos.logback\logback-core\jars\logback-core-0.9.28.jar

Once in a while this is a lifesaver as it makes it easy to see a simple list of our dependencies. Since we print it out line-by-line it's now nice and easy to grep through and find specific things. Like say what version of the components in such and such a group we are getting.

Thursday, April 21, 2011

Deceiving yourself in Scala

I found a cool way to trip myself in Scala today. This is what I meant:
private val lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object]
...
locks.get(key)
This is what I actually typed:
private def lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object]
...
locks.get(key)
Because I put 'def lockByKey' instead of 'val lockByKey' I created myself a function that creates a new ConcurrentHashMap every time it gets called instead of a single instance. Scala allows leaving out brackets in many cases so code like lockByKey.get(...) compiled fine, but as every access was creating a new map things didn't work entirely correctly.

Thankfully unit tests of basic functionality (eg ability to get the same value back twice) caught that something was awry and I was able to identfy that I had cleverly created a factory function rather than a static instance. Thank god for unit tests ;)

Dynamic lock allocation

Suppose you had some code that manipulated things in the filesystem and you wanted to ensure only one thread worked on any given file at a time. Various parts of the code may construct a File object and they then desire to synchronize access to it. However, each part of the code may have a different instance of the File so while the instances are equal (.equals would return true in Java) they are not the same instance (== would not return true in Java. Visually:
We could just declare a lock and synchronize on that, but that means that regardless of how many different resources we have only one thread can use any of them. It would be handy if we could say (pseudo-code):

val resource = new SomeResourceOrOther("""blah""")
   val lock = lockFor(resource )
   lock synchronized {
      //do stuff w/exclusive access to resource
   }

By acquiring a different lock for each resource we ensure optimal concurrency. This is somewhat similar to how structures like java.util.concurrent.ConcurrentHashMap partition their data into many segments and allocate a key per segement, thus potentially allowing as many threads as there are segments to safely operate on the data structure concurrently. We can build such a structure fairly easily:

object Locks {
 private val lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object] 
 @tailrec
 def lockFor(key: Any): Object = {
  val lock = lockByKey.get(key)
  if (lock == null) {
   lockByKey.putIfAbsent(key, new Object())
   lockFor(key)
  } else {
   lock
  }
 }

...

class ConsumerOfLocks {
 val resource = new String("Maybe its a string?") //contrived means of insuring different instance!
 Locks.lockFor(resource) synchronized {
  //wherein I have exclusive access to resource
 }
}


Obviously the resource could be a file/directory or just about anything else. This structure can be particularly useful if you have multiple dynamic local resources you want to synchronize access to. If you move to a lock that is distributed in some manner then the same structure works across multiple nodes. For example, I have used a logically similar structure to ensure that only one node runs a given batch process.

This structure works best if the same set of resources are used consistently. If over time we are continually creating and locking new resources then we have to introduce additional complexity to ensure the lock map doesn't simply grow forever.

Source samples are for Scala 2.8.1.

Thursday, March 10, 2011

100% - 30px: Mixed-unit calculation in Firefox 4

Firefox 4 (see release notes, or dev edition thereof) adds some pretty cool new features. I love the tab candy.

A fairly big one that almost slipped under my radar is that they shipped an initial implementation of CSS calc: "Support for -moz-calc has been added. This lets you specify  values as mathematical expressions." (see https://developer.mozilla.org/en/CSS/-moz-calc).

Support for calc is kind of a big deal as it allows you to do things that are annoying as hell to accomplish today. For example, perhaps you'd like to have three columns, left and right fixed width and center resized to fill available space:

[200px][100% - 400px, minimum 250px][200px]

The problematic bit is the middle column - it's actually fairly arcane to setup mixed unit calculations presently. calc makes it easy:
<!DOCTYPE html> 
<html>
 <head>
  <title>-moz-calc</title>
  <style type="text/css">
   body { margin: 0px }
   #container {
    min-width: 650px;
    width: 100%;
   }
   #left {
    width: 200px;
    background-color: #F0F0F0; 
    float: left;
   }
   #center {
    min-width: 250px;
    width: -moz-calc(100% - 400px);
    background-color: #A0A0A0;
    float: left; 
   }
   #right {
    width: 200px;
    background-color: #F0F0F0;
    float: left; 
   }
  </style>
 </head>
 <body>
  <div id="container">
   <div id="left">left!</div>
   <div id="center">center!!</div>
   <div id="right">righter!</div>
  </div> 
 </body>
</html>
This snippet produces fixed-width left and right columns with a center column whose width is dynamically calculated using a mixture (% and px) of length units. The container div is used to prevent the left/center/right divs from dropping to the next line if the browser is sized down far enough the min-width on the center kicks in (we don't want it infinitely narrow!). Images below show the result before/after resizing the browser window:


The ability to easily (the ease being the key here; it was possible to pull off similar results before but not easily) create page structures using mixed-unit calculated sizes is an AWESOME improvement. Now all we need is a full implementation in Firefox and the other browsers :) Oh and user adoption sufficient to justify use on significant sites.