tag:blogger.com,1999:blog-49499646677799448812024-03-18T04:19:02.573-07:00One Forkwhile(1) fork();Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.comBlogger56125tag:blogger.com,1999:blog-4949964667779944881.post-90735271820685398612012-11-09T16:14:00.001-08:002012-11-09T16:14:31.840-08:00Unit Testing with an In-Memory Database for a Spring/Hibernate Project<br />
<h2>
The joy of Hibernate</h2>
<br />
For tasks not on a critical load/performance path I often find Hibernate handy to whip together some simple data access, particularly for simple CRUD. However, being a bungling buffoon I usually mess up my Hibernate mapping, setup queries that refer to non-existent properties, and so on. Naturally I aspire to resolve this by having automated tests of my DB code.<br />
<br />
Testing DB code is often painful with any significant number of people. Often you either need every developer to have their own database, with it's own uniquely inconsistent data and ways of failing, or to use a shared database in which case people often manage to produce tests that don't work if more than one person runs the tests concurrently. Also testing on a "real" DB is slow and subject to failure for reasons not related to your code. To give a few of many examples, Joe removes columns at random, Wendy has a gift for locking tables, Tim has a test replication setup that doesn't work and stops his tests from running, Alice writes tests that only work if run in the correct order (Test B presumes data from test A still exists), and the continuous integration system doesn't have a DB of it's very own. Also the DBA randomly shuts down the shared DB and causes everyone's tests to fail.<br />
<br />
So, let's try to set ourselves up so our tests can't share data, run fast, yield consistent results, and are not subject to external changes (like a common database changing) causing them to break. Our goal is that we spin up a new, clean, in-memory database for each test, run the test, and then dump the in-memory db.<br />
<br />
In this article I walk through a complete test setup using Spring 3.1, Hibernate 4, and HSQLDB in a Maven project using Eclipse (with Maven Integration for Eclipse) as an editor. I built a fairly similar setup (same idea but built with Ant and Ivy) for tests on some projects I work with at my day job with considerable success so perhaps this will be helpful to someone else.<br />
<br />
A significant portion of the post is devoted to setting up the project with dependencies, Hibernate configuration, Spring configuration, and all the other nonsense accompanying a modern Java project. Feel free to skip to the final section if you just want to glance over how the in-memory testing is wired in.<br />
<h2>
Basic Project Setup</h2>
<div>
Create a new Java Project (force of habit; I never remember the 'new Maven project option until too late, including when blogging):</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-0LYvyOpID81Q_RRKoRd6RdcvnzgyTTi-PF37_3sCNaNwkt_eeCLHZwtDE88ySCLP7vPSyK1zoD741VheUbjSoR37RZMiCl9bNX1rwsTVuItnfFEvNbzO3zXOLZp4loqCi3i0qTC3KejD/s1600/InMemoryDb-newjavaproject.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-0LYvyOpID81Q_RRKoRd6RdcvnzgyTTi-PF37_3sCNaNwkt_eeCLHZwtDE88ySCLP7vPSyK1zoD741VheUbjSoR37RZMiCl9bNX1rwsTVuItnfFEvNbzO3zXOLZp4loqCi3i0qTC3KejD/s320/InMemoryDb-newjavaproject.png" width="238" /></a></div>
<div>
<br /></div>
<div>
Setup Maven style folders:</div>
<div>
<ol>
<li>Delete the 'src' folder</li>
<li>Create four new source folders</li>
<ol>
<li>src/main/java</li>
<li>src/main/resources</li>
<li>src/test/java</li>
<li>src/test/resources</li>
</ol>
</ol>
<div>
Your project should look something like this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3uFv5DZIxuv-T1-QzjPNG9nSNTRwYGKCpPCmnVDheVSe20JkiOJeFkP07OJHKyKohlXaDOFKHzEOws_LpeFAvpb73TVg0KVTrjSP-_6S2BF7SyUpBAk976s8YHlBL1cX-mioJfWb6be-S/s1600/InMemoryDb-sourcefolders.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3uFv5DZIxuv-T1-QzjPNG9nSNTRwYGKCpPCmnVDheVSe20JkiOJeFkP07OJHKyKohlXaDOFKHzEOws_LpeFAvpb73TVg0KVTrjSP-_6S2BF7SyUpBAk976s8YHlBL1cX-mioJfWb6be-S/s1600/InMemoryDb-sourcefolders.png" /></a></div>
<div>
<br /></div>
<div>
Now right-click the project and choose Configure>Convert to Maven Project. Use the default values for Maven setup:</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdVb1Eo6dfAuDpy0Tyfk9asu1liVifHcKk9L-7XsOCYMRQIvOX4llqQuhu2hpJgtHDzTbN7vkLG0FsjwtqJ5ntOPCQA48vWwwt4vsWDgFK9F61qapYUrI1aYnLo8hhkqActcZcu-G6Y1BR/s1600/InMemoryDb-converttomaven.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdVb1Eo6dfAuDpy0Tyfk9asu1liVifHcKk9L-7XsOCYMRQIvOX4llqQuhu2hpJgtHDzTbN7vkLG0FsjwtqJ5ntOPCQA48vWwwt4vsWDgFK9F61qapYUrI1aYnLo8hhkqActcZcu-G6Y1BR/s320/InMemoryDb-converttomaven.png" width="320" /></a></div>
<div>
<br /></div>
<div>
Now you have a pom.xml in the root of your project where you can setup dependencies and compilation options. The first change we want to make is to tell our project to build with Java 1.6 so we can use the spiffy <a href="http://docs.oracle.com/javaee/6/api/javax/annotation/Resource.html">@Resource</a> annotation in our tests:</div>
<div>
<pre class="Xml" name="code"><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>InMemoryDbTests</groupId>
<artifactId>InMemoryDbTests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<plugins>
<!-- Please be a Java 1.6 application -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
</pre>
</div>
<div>
<br />
Now we're ready for dependencies.<br />
<h2>
Dependencies</h2>
</div>
We "simply" need to get an obscure set of jars in compatible versions and all will be ready to go! Trying to get all required dependencies with everything compatible is possibly one of the least fun jobs in all of Java. But I digress. Add dependencies to pom.xml, shown below with comments indicating what each library is for:<br />
<br />
<pre class="Xml" name="code"><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>InMemoryDbTests</groupId>
<artifactId>InMemoryDbTests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<!-- hibernate is a core part of the exercise -->
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>4.1.7.Final</version>
</dependency>
<!-- connection pooling -->
<dependency>
<groupId>commons-dbcp</groupId>
<artifactId>commons-dbcp</artifactId>
<version>1.4</version>
</dependency>
<!-- for the Spring org.springframework.orm.hibernate4.LocalSessionFactoryBean -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>3.1.2.RELEASE</version>
</dependency>
<!-- we want Spring -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>3.1.2.RELEASE</version>
</dependency>
<!-- we want Spring to manage our transactions -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<version>3.1.2.RELEASE</version>
</dependency>
<!-- Spring annotations work via proxies which use CGLIB -->
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib-nodep</artifactId>
<version>2.2.2</version>
</dependency>
<!-- TEST DEPENDENCIES -->
<!-- provides the Spring JUnit test runner & @ContextConfiguration -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>3.1.2.RELEASE</version>
<scope>test</scope>
</dependency>
<!-- to run our tests with -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
<!-- our in-memory database provider -->
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.2.8</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Please be a Java 1.6 application -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
</pre>
If you think this is not strictly simple nor particularly obvious I would say you are right. <a href="http://mvnrepository.com/">http://mvnrepository.com/</a> is your friend if you know you want a dependency (say Spring test) but don't know the exact artifact, version, or what have you for it.<br />
<h2>
Data Access Configuration</h2>
<div>
OK, so we've got all our dependencies. Now we're ready to setup for our tests. Let us hypothesize that:</div>
<div>
<ul>
<li>in src/main/resources we have persistence.xml, a Spring file that defines our key data access beans</li>
<ul>
<li>A data source bean named "dataSource"</li>
<li>A session factory named "sessionFactory"</li>
<li>A transaction manager, in this case the Spring Hibernate 4 Transaction manager</li>
<ul>
<li>Applied automatically via Spring's annotation-driven transaction management</li>
</ul>
<li>The key properties of all beans are set via properties</li>
</ul>
<li>in src/main/resources we have db.properties</li>
<ul>
<li>This specifies the DB driver, URL, password, and so on</li>
</ul>
</ul>
<div>
For test purposes we could override the data source and/or session factory completely, but because we factored out most of the key settings into properties we can just override a few key values from db.properties at test time.</div>
</div>
<div>
<br /></div>
<div>
On to the gory details. Create a persistence.xml file in src/main/resources with the following content:</div>
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.1.xsd">
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.pwd}"/>
<property name="defaultAutoCommit" value="${jdbc.defaultAutoCommit}"/>
<property name="accessToUnderlyingConnectionAllowed" value="${jdbc.accessToUnderlyingConnectionAllowed}"/>
<property name="initialSize" value="${jdbc.initialSize}"/>
<property name="maxActive" value="${jdbc.maxActive}"/>
<property name="maxIdle" value="${jdbc.maxIdle}"/>
<property name="minIdle" value="${jdbc.minIdle}"/>
<property name="maxWait" value="${jdbc.maxWait}"/>
<property name="validationQuery" value="${jdbc.validationQuery}"/>
<property name="testOnBorrow" value="${jdbc.testOnBorrow}"/>
<property name="testOnReturn" value="${jdbc.testOnReturn}"/>
<property name="testWhileIdle" value="${jdbc.testWhileIdle}"/>
<property name="timeBetweenEvictionRunsMillis" value="${jdbc.timeBetweenEvictionRunsMillis}"/>
<property name="numTestsPerEvictionRun" value="${jdbc.numTestsPerEvictionRun}"/>
<property name="minEvictableIdleTimeMillis" value="${jdbc.minEvictableIdleTimeMillis}"/>
</bean>
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="mappingResources">
<list>
<value>hibernate-mapping.xml</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.hbm2ddl.auto">none</prop>
<prop key="hibernate.connection.release_mode">auto</prop>
<prop key="hibernate.show_sql">${hibernate.showSql}</prop>
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.use_sql_comments">true</prop>
<prop key="hibernate.generate_statistics">true</prop>
<prop key="hibernate.jdbc.use_scrollable_resultset">true</prop>
<prop key="hibernate.jdbc.use_streams_for_binary">true</prop>
<prop key="hibernate.jdbc.batch_size">${hibernate.batchSize}</prop>
<prop key="hibernate.cache.region_prefix">hibernate.cache</prop>
<prop key="hibernate.cache.use_query_cache">false</prop>
<prop key="hibernate.cache.use_second_level_cache">false</prop>
</props>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
</beans>
</pre>
<div>
<br /></div>
<div>
Next, create a db.properties file in src/main/resources with the following (obviously dummy; we don't care about the "real" db connection setup for this example) values:<br />
<pre>jdbc.driverClassName=DB DRIVER CLASS, SAY SQL OR ORACLE
jdbc.url=SOME DB URL
jdbc.username=USERNAME
jdbc.pwd=PWD
jdbc.defaultAutoCommit=false
jdbc.accessToUnderlyingConnectionAllowed=false
jdbc.initialSize=5
jdbc.maxActive=100
jdbc.maxIdle=-1
jdbc.minIdle=${jdbc.initialSize}
jdbc.maxWait=15000
jdbc.validationQuery=SELECT DATE OR SOMETHING
jdbc.testOnBorrow=true
jdbc.testOnReturn=false
jdbc.testWhileIdle=false
jdbc.timeBetweenEvictionRunsMillis=-1
jdbc.numTestsPerEvictionRun=3
jdbc.minEvictableIdleTimeMillis=1800000
hibernate.dialect=A DIALECT
hibernate.showSql=false
#0 = off
hibernate.batchSize=0
</pre>
In reality some of these would vary by environment (DB URL, user/pwd for sure) and might want to be further externalized rather than bundled in here.<br />
<br />
<h2>
Hibernate Configuration</h2>
<div>
We need some sort of Hibernate entity we can use to create a sample in-memory test. We'll setup a simplified person class and map it. In src/main/resources create hibernate-mapping.xml with the following mapping:</div>
<div>
<pre class="Xml" name="code"><?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping package="com.blogspot.whileonefork.domain">
<class name="com.blogspot.whileonefork.domain.Person" table="people">
<id name="id" type="long" column="id" unsaved-value="0">
<generator class="identity"/>
</id>
<property name="firstName" column="first_name" />
<property name="lastName" column="last_name" />
</class>
</hibernate-mapping>
</pre>
</div>
<div>
In src/main/java in the com.blogspot.whileonefork.domain create the Person class:</div>
<div>
<pre class="Java" name="code">package com.blogspot.whileonefork.domain;
public class Person {
private Long id;
private String firstName;
private String lastName;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
}
</pre>
<div>
Finally we'll make ourselves a simple helper class to manage transactions and encapsulate simple CRUD operations. In src/main/java in the com.blogspot.whileonefork.db package create DaoAid.java with the following content:
<br />
<pre class="Java" name="code">package com.blogspot.whileonefork.db;
import java.util.List;
import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.criterion.Criterion;
import org.springframework.transaction.annotation.Isolation;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
public class DaoAid {
private static final int MAX_RESULT_SIZE = 128;
private SessionFactory sessionFactory;
public SessionFactory getSessionFactory() {
return sessionFactory;
}
public void setSessionFactory(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}
private Session getSession() {
return sessionFactory.getCurrentSession();
}
@Transactional(isolation = Isolation.READ_UNCOMMITTED, propagation = Propagation.REQUIRED)
public <T> List<T> list(Class<T> classOfT,
List<? extends Criterion> criteriaList, int maxResults) {
if (null == classOfT) {
throw new IllegalArgumentException("classOfT");
}
if (maxResults < 1) {
throw new IllegalArgumentException("maxResults must be >= 1");
}
Session session = getSession();
org.hibernate.Criteria criteria = session.createCriteria(classOfT);
criteria.setMaxResults(Math.min(maxResults, MAX_RESULT_SIZE));
if (null != criteriaList) {
for (Criterion criterion : criteriaList) {
criteria.add(criterion);
}
}
@SuppressWarnings("unchecked")
List<T> list = criteria.list();
return list;
}
@Transactional(isolation = Isolation.READ_UNCOMMITTED, propagation = Propagation.REQUIRED)
public <T> T findById(Class<T> classOfT, long id) {
@SuppressWarnings("unchecked")
T result = (T) getSession().get(classOfT, id);
return result;
}
@Transactional(isolation = Isolation.READ_COMMITTED, propagation = Propagation.REQUIRED)
public <T> T write(T instance) {
@SuppressWarnings("unchecked")
T savedInstance = (T) getSession().merge(instance);
return savedInstance;
}
@Transactional(isolation = Isolation.READ_COMMITTED, propagation = Propagation.REQUIRED)
public <T> void delete(T instance) {
getSession().delete(instance);
}
}
</pre>
</div>
<h2 style="font-family: 'Times New Roman'; white-space: normal;">
In Memory Testing</h2>
<div>
At long last we find our way to the point, doing some Hibernate operations against an in-memory database. We want to verify we can interact with a clean database for each test. We'll create a base class that sets up and tears down our in-memory database before/after each test, then write a simple test to do some data access to show it all works.<br />
<br />
First of all, our base class for in-memory tests. In src/test/java, in the com.blogspot.whileonefork.db package, create InMemoryDbTestBase.java with the following content:<br />
<pre class="Java" name="code">package com.blogspot.whileonefork.db;
import java.sql.Connection;
import javax.annotation.Resource;
import javax.sql.DataSource;
import org.hibernate.cfg.Configuration;
import org.hibernate.tool.hbm2ddl.SchemaExport;
import org.junit.After;
import org.junit.Before;
import org.springframework.orm.hibernate4.LocalSessionFactoryBean;
import org.springframework.test.context.ContextConfiguration;
@ContextConfiguration
public class InMemoryDbTestBase {
//&: get the actual factory, not the object it produced
@Resource(name="&sessionFactory")
private LocalSessionFactoryBean sf;
@Resource
private DataSource dataSource;
private Configuration cfg;
private Connection conn;
@Resource
private DaoAid daoAid;
@Before
public void setup() throws Exception {
if (null == cfg) {
if (null == sf) {
throw new IllegalStateException("No LocalSessionFactoryBean; perhaps you didn't setup @RunWith and @ContextConfiguration on your test?");
}
cfg = sf.getConfiguration();
}
conn = dataSource.getConnection();
SchemaExport exporter = new SchemaExport(cfg, conn);
exporter.execute(true, true, false, true);
if (null != exporter.getExceptions() && exporter.getExceptions().size() > 0) {
throw new IllegalStateException("Unable to setup schema; export failed");
}
}
@After
public void teardown() throws Exception {
//see http://hsqldb.org/doc/guide/running-chapt.html#rgc_closing_db
if (null != conn) {
conn.createStatement().execute("SHUTDOWN");
conn.close();
conn = null;
}
}
public DaoAid getDaoAid() {
return daoAid;
}
public void setDaoAid(DaoAid daoAid) {
this.daoAid = daoAid;
}
}
</pre>
Creating a new connection to the HSQLDB will spin up a new, blank, database. The SchemaExport class will then create schema to match our hibernate mapping. Finally in teardown, run after each test, we shutdown the database which will release the in-memory db.<br />
<br />
We have slapped @ContextConfiguration onto InMemoryDbTestBase so we need a Spring file to setup the beans for the test. Create src/test/resources/com/blogspot/whileonefork/db/InMemoryDbTestBase-context.xml with the following content:<br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.1.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.1.xsd">
<!-- for an in-memory test we need to load the db properties, then the test properties, then the persistence bits -->
<context:property-placeholder location="classpath:db.properties, classpath:db.inmemory-overrides.properties"/>
<import resource="classpath:/persistence.xml" />
</beans>
</pre>
Note that we load db.properties <i>before</i> db.inmemory-overrides.properties. This allows db.inmemory-overrides.properties to overwrite any properties of the same name in db.properties. We then import persistence.xml, which will load using the in memory db driver and connection URL. We only need to replace a few properties. Create src/test/resources/db.inmemory-overrides.properties with the following content:<br />
<pre>jdbc.driverClassName=org.hsqldb.jdbcDriver
jdbc.url=jdbc:hsqldb:mem:unittestdb
jdbc.username=sa
jdbc.pwd=
jdbc.validationQuery=values(current_timestamp)
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.showSql=true
</pre>
So, we should now be able to author a test that does some data access and verifies that it operates on a clean database. Create PersonListOfOneTest.java in src/test/java, in the com.blogspot.whileonefork.domain package with the following content:<br />
<pre class="Java" name="code">package com.blogspot.whileonefork.domain;
import static org.junit.Assert.assertEquals;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import com.blogspot.whileonefork.db.InMemoryDbTestBase;
/**
* Save an item and list Person's with no criteria, twice.
* Should both see 1 item because DB is scrubbed each time.
*
*/
@RunWith(SpringJUnit4ClassRunner.class)
public class PersonListOfOneTest extends InMemoryDbTestBase {
@Test
public void list1() {
assertEquals(0, getDaoAid().list(Person.class, null, 8).size());
getDaoAid().write(new Person());
assertEquals(1, getDaoAid().list(Person.class, null, 8).size());
}
@Test
public void list2() {
assertEquals(0, getDaoAid().list(Person.class, null, 8).size());
getDaoAid().write(new Person());
assertEquals(1, getDaoAid().list(Person.class, null, 8).size());
}
}
</pre>
Execute the test; it should pass. This means our code successfully:<br />
<br />
<ol>
<li>Created a connection to a new HSQLDB in-memory instance for each test</li>
<li>Ran the Hibernate SchemaExport to build the desired table structure to allow data access</li>
<li>Wrote a record into the in-memory database and loaded it back</li>
</ol>
<div>
If the in-memory database wasn't getting cleaned up between tests then the second test to execute would return two records from the call to list(). Mission accomplished! </div>
<div>
<br /></div>
<div>
Your final project should look like this:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5gel8xjOPfbnQh3k04Tw_bTc52iSyqJPQbee-04Xnn-QqjgWvd6TFzIhNQn3oASJxT_4ASL9r9uRMoNdUO6Z0Ga9ssx4fxCRIY8az_-rDQenrPCIOY5DuK2FsoIRXi_ABxOdtNAJdSBIn/s1600/InMemoryDb-final.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5gel8xjOPfbnQh3k04Tw_bTc52iSyqJPQbee-04Xnn-QqjgWvd6TFzIhNQn3oASJxT_4ASL9r9uRMoNdUO6Z0Ga9ssx4fxCRIY8az_-rDQenrPCIOY5DuK2FsoIRXi_ABxOdtNAJdSBIn/s320/InMemoryDb-final.png" width="228" /></a></div>
<div>
<br /></div>
<div>
In practice I have used a similar approach to test both simple and complex data access across a large project with an extensive test suite executing fast and with reliable results and near elimination of problems involving data access. </div>
</div>
</div>
<div>
<br /></div>
</div>
Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com63tag:blogger.com,1999:blog-4949964667779944881.post-23718887100515620862012-02-17T13:40:00.000-08:002012-02-17T13:40:51.315-08:00Moving from Google Translate API to Microsoft Translate API in ScalaGoogle Translate used to be gods gift to developers who want to verify their internationalization works. It made localizing to some random locale for test purposes downright trivial. And then the bastards went and deprecated it (<a href="http://googlecode.blogspot.com/2011/05/spring-cleaning-for-some-of-our-apis.html">http://googlecode.blogspot.com/2011/05/spring-cleaning-for-some-of-our-apis.html</a>), ultimately making it it pay to play.<br />
<br />
The rates for Google Translate are so low it is unlikely we can justify switching to Microsoft Translate based on cost but we love "free" so we'll spend a few expensive hours of developer time on it anyway. M$ translate is free for your first 2M characters and you can pay for more.<br />
<br />
The first thing that one notices upon trying to call a Microsoft web service is that it isn't as easy as you'd like. From reading the <a href="http://api.microsofttranslator.com/V2/Http.svc/Translate">http://api.microsofttranslator.com/V2/Http.svc/Translate</a> API to getting a successful translation call through from code took WAY longer than Google Translate (or other APIs) and had more bumps in the road. I'm sure others have differing experiences but for me I had programmatic access to Google Translate working maybe 30-60 minutes (it was a while ago) after I decided to do it versus 2-3 hours to get Microsoft Translate working.<br />
<br />
Trying to get Microsoft Translate working I ran into the following complications:<br />
<ol>
<li>A multi-step registration process yielding numerous constants you send in to their API’s</li>
<ol>
<li><a href="https://code.google.com/apis/console">https://code.google.com/apis/console</a> beats the hell out of <a href="https://datamarket.azure.com/account">https://datamarket.azure.com/account </a> (the Microsoft equivalent as far as I can tell)</li>
</ol>
<li>An extra http call to get a token that you have to modify before you send it back to them (addRequestHeader("Authorization", "Bearer " + tok.access_token))</li>
<ol>
<li>Bonus points for inconsistent description of how this worked, although I believe this is now fixed</li>
</ol>
<li>Inaccurate documentation of arguments</li>
<ol>
<li>I believe this is now fixed</li>
</ol>
<li>Unhelpful error messages</li>
<li>Inconsistent documentation of how to send in authorization data </li>
<ol>
<li>I believe this is now fixed</li>
</ol>
<li>ISO-639 2-char language codes for all languages <i>except Chinese</i>.</li>
<ol>
<li>Chinese requires use of zh-CHS or zh-CHT to distinguish traditional vs simplified. Apparently having "zh" default to one or the other (probably simplified) is less trouble than having this be the exception case to how everything else works.</li>
</ol>
</ol>
In case this is useful to someone else, here is the code (Scala 2.9.1.final, . First up, the API our clients will call into:<br />
<br />
<pre class="Scala" name="code">class Translator(val client: HttpClient) extends Log {
//We kept Google around just in case we decide to pay for the service one day
private var translationServices = List(new Google(client), new Microsoft(client))
def this() = this(new HttpClient())
def apply(text: String, fromLang: String, toLang: String): String = {
if (fromLang != toLang && StringUtils.isNotBlank(text))
translate(text, fromLang, toLang)
else
text
}
private def translate(text: String, fromLang: String, toLang: String): String = {
for (svc <- translationServices) {
try {
val tr = svc(text, fromLang, toLang)
if (StringUtils.isNotBlank(tr)) {
return tr
}
} catch {
case e: Exception =>
logger.warn("Translation failed using " + svc.getClass().getSimpleName() + ": " + e.getMessage() + ", moving on...")
}
}
return ""
}
}
</pre>
<br />
Consumers will call into the Translator using code similar to:<br />
<br />
<pre class="Scala" name="code"> //translate Hello, World from English to Chinese
var tr = new Translator()
tr("Hello, World", "en", "zh")
</pre>
<br />
The interesting part is of course the actual Microsoft implementation:<br />
<br />
<pre class="Scala" name="code">/**
* The parent TranslationService just defines def apply(text: String, fromLang: String, toLang: String): String
*/
class Microsoft(client: HttpClient) extends TranslationService(client) with Log {
private val tokenUri = "https://datamarket.accesscontrol.windows.net/v2/OAuth2-13"
private val translateUri = "http://api.microsofttranslator.com/V2/Http.svc/Translate"
private val encoding = "ASCII"
private val appKey = Map("client_id" -> "THE NAME OF YOUR APP", "client_secret" -> "YOUR CLIENT SECRET")
private var token = new MsAccessToken
def this() = this(new HttpClient())
/**
* Ref http://msdn.microsoft.com/en-us/library/ff512421.aspx
*/
override def apply(text: String, fromLang: String, toLang: String): String = {
/**
* Always try to re-use an existing token
*/
val firstTry:Option[String] = try {
Some(callTranslate(token, text, fromLang, toLang))
} catch {
case e: Exception =>
logger.info("Failed to re-use token, will retry with a new one. " + e.getMessage())
None
}
/**
* If we didn't get it using our old token try try again.
* 99% of the time we do a bunch in a row and it works first time; occasionally we end up
* needing a new key.
* Code in block won't run unless firstTry is None.
*/
val response = firstTry getOrElse {
this.token = requestAccessToken()
callTranslate(token, text, fromLang, toLang)
}
//response is similar to: <string xmlns="http://schemas.microsoft.com/2003/10/Serialization/">Hallo Welt</string>
val translation = StringUtils.substringAfter(StringUtils.substringBeforeLast(response, "</string>"), ">")
translation
}
private def callTranslate(tok: MsAccessToken, text: String, fromLang: String, toLang: String) = {
val get = new GetMethod(translateUri)
//Thanks MSFT, it's awesome that the language codes are *almost* ISO 639...
//We need to specify our type of Chinese, http://www.emreakkas.com/internationalization/microsoft-translator-api-languages-list-language-codes-and-names
val adjustedToLang = if (toLang.equalsIgnoreCase("zh")) "zh-CHS" else toLang
val queryPairs = Array(
new NameValuePair("appId", ""),
new NameValuePair("text", text),
new NameValuePair("from", fromLang),
new NameValuePair("to", adjustedToLang))
get.setQueryString(queryPairs)
/**
* http://msdn.microsoft.com/en-us/library/hh454950.aspx
*/
get.addRequestHeader("Authorization", "Bearer " + tok.access_token)
val rawResponse = try {
val sc = client executeMethod get
val response = get getResponseBodyAsString ()
if (sc != HttpStatus.SC_OK) {
throw new IllegalArgumentException("Error translating; Microsoft translate request '"
+ translateUri + "?" + get.getQueryString()
+ "' failed with unexpected code " + sc + ", response: " + response)
}
response
}
rawResponse
}
/**
* Ref http://msdn.microsoft.com/en-us/library/hh454950.aspx
*/
def requestAccessToken(): MsAccessToken = {
val post = new PostMethod(tokenUri)
post.setParameter("grant_type", "client_credentials")
post.setParameter("client_id", appKey("client_id"))
post.setParameter("client_secret", appKey("client_secret"))
post.setParameter("scope", "http://api.microsofttranslator.com")
val rawResponse = try {
val sc = client executeMethod post
val response = post getResponseBodyAsString ()
if (sc != HttpStatus.SC_OK) {
throw new IllegalArgumentException("Error translating; Microsoft access token request failed with unexpected code " + sc + ", response: " + response)
}
response
} finally {
post releaseConnection
}
val tok = Json.fromJson[MsAccessToken](rawResponse, classOf[MsAccessToken])
tok
}
}
</pre>
<br />
The MsAccessToken is a rich and exciting class:<br />
<pre class="Scala" name="code"> /**
* Ref http://msdn.microsoft.com/en-us/library/hh454950.aspx
*/
class MsAccessToken(var access_token: String, var token_type: String, var expires_in: String, var scope: String) {
def this() = this(null, null, null, null)
}
</pre>
<br />
<br />
A few third party libraries are in play here. For Http we are using the Apache HttpClient. For JSON we are using Google's excellent <a href="http://code.google.com/p/google-gson/">Gson</a> library, with a simple implementation of 'using' to make working with java.io cleaner:<br />
<br />
<pre class="Scala" name="code">object Json {
def writeJson(something: Any): String = new Gson().toJson(something)
def writeJson(something: Any, os: OutputStream):Unit
= using(new OutputStreamWriter(os)) { osw => new Gson().toJson(something, osw) }
def fromJson[T](json: String, graphRoot: Type):T
= new Gson().fromJson(json, graphRoot)
}
</pre>
<br />
The using() function is in another class; it looks like this (ref http://whileonefork.blogspot.com/2011/03/c-using-is-loan-pattern-in-scala.html):
<br />
<pre class="Scala" name="code"> def using[T <: {def close(): Unit}, R](c: T)(action: T => R): R = {
try {
action(c)
} finally {
if (null != c) c.close
}
}
</pre>
<br />
And with that our Scala calling of M$ Translate is complete!Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com18tag:blogger.com,1999:blog-4949964667779944881.post-970684800306725932011-11-14T08:43:00.001-08:002011-11-14T09:09:45.248-08:00What is in my Core Data managed store?Suppose you have an iOS application. Suppose it uses the <a href="http://developer.apple.com/library/mac/#documentation/cocoa/conceptual/coredata/cdprogrammingguide.html">Core Data</a> API to persist data. And let us further suppose that you are having a few problems; it seems like the data in your store isn't exactly as you expect.<br />
<br />
It might prove useful to be able to get some basic information about what the heck is happening with your data at runtime. For example, let us suppose you have an API in your code that lets you get a reference to the <a href="http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/CoreDataFramework/Classes/NSManagedObjectModel_Class/Reference/Reference.html">NSManagedObjectModel</a>. We can then print out what entities are in your database and how many of them. Create a simple ViewController with a multi-line text view on it and setup an outlet named 'output'. In the viewDidAppear method gather up some basic diagnostic information and spit it out to the output text view:<br />
<pre class="ObjC" name="code">//NOTE: sample assumes ARC, hence no calls to release
//in the .m file for your diagnostic output
- (void)viewDidAppear:(BOOL)animated
{
[super viewDidAppear:animated];
NSString *diagnostics = @"";
@try {
diagnostics = [NSString stringWithFormat:@"%@%@", diagnostics, @"Data Diagnostics\n"];
NSManagedObjectModel *mom = //CALL WHATEVER YOUR APP USES TO GET NSManagedObjectModel
if (mom) {
for (NSEntityDescription *entity in mom.entities) {
NSUInteger countOf = [self countOfEntity:entity];
diagnostics = [NSString stringWithFormat:@"%@ entity:%@ (%d records)\n", diagnostics, entity.name, countOf];
}
} else {
diagnostics = [NSString stringWithFormat:@"%@%@", diagnostics, @" objectModel is nil :(\n"];
}
}
@catch (NSException *exception) {
diagnostics = [NSString stringWithFormat:@"%@%@", diagnostics, @" ERROR getting data diagnostics\n"];
}
output.text = diagnostics;
}
//note that this could return "NSNotFound if an error occurs"; this typically shouldn't be the case since we got the entity description directly from our object model
- (NSUInteger) countOfEntity :(NSEntityDescription*)entity
{
NSFetchRequest *fr = [[NSFetchRequest alloc] init];
fr.entity = entity;
[fr setIncludesSubentities:NO];
NSError *err;
NSManagedObjectContext *moc = //CALL WHATEVER YOUR APP USES TO GET NSManagedObjectContext
NSUInteger count = [moc countForFetchRequest:fr error:&err];
return count;
}
</pre>
If you print this to a text view your output should end up something like this:
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__yB3pZ3sZ3deZxtNV9UBR3JgO9RYjqasuYtgJDI-GQ9DYIyg5cmDVaI8Y65y_B1IMQcjLb2vPyEOjQ2JTPEoUL-164miQcLlv2AiPPol4RbN31O-2nmexCi4H4-FD90UZVO2n8FCjO7E/s1600/diagnostic.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__yB3pZ3sZ3deZxtNV9UBR3JgO9RYjqasuYtgJDI-GQ9DYIyg5cmDVaI8Y65y_B1IMQcjLb2vPyEOjQ2JTPEoUL-164miQcLlv2AiPPol4RbN31O-2nmexCi4H4-FD90UZVO2n8FCjO7E/s320/diagnostic.png" width="320" /></a></div>
For some types of app this is surprisingly useful.<br />
<br />
The main things I personally find this useful for are:<br />
<br />
<ol>
<li>Identifying glaring errors, such as "your model is completely missing an entity type; something is <b>very </b>wrong"</li>
<li>Identifying errors where data isn't added or removed as expected, such as "when you do X in the UI there should be +1 of those (or -2 of those or whatever) if I quickly peek at diagnostics"</li>
<ol>
<li>Hmm ... perhaps it would be useful if the diagnostic view actually told you the delta for each type since you last looked?</li>
</ol>
</ol>
<div>
There are many ways to wire this in; it doesn't have to be via it's own view. Another approach I often find useful to to have a debug/logged mode enabled only in development where this type of information prints to the console when key events occur in the app.</div>
<br />
<br />Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com4tag:blogger.com,1999:blog-4949964667779944881.post-4976558083239017782011-10-16T17:56:00.000-07:002011-10-16T20:02:48.743-07:00How to wait while an NSTimer runs in a unit testSuppose you were a lowly noob to iOS development. Further suppose you had a Cool Idea (tm) which involved timers. The internets might rapidly guide you to <a href="http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Classes/NSTimer_Class/Reference/NSTimer.html">NSTimer</a> and you might decide to try to get it to log to the console in a unit test. The most obvious approach seems to be to setup a timer to tick frequently, lets say every 0.1 seconds, and setup a timer callback that logs something, then make a test that sleeps for a couple of seconds. Presumably during the sleep period we'll see a bunch of timer output. The code might look like this (inside an XCode 4.2 test implementation class):<br />
<br />
<pre class="objc" name="code">- (void)onTimerTick:(NSTimer*)timer
{
NSLog(@"MY TIMER TICKED");
}
- (void)testTimerBasics
{
NSLog(@"timer time");
[NSTimer scheduledTimerWithTimeInterval:0.1
target:self
selector:@selector(onTimerTick:)
userInfo:nil
repeats:YES];
[timer fire]; //manually calling fire DOES log 'MY TIMER TICKED'
NSLog(@"about to wait");
[NSThread sleepForTimeInterval:2.0]; //absolutely no logs of 'MY TIMER TICKED' occur; somehow the time doesn't fire during a thread sleep :(
NSLog(@"wait time is over");
}
</pre>
<br />
Sadly absolutely no log messages are printed during our two second sleep (<span class="Apple-style-span" style="font-family: monospace; white-space: pre;">[NSThread sleepForTimeInterval:2.0]</span>) ; WTF?!<br />
<br />
After much Google and literally in the midst of typing a Stack Overflow question I came across a question involving waiting for something else that mentioned <a href="http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Classes/NSRunLoop_Class/Reference/Reference.html">NSRunLoop</a> in passing. The very existence of a run loop class suggests an answer: <i>our tests run on the same thread as the run loop</i>. This means if we put the run loop to sleep nothing gets processed. Instead of sleep we need some sort of "run the run loop for a while" approach. Luckily it turns out that NSRunLoop provides a runUntilDate API so we can re-write the test above as follows:<br />
<br />
<pre class="objc" name="code">- (void)onTimerTick:(NSTimer*)timer
{
NSLog(@"MY TIMER TICKED");
}
- (void)testTimerBasics
{
NSLog(@"timer time");
NSTimer *timer = [NSTimer scheduledTimerWithTimeInterval:0.1
target:self
selector:@selector(onTimerTick:)
userInfo:nil
repeats:YES];
//[timer fire];
NSDate *runUntil = [NSDate dateWithTimeIntervalSinceNow: 3.0 ];
NSLog(@"about to wait");
[[NSRunLoop currentRunLoop] runUntilDate:runUntil];
NSLog(@"wait time is over");
}
</pre>
We've found the right magic incantation! Knuth would be proud.<br />
<br />
Speaking of magic incantations, I am using the SyntaxHighlighter libraries hosted @ <a href="http://syntaxhighlighter.googlecode.com/svn/trunk/Scripts/">http://syntaxhighlighter.googlecode.com/svn/trunk/Scripts/</a>. However, there is no Objectionable-C brush there so I took the one posted @ http://www.undermyhat.org/blog/wp-content/uploads/2009/09/shBrushObjectiveC.js and updated it the casing and namespace names to the newer highlighter standard. The updated brush looks like this:<br />
<br />
<br />
<pre class="Javascript" name="code">dp.sh.Brushes.ObjC = function()
{
var datatypes = 'char bool BOOL double float int long short id void';
var keywords = 'IBAction IBOutlet SEL YES NO readwrite readonly nonatomic nil NULL ';
keywords += 'super self copy ';
keywords += 'break case catch class const copy __finally __exception __try ';
keywords += 'const_cast continue private public protected __declspec ';
keywords += 'default delete deprecated dllexport dllimport do dynamic_cast ';
keywords += 'else enum explicit extern if for friend goto inline ';
keywords += 'mutable naked namespace new noinline noreturn nothrow ';
keywords += 'register reinterpret_cast return selectany ';
keywords += 'sizeof static static_cast struct switch template this ';
keywords += 'thread throw true false try typedef typeid typename union ';
keywords += 'using uuid virtual volatile whcar_t while';
// keywords += '@property @selector @interface @end @implementation @synthesize ';
this.regexList = [
{ regex: dp.sh.RegexLib.SingleLineCComments, css: 'comments' }, // one line comments
{ regex: dp.sh.RegexLib.MultiLineCComments, css: 'comments' }, // multiline comments
{ regex: dp.sh.RegexLib.DoubleQuotedString, css: 'string' }, // double quoted strings
{ regex: dp.sh.RegexLib.SingleQuotedString, css: 'string' }, // single quoted strings
{ regex: new RegExp('^ *#.*', 'gm'), css: 'preprocessor' }, // preprocessor
{ regex: new RegExp(this.GetKeywords(datatypes), 'gm'), css: 'datatypes' }, // datatypes
{ regex: new RegExp(this.GetKeywords(keywords), 'gm'), css: 'keyword' }, // keyword
{ regex: new RegExp('\\bNS\\w+\\b', 'g'), css: 'keyword' }, // keyword
{ regex: new RegExp('@\\w+\\b', 'g'), css: 'keyword' }, // keyword
];
this.CssClass = 'dp-objc';
this.Style = '.dp-objc .datatypes { color: #2E8B57; font-weight: bold; }';
}
dp.sh.Brushes.ObjC.prototype = new dp.sh.Highlighter();
dp.sh.Brushes.ObjC.Aliases = ['objc'];
</pre>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com41tag:blogger.com,1999:blog-4949964667779944881.post-21217152223628975642011-10-14T16:13:00.000-07:002011-10-14T16:13:53.502-07:00Integrating Javascript tests into a CLI buildWherein we walk through a basic setup for running Javascript unit tests on the command line. After some initial investigation (<a href="http://whileonefork.blogspot.com/2011/07/javascript-unit-tests-with-qunit-ant.html">here</a>) I didn't find time to get back to Javascript unit testing until recently. I have now managed to get Javascript unit tests running fairly gracefully in a command line build at work; here is an outline of how, simplified from the "real" implementation to highlight the basics. Fortunately a great deal of the work is done for us, always nice when it turns out that way.<br />
<br />
We are going to run everything off the filesystem to avoid having our tests impacted by external influences.<br />
<br />
<b>Part 1: Basic test setup</b><br />
<ol>
<li>Create a directory to house Javascript unit test files; we will refer to this as \jsunit henceforth when giving paths. </li>
<li>Download <a href="https://raw.github.com/jquery/qunit/master/qunit/qunit.js">QUnit.js</a> and <a href="https://raw.github.com/jquery/qunit/master/qunit/qunit.css">QUnit.css</a> into \jsunit</li>
<li>Download <a href="http://phantomjs.googlecode.com/git/examples/run-qunit.js">run-qunit.js</a> into \jsunit</li>
<li>Create a file testme.js in \jsunit with the following content<br />
<pre class="Javascript" name="code">/**
* var-args; adds up all arguments and returns sum
*/
function add() {
}
</pre>
</li>
<li>Create a file testme.test.htm in \jsunit with the following content</li>
<ul>
<li>Note we are using local filesystem paths to load all content; we have no external dependencies</li>
<li>
<pre class="Html" name="code"><!DOCTYPE html>
<html>
<head>
<!-- we need QUnit as a test runner -->
<link rel="stylesheet" href="qunit.css" type="text/css" media="screen" />
<script src="qunit.js"></script>
<!-- we'd like to have the file we're going to test -->
<script src="testme.js"></script>
<!-- and finally lets write some tests -->
<script>
console.log("test time baby");
test("add is defined", function() {
equals(typeof window.add, "function", "add isn't a function :(");
});
</script>
</head>
<body>
<h1 id="qunit-header">QUnit Tests</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
<div id="qunit-fixture"></div>
</body>
</html>
</pre>
</li>
</ul>
<li>Download <a href="http://www.phantomjs.org/">PhantomJS</a> (1.3.0 at time of writing)</li>
<ul>
<li>For example PhantomJS commands I will assume it is on PATH (eg <i>phantomjs args</i>); use the qualified path if not (eg <i>C:\where\phantom\is\phantomjs args</i>)</li>
</ul>
<ol>
</ol>
<li>Open testme.test.htm in a browser; it should look like this:</li>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaxH-mFzjnREVgnuAiQtVtMxVAp5OKChyphenhyphenIcY5OivB3u-6QKZkXPDGLW3CsjRlRfFe4-h5x10ZJlgSEJnke-5chnBhyphenhyphenpf8T9KwMMjRk1MWd9kQqCHkKOvjKrg383-I3ljpK8kV3U3VkuiH3/s1600/jstestsample.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="93" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaxH-mFzjnREVgnuAiQtVtMxVAp5OKChyphenhyphenIcY5OivB3u-6QKZkXPDGLW3CsjRlRfFe4-h5x10ZJlgSEJnke-5chnBhyphenhyphenpf8T9KwMMjRk1MWd9kQqCHkKOvjKrg383-I3ljpK8kV3U3VkuiH3/s320/jstestsample.png" width="320" /></a></div>
<ul>
<li><br /></li>
</ul>
<li>Open a command prompt, navigate to \jsunit and run <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">phantomjs run-qunit.js testme.test.htm</span></li>
<ul>
<li>Output should be similar to:<br />
<pre>test time baby
'waitFor()' finished in 211ms.
Tests completed in 57 milliseconds.
1 tests of 1 passed, 0 failed.
</pre>
</li>
<li>Note we don't see any "test blah pass" or "test two fail" style output</li>
</ul>
</ol>
<div>
<b>Part 2: CLI build integration prep</b></div>
<div>
So far so good, now we need to get setup to run in a CLI build. There a couple of things we'd like here, most of which are already implemented in run-qunit.js:</div>
<div>
<ol>
<li>Output each test pass/fail</li>
<li>Output log messages from tests to the console</li>
<ol>
<li>This "just works" courtesy of run-qunit.js, yay!</li>
</ol>
<li>Exit with non-zero error code if tests fail</li>
<ol>
<li>This makes it easy for build to detect failure and do something in response; for example an Ant build could simply set failonerror</li>
<li>This "just works" courtesy of run-qunit.js, yay!</li>
</ol>
</ol>
<div>
We just have to setup output of test pass/fail information to the console. We'll add a test that fails to show what that looks like. Proceed as follows:</div>
</div>
<div>
<ol>
<li>Create a file test-support.js in \jsunit with the following content:<br />
<pre class="Javascript" name="code">//create a scope so we don't pollute global
(function() {
var testName;
//arg: { name }
QUnit.testStart = function(t) {
testName = t.name;
};
//arg: { name, failed, passed, total }
QUnit.testDone = function(t) {
console.log('Test "' + t.name + '" completed: ' + (0 === t.failed ? 'pass' : 'FAIL'))
};
//{ result, actual, expected, message }
QUnit.log = function(t) {
if (!t.result) {
console.log('Test "' + testName + '" assertion failed. Expected <' + t.expected + '> Actual <' + t.actual + '>' + (t.message ? ': \'' + t.message + '\'' : ''));
}
};
}());
</pre>
</li>
<li>Edit testme.test.htm to pull in test-support.js and add a test that will currently fail</li>
<ul>
<li>
<pre class="Html" name="code"><!DOCTYPE html>
<html>
<head>
<!-- we need QUnit as a test runner -->
<link rel="stylesheet" href="qunit.css" type="text/css" media="screen" />
<script src="qunit.js"></script>
<!-- where would our tests be without support! -->
<script src="test-support.js"></script>
<!-- we'd like to have the file we're going to test -->
<script src="testme.js"></script>
<!-- and finally lets write some tests -->
<script>
test("add is defined", function() {
equals(typeof window.add, "function", "add isn't a function :(");
});
test("add 1+1", function() {
equals(add(1, 1), 2);
});
</script>
</head>
<body>
<h1 id="qunit-header">QUnit Tests</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
<div id="qunit-fixture"></div>
</body>
</html>
</pre>
</li>
</ul>
<li>Open a command prompt, navigate to \jsunit and run <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">phantomjs run-qunit.js testme.test.htm</span></li>
<ul>
<li>Output should be similar to:<br /><pre>Test "add is defined" completed: pass
Test "add 1+1" assertion failed. Expected <2> Actual <undefined>
Test "add 1+1" completed: FAIL
'waitFor()' finished in 209ms.
Tests completed in 70 milliseconds.
1 tests of 2 passed, 1 failed.</pre>
</li>
<li>If you print the exit code (<span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">echo %ERRORLEVEL%</span> in Windoze) you should get a 1, indicating we have fulfilled the 'exit with non-zero exit code on failure' requirement :)</li>
</ul>
<ul>
</ul>
</ol>
</div>
<b>Part 3: Ant integration</b><br />
At long last we are ready to integrate this mess into a build. For this example I will use Ant and will assume Ant is on PATH. At time of writing I am using Ant 1.8.2.<br />
<br />
<ol>
<li>Create a phantomjs.bat file in \jsunit with the following content<br />
<pre>@echo off
C:\where\you\put\phantom\phantomjs.exe %*
</pre>
</li>
<ul>
<li>Alternately create phantomjs.sh with equivalent functionality if on *nix</li>
</ul>
<li>Create a build.xml file in \jsunit with the following content</li>
<ul>
<li>
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<project name="jsunittests" basedir="." default="main">
<property name="builddir" location="${basedir}/target"/>
<condition property="phantom.filename" value="phantomjs.bat"><os family="windows"/></condition>
<condition property="phantom.filename" value="phantomjs.sh"><os family="unix"/></condition>
<target name="clean">
<delete dir="${builddir}"/>
</target>
<target name="prep">
<mkdir dir="${builddir}"/>
</target>
<target name="jstest">
<!--Run all tests w/phantom, fail if tests fail. Execute all files w/extension .test.htm. -->
<apply executable="${phantom.filename}" failonerror="true" dir="${basedir}" relative="true">
<arg value="run-qunit.js"/>
<fileset dir="${basedir}">
<include name="**/*.test.htm" />
</fileset>
</apply>
</target>
<target name="main" depends="clean, prep, jstest">
</target>
</project>
</pre>
</li>
</ul>
<li>Run 'ant'; you should get output similar to the following (yes, it's supposed to fail, remember we have a test that fails setup on purpose)</li>
<ul>
<li>
<pre>Buildfile: build.xml
clean:
[delete] Deleting directory C:\Code\jsunit-trial\target
prep:
[mkdir] Created dir: C:\Code\jsunit-trial\target
jstest:
[apply] Test "add is defined" completed: pass
[apply] Test "add 1+1" assertion failed. Expected <2> Actual <undefined>
[apply] Test "add 1+1" completed: FAIL
[apply] 'waitFor()' finished in 218ms.
[apply] Tests completed in 58 milliseconds.
[apply] 1 tests of 2 passed, 1 failed.
BUILD FAILED
C:\Code\jsunit-trial\build.xml:18: apply returned: 1
Total time: 0 seconds
</undefined></pre>
</li>
</ul>
<li>Edit testme.js just enough to fix the test</li>
<ul>
<li>
<pre class="Javascript" name="code">/**
* var-args; adds up all arguments and returns sum
*/
function add() {
var sum =0;
for (var i=0; i<arguments.length; i++)
sum += arguments[i];
return sum;
}</pre>
</li>
</ul>
<li>Run 'ant'; you should get output similar to the following</li>
<ul>
<li><pre>Buildfile: build.xml
clean:
[delete] Deleting directory C:\Code\jsunit-trial\target
prep:
[mkdir] Created dir: C:\Code\jsunit-trial\target
jstest:
[apply] Test "add is defined" completed: pass
[apply] Test "add 1+1" completed: pass
[apply] 'waitFor()' finished in 214ms.
[apply] Tests completed in 59 milliseconds.
[apply] 2 tests of 2 passed, 0 failed.
main:
BUILD SUCCESSFUL
Total time: 0 seconds</pre>
</li>
</ul>
</ol>
Pretty sweet, we've got Javascript tests running in an Ant build as a first-class citizen. Now if you break my Javascript my Continuous Integration server will let me know!<br />
<br />
<b>Part 4: Code coverage</b><br />
Finally we are ready to get some code coverage. We are going to get code coverage by instrumenting our js files using JSCoverage, running our QUnit tests such that the relative paths resolve to the instrumented copies, and then using the PhantomJS file system APIs to create a colorized copy of the original js file to visually display coverage. We'll do a quick and dirty percentage coverage output to the console as well.<br />
<br />
<br />
<ol>
<li>Download <a href="http://siliconforks.com/jscoverage/download.html">JSCoverage 0.5.1</a></li>
<li>Create a jscoverage.bat file in \jsunit with the following content<br /><pre style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">@echo off
C:\where\you\put\jscoverage\jscoverage.exe %*
</pre>
</li>
<li>Create a template file for coverage information named coverageBase.htm in \jsunit</li>
<ul>
<li><pre class="Html" name="code"><!DOCTYPE html>
<html>
<head>
<style>
.code {
white-space: pre;
font-family: courier new;
width: 100%;
}
.miss {
background-color: #FF0000;
}
.hit {
background-color: #94FF7C;
}
.undef {
background-color: #AFFF9E;
}
</style>
</head>
<body>
COLORIZED_LINE_HTML
</body>
</html>
</pre>
</li>
</ul>
<li>Update build.xml to perform a few new steps</li>
<ol>
<li>Create a \target\testjs\js directory and copy our js files into it</li>
<li>Index our js files for code coverage, putting the indexed version into \target\testjs\jsinstrumented</li>
<li>Copy *.test.htm into \target\testhtm</li>
<li>Copy base resources to run tests (run-qunit.js, qunit.js, qunit.css) into \target\testhtm</li>
<li>Copy the instrumented js files into \target\testhtm</li>
<ol>
<li>Note that because we used relative paths to our test js files the *.test.htm QUnit html files will now resolve js to the instrumented version when we run the files out of \target\testhtm</li>
</ol>
<li>Run PhantomJS on *.test.htm in \target\testhtm</li>
<li>The updated build.xml looks like this:</li>
<pre class="Javascript" name="code"><?xml version="1.0" encoding="UTF-8"?>
<project name="jsunittests" basedir="." default="main">
<property name="builddir" location="${basedir}/target"/>
<property name="jstestdir" location="${builddir}/testjs"/>
<property name="jsdir" location="${jstestdir}/js"/>
<property name="jsinstrumenteddir" location="${jstestdir}/jsinstrumented"/>
<property name="testhtmdir" location="${builddir}/testhtm"/>
<condition property="phantom.filename" value="phantomjs.bat"><os family="windows"/></condition>
<condition property="phantom.filename" value="phantomjs.sh"><os family="unix"/></condition>
<property name="jscoverage.filename" value="jscoverage.bat" />
<target name="clean">
<delete dir="${builddir}"/>
</target>
<target name="prep">
<mkdir dir="${jsdir}"/>
<mkdir dir="${jsinstrumenteddir}"/>
<mkdir dir="${testhtmdir}"/>
<!-- copy non-test js files to target so we can mess with 'em. how we select which files may vary; for this
example just pick the one file we are testing.-->
<copy todir="${jsdir}">
<fileset dir="${basedir}">
<include name="testme.js" />
</fileset>
</copy>
<!-- run jscoverage to produce a version of the file instrumented for code coverage -->
<exec executable="${jscoverage.filename}" failonerror="true">
<arg value="${jsdir}"/>
<arg value="${jsinstrumenteddir}"/>
</exec>
<!-- copy our test htm files and modify them to point to the coverage indexed version of the test file. -->
<copy todir="${testhtmdir}">
<fileset dir="${basedir}">
<include name="**/*.test.htm" />
</fileset>
</copy>
<!-- copy core resources to testhtmdir so we can load them with same paths as when executing test htm files directly -->
<copy todir="${testhtmdir}">
<fileset dir="${jsinstrumenteddir}">
<include name="**/*.js" />
<exclude name="jscoverage.js"/>
</fileset>
</copy>
<copy todir="${testhtmdir}">
<fileset dir="${basedir}">
<include name="test-support.js" />
<include name="run-qunit.js" />
<include name="qunit.css" />
<include name="qunit.js" />
</fileset>
</copy>
</target>
<target name="jstest">
<!--Run all tests w/phantom, fail if tests fail. Execute all files w/extension .test.htm. -->
<apply executable="${basedir}/${phantom.filename}" failonerror="true" dir="${testhtmdir}" relative="false">
<arg value="run-qunit.js"/>
<srcfile/>
<arg value="${basedir}"/>
<fileset dir="${testhtmdir}">
<include name="**/*.test.htm" />
</fileset>
</apply>
</target>
<target name="main" depends="clean, prep, jstest">
</target>
</project>
</pre>
</ol>
<li>Modify our test-support.js to look for jscoverage data and output a rough count of lines hit, missed, and irrelevant (non-executable). Also expose a function a caller outside of page context can use to access coverage information. The new version should look like this:</li>
<ul>
<li>
<pre class="Javascript" name="code">//create a scope so we don't pollute global
(function() {
var testName;
//arg: { name }
QUnit.testStart = function(t) {
testName = t.name;
};
//arg: { name, failed, passed, total }
QUnit.testDone = function(t) {
console.log('Test "' + t.name + '" completed: ' + (0 === t.failed ? 'pass' : 'FAIL'))
};
//{ result, actual, expected, message }
QUnit.log = function(t) {
if (!t.result) {
console.log('Test "' + testName + '" assertion failed. Expected <' + t.expected + '> Actual <' + t.actual + '>' + (t.message ? ': \'' + t.message + '\'' : ''));
}
};
//we want this at global scope so outside callers can find it. In a more realistic implementation we
//should probably put it in a namespace.
window.getCoverageByLine = function() {
var key = null;
var lines = null;
//look for code coverage data
if (typeof _$jscoverage === 'object') {
for (key in _$jscoverage) {}
lines = _$jscoverage[key];
}
if (!lines) {
console.log('code coverage data is NOT available');
}
return { 'key': key, 'lines': lines };
};
QUnit.done = function(t) {
var cvgInfo = getCoverageByLine();
if (!!cvgInfo.lines) {
var testableLines = 0;
var testedLines = 0;
var untestableLines = 0;
for (lineIdx in cvgInfo.lines) {
var cvg = cvgInfo.lines[lineIdx];
if (typeof cvg === 'number') {
testableLines += 1;
if (cvg > 0) {
testedLines += 1;
}
} else {
untestableLines += 1;
}
}
var coverage = '' + Math.floor(100 * testedLines / testableLines) + '%';
var result = document.getElementById('qunit-testresult');
if (result != null) {
result.innerHTML = result.innerHTML + ' ' + coverage + ' test coverage of ' + cvgInfo.key;
} else {
console.log('can\'t find test-result element to update');
}
}
};
}());
</pre>
</li>
</ul>
<li>Finally, modify run-qunit.js to load the original js file and produce a colorized version based on the coverage data we get by running the test against the version of the js file indexed for coverage. The new version should look like this:</li>
<ul>
<li>
<pre class="Javascript" name="code">/**
* Wait until the test condition is true or a timeout occurs. Useful for waiting
* on a server response or for a ui change (fadeIn, etc.) to occur.
*
* @param testFx javascript condition that evaluates to a boolean,
* it can be passed in as a string (e.g.: "1 == 1" or "$('#bar').is(':visible')" or
* as a callback function.
* @param onReady what to do when testFx condition is fulfilled,
* it can be passed in as a string (e.g.: "1 == 1" or "$('#bar').is(':visible')" or
* as a callback function.
* @param timeOutMillis the max amount of time to wait. If not specified, 3 sec is used.
*/
function waitFor(testFx, onReady, timeOutMillis) {
var maxtimeOutMillis = timeOutMillis ? timeOutMillis : 3001, //< Default Max Timout is 3s
start = new Date().getTime(),
condition = false,
interval = setInterval(function() {
if ( (new Date().getTime() - start < maxtimeOutMillis) && !condition ) {
// If not time-out yet and condition not yet fulfilled
condition = (typeof(testFx) === "string" ? eval(testFx) : testFx()); //< defensive code
} else {
if(!condition) {
// If condition still not fulfilled (timeout but condition is 'false')
console.log("'waitFor()' timeout");
phantom.exit(1);
} else {
// Condition fulfilled (timeout and/or condition is 'true')
console.log("'waitFor()' finished in " + (new Date().getTime() - start) + "ms.");
typeof(onReady) === "string" ? eval(onReady) : onReady(); //< Do what it's supposed to do once the condition is fulfilled
clearInterval(interval); //< Stop this interval
}
}
}, 100); //< repeat check every 250ms
};
if (phantom.args.length === 0 || phantom.args.length > 3) {
console.log('Usage: run-qunit.js URL basedir');
phantom.exit(1);
}
var fs = require('fs');
var page = require('webpage').create();
// Route "console.log()" calls from within the Page context to the main Phantom context (i.e. current "this")
page.onConsoleMessage = function(msg) {
console.log(msg);
};
var openPath = phantom.args[0].replace(/^.*(\\|\/)/, '');
var basedir = phantom.args[1];
var coverageBase = fs.read(basedir + fs.separator + 'coverageBase.htm');
page.open(openPath, function(status){
if (status !== "success") {
console.log("Unable to access network");
phantom.exit(1);
} else {
waitFor(function(){
return page.evaluate(function(){
var el = document.getElementById('qunit-testresult');
if (el && el.innerText.match('completed')) {
return true;
}
return false;
});
}, function(){
//BEGIN MODIFIED: output colorized code coverage
//reach into page context and pull out coverage info. stringify to pass context boundaries.
var coverageInfo = JSON.parse(page.evaluate(function() { return JSON.stringify(getCoverageByLine()); }));
var lineCoverage = coverageInfo.lines;
var originalFile = basedir + fs.separator + coverageInfo.key;
var fileLines = readFileLines(originalFile);
var colorized = '';
console.log('lines=' + JSON.stringify(lineCoverage));
for (var idx=0; idx < lineCoverage.length; idx++) {
//+1: coverage lines count from 1.
var cvg = lineCoverage[idx + 1];
var hitmiss = '';
if (typeof cvg === 'number') {
hitmiss = ' ' + (cvg>0 ? 'hit' : 'miss');
} else {
hitmiss = ' ' + 'undef';
}
var htmlLine = fileLines[idx].replace('<', '&lt;').replace('>', '&gt;');
colorized += '<div class="code' + hitmiss + '">' + htmlLine + '</div>\n';
};
colorized = coverageBase.replace('COLORIZED_LINE_HTML', colorized);
var coverageOutputFile = phantom.args[0].replace('.test.htm', '.coverage.htm');
fs.write(coverageOutputFile, colorized, 'w');
console.log('Coverage for ' + coverageInfo.key + ' in ' + coverageOutputFile);
//END MODIFIED
var failedNum = page.evaluate(function(){
var el = document.getElementById('qunit-testresult');
console.log(el.innerText);
try {
return el.getElementsByClassName('failed')[0].innerHTML;
} catch (e) { }
return 10000;
});
phantom.exit((parseInt(failedNum, 10) > 0) ? 1 : 0);
});
}
});
//MODIFIED: add new fn
function readFileLines(filename) {
var stream = fs.open(filename, 'r');
var lines = [];
var line;
while (!stream.atEnd()) {
lines.push(stream.readLine());
}
stream.close();
return lines;
}
</pre>
</li>
</ul>
<li>Run 'ant'; you should see output similar to:</li>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghmHvmlA-sodzZeIhgBZ1AnRQ_BJ7HnpvUEsFb6v9QYNRsfUquNOLDMC-xUTdO5AqogVkRQ-qTvbP5eDoJ6e2w3Yu9uCZmQxkBI-nzW2fH_a_vGZhdFig4BK5kkhd50l-Bs7WMn-Sst8gJ/s1600/coveragetestout.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="56" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghmHvmlA-sodzZeIhgBZ1AnRQ_BJ7HnpvUEsFb6v9QYNRsfUquNOLDMC-xUTdO5AqogVkRQ-qTvbP5eDoJ6e2w3Yu9uCZmQxkBI-nzW2fH_a_vGZhdFig4BK5kkhd50l-Bs7WMn-Sst8gJ/s320/coveragetestout.png" width="320" /></a></div>
<ul>
<li><br /></li>
</ul>
<li>Open \jsunit\target\testhtm\testme.test.htm in a browser; you should see something similar to this (note coverage % appears):</li>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2uOe8O3AM0CspJC3Lf7qHglGVxq1WDEPEO-yo_HL1K1k9nA22N7LT4iAp4I58sEftaZduO0rVJQNLuK8Y_nITMWAFHH8DKK0KaYX9g_kP2B3BiiULdI6NBtpTGqXT5lQXFndSGfeL8VJq/s1600/cvg1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="110" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2uOe8O3AM0CspJC3Lf7qHglGVxq1WDEPEO-yo_HL1K1k9nA22N7LT4iAp4I58sEftaZduO0rVJQNLuK8Y_nITMWAFHH8DKK0KaYX9g_kP2B3BiiULdI6NBtpTGqXT5lQXFndSGfeL8VJq/s320/cvg1.png" width="320" /></a></div>
<ul>
<li><br /></li>
</ul>
<li>Open \jsunit\target\testhtm\testme.coverage.htm in a browser; you should see something similar to this (red for untested, green for tested, light green for non-executable lines):</li>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrWMDuL3L7iH2Dn55Om-blFdG187d34Jpgp_cBHsC8UM8MhkAp_iud2dDz2BApXeg-pCpbMAzeN_scruEzqmEj9hIn2zEwU0_F8hSAoe8AeTZeN0q9FbDA4RBiMM9eNvjKzIxGRautuIAc/s1600/cvg2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="133" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrWMDuL3L7iH2Dn55Om-blFdG187d34Jpgp_cBHsC8UM8MhkAp_iud2dDz2BApXeg-pCpbMAzeN_scruEzqmEj9hIn2zEwU0_F8hSAoe8AeTZeN0q9FbDA4RBiMM9eNvjKzIxGRautuIAc/s320/cvg2.png" width="320" /></a></div>
<ul>
<li><br /></li>
</ul>
</ol>
<b>So where does that leave us?</b><br />
We have clearly displayed we can accomplish some important things:<br />
<br />
<br />
<ul>
<li>Write unit tests for Javascript</li>
<li>Run unit tests for Javascript in a command line build</li>
<li>Index Javascript files for code coverage</li>
<li>Output coverage percentage to the test runner (QUnit html file)</li>
<li>Render a colorized version of the Javascript under test clearly indicating which lines are/aren't being tested</li>
</ul>
<div>
I think this is awesome! Bear in mind in a real version we would of course make numerous refinements to this rather basic implementation; what we have is a proof of concept not by any stretch of the imagination an implementation ready for a team to consume.</div>
Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com87tag:blogger.com,1999:blog-4949964667779944881.post-40047204136454362032011-09-23T14:10:00.000-07:002011-09-30T11:01:56.844-07:00Tracking a running standard deviationIt is fairly common to want to track statistics on your software as it runs. Sometimes it makes sense to scrape logs and aggregate a massive set of samples. Other times you really want "live" numbers; eg running statistics. Most commonly, arithmetic mean and standard deviation. Tracking arithmetic mean alone is relatively easy but it can be very misleading. I frequently see claims of "good performance" based on mean alone which completely break down when it turns out that if we consider standard deviation a significant percentage of our population is actually experiencing something much, much worse than the mean.<br />
<br />
So, how do we track a standard deviation from a series of samples? We could keep all samples in memory but that sounds bad; ideally we'd just like a few variables used to keep an running count of the stat. Luckily Knuth solved this problem for us, and even more luckily a statistician has written sample code in C# for us at <a href="http://www.johndcook.com/standard_deviation.html">http://www.johndcook.com/standard_deviation.html</a>.<br />
<br />
In Java this winds up looking remarkably similar:<br />
<pre class="Java" name="code">//class
public class RunningStat {
private int m_n;
private double m_oldM;
private double m_newM;
private double m_oldS;
private double m_newS;
public RunningStat() {
m_n = 0;
}
public void clear() { m_n = 0; }
public void addSample(double sample) {
m_n++;
// See Knuth TAOCP vol 2, 3rd edition, page 232
if (m_n == 1)
{
m_oldM = m_newM = sample;
m_oldS = 0.0;
}
else
{
m_newM = m_oldM + (sample - m_oldM)/m_n;
m_newS = m_oldS + (sample - m_oldM)*(sample - m_newM);
// set up for next iteration
m_oldM = m_newM;
m_oldS = m_newS;
}
}
public int getNumSamples() { return m_n; }
public double getMean() { return (m_n > 0) ? m_newM : 0.0; }
public double getVariance() { return ( (m_n > 1) ? m_newS/(m_n - 1) : 0.0 ); }
public double getStdDev() { return Math.sqrt(getVariance()); }
}
//usage
RunningStat timeStat = new RunningStat();
...
long time = System.nanoTime();
//do the thing we are tracking...
time = (System.nanoTime() - time) / (1000*1000); //ms
timeStat.addSample(time);
</pre>
<br />
This does have some limitations for a typical Java web application, in particular it isn't threadsafe. We can simply slap synchronized on it, using a private lock to avoid unwanted publicity of our synchronization primatives:<br />
<pre class="Java" name="code">public class RunningStat {
private int m_n;
private double m_oldM;
private double m_newM;
private double m_oldS;
private double m_newS;
private Object m_lock = new Object();
public RunningStat() {
m_n = 0;
}
public void clear() {
synchronized(m_lock) {
m_n = 0;
}
}
public void addSample(double sample) {
synchronized(m_lock) {
m_n++;
// See Knuth TAOCP vol 2, 3rd edition, page 232
if (m_n == 1)
{
m_oldM = m_newM = sample;
m_oldS = 0.0;
}
else
{
m_newM = m_oldM + (sample - m_oldM)/m_n;
m_newS = m_oldS + (sample - m_oldM)*(sample - m_newM);
// set up for next iteration
m_oldM = m_newM;
m_oldS = m_newS;
}
}
}
public int getNumSamples() { synchronized(m_lock) { return m_n; } }
public double getMean() { synchronized(m_lock) { return (m_n > 0) ? m_newM : 0.0; } }
public double getVariance() { synchronized(m_lock) { return ( (m_n > 1) ? m_newS/(m_n - 1) : 0.0 ); } }
public double getStdDev() { synchronized(m_lock) { return Math.sqrt(getVariance()); } }
}
</pre>
This leaves open the question of how we actually distribute references to the stat. For example, it could be that anytime a given handler executes we want to update a stat, and we want to grab that stat to use in a periodic stat logger, and we want to grab it for our view statistics page. All these locations need to get a reference to the same instance of RunningStat. If we use something like Spring we can trivially inject the same reference. If not we may wish to manage a stats registry ourselves. A simple implementation using ConcurrentHashMap might look something like this:
<br />
<pre class="Java" name="code">import java.util.Collection;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
public class StatisticsRegistry {
private static final StatisticsRegistry instance = new StatisticsRegistry();
private final ConcurrentMap<string, runningstat=""> statById = new ConcurrentHashMap<string, runningstat="">();
private final Object writeLock = new Object();
public static StatisticsRegistry getInstance() { return instance; }
public RunningStat getNamedStatistic(String name) {
/**
* Usually the stat will exist; avoid extra sync ops on write lock when possible
*/
RunningStat stat = statById.get(name);
if (null == stat) {
synchronized(writeLock) {
//someone else could have just inserted it; if so putIfAbsent will return the value they put in
RunningStat existing = statById.putIfAbsent(name, new RunningStat());
if (existing != null) {
stat = existing;
}
}
}
return stat;
}
public Collection<runningstat> getAllStatistics() {
return statById.values();
}
}
</runningstat></string,></string,></pre>
Clients can request a stat by name and clients like a stat logger or viewer can grab either all stats or specific ones as needed. As the same instance is always returned it is safe for clients to write code to retain RunningStat references. Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com5tag:blogger.com,1999:blog-4949964667779944881.post-57779838724114265502011-07-11T10:40:00.000-07:002011-07-11T10:40:06.974-07:00Javascript Unit Tests with QUnit, Ant, and PhantomJS, Take 1Recently I have been finding bugs in Javascript slip by a lot more easily than bugs in Scala, Java, or various other languages for which we write unit tests. jQuery seems to use QUnit (http://docs.jquery.com/QUnit) but QUnit appears to expect a web page to be setup to host it. This is better than nothing but really I want to run my js unit tests in an automated build, in my case using Ant.<br />
<br />
The problem seemed remarkably likely to be solved already so I took to the Google and discovered some blogposts (http://twoguysarguing.wordpress.com/2010/11/26/qunit-cli-running-qunit-with-rhino/, http://twoguysarguing.wordpress.com/2010/11/06/qunit-and-the-command-line-one-step-closer/) where the author was attempting to achieve a command line unit test runner using Rhino and QUnit. Apparently John Resig tweeted some time ago (http://twitter.com/#!/jeresig/status/4477641447) to indicate QUnit should be operable in this manner so things seemed promising.<br />
<br />
The twoguysarguing (great title) blog posts I found seemed to require modifications to QUnit source, plus Rhino not being a full browser apparently caused some issues as well. I really didn't want a custom version of QUnit, but the general approach seemed promising. In the comments for the second post someone suggested use of PhantomJS (http://twoguysarguing.wordpress.com/2010/11/06/qunit-and-the-command-line-one-step-closer/#comment-599), a headless WebKit browser. I decided to give this a try as it sounded remarkably reasonable.<br />
<br />
My first step was to verify PhantomJS worked for me at all. It ran my first test without any issue:<br />
<pre class="Javascript" name="code">//try1.js
console.log('Hello, World');
phantom.exit();
</pre>Executed similar to <span class="Apple-style-span" style="color: blue;">phantomjs try1.js</span> this prints <span class="Apple-style-span" style="color: blue;">Hello, World</span> just as one might hope.<br />
<br />
The next question seemed to be whether or not PhantomJS could actually load QUnit using injectJs (ref <a href="http://code.google.com/p/phantomjs/wiki/Interface">http://code.google.com/p/phantomjs/wiki/Interface</a>). I git cloned QUnit and attempted to invoke inject.Js on it:<br />
<pre class="Javascript" name="code">//try2.js
if (window.QUnit == undefined)
console.log('no QUnit yet!');
else
console.log('somehow we already haz QUnit !!');
phantom.injectJs('D:\\Code\\3.7-scalaide\\JavaScriptUnitTests\\QUnit\\qunit.js');
if (window.QUnit != undefined)
console.log('goodnes; injectJs seems to have worked');
phantom.exit();
</pre>This prints:<br />
<pre>no QUnit yet!
goodnes; injectJs seems to have worked
</pre>So far so good!!<br />
<br />
So, that means we should be able to setup and run a test, right? Something like this:<br />
<pre class="Javascript" name="code">test("This test should fail", function() {
console.log('the test is running!');
ok( true, "this test is fine" );
var value = "hello";
equals( "hello", value, "We expect value to be hello" );
equals( "duck", value, "We expect value to be duck" );
});
test("This test should pass", function() {
console.log('the test is running!');
ok( true, "this test is fine" );
var value = "hello";
equals( "hello", value, "We expect value to be hello" );
equals( "duck", value, "We expect value to be duck" );
});
</pre>Well ... sadly this part didn't "just work". QUnit tries to execute the test queue on timers and despite PhantomJS supporting timers they just never seemed to execute. Furthermore, QUnit default feedback is via DOM modifications that are rather unhelpful to the PhantomJS runner. My first draft was to modify QUnit by adding a function that directly executed the test queue, inline, without using timers. This worked, but it required modifying QUnit source, which I specifically wish to avoid.<br />
<br />
Luckily something similar to the changes to add a function to run QUnits tests directly works just fine outside QUnit as well. The key is that we will:<br />
<ol><li>Track test pass/fail count via the QUnit.testDone callback (<a href="http://docs.jquery.com/Qunit#Integration_into_Browser_Automation_Tools">http://docs.jquery.com/Qunit#Integration_into_Browser_Automation_Tools</a>)</li>
<ol><li>We need our own pass/fail counters as QUnit tells us how many<i> assertions</i> passed/failed rather than how many <i>tests </i>passed/failed.</li>
</ol><li>Track whether or not the test run is done overall via the QUnit.done callback (<a href="http://docs.jquery.com/Qunit#Integration_into_Browser_Automation_Tools">http://docs.jquery.com/Qunit#Integration_into_Browser_Automation_Tools</a>)</li>
<li>Directly execute the QUnit test queue from our own code</li>
<ol><li>hack but the point here is to see if we can make this work at all</li>
</ol><li>Split tests into their own file</li>
<ol><li>This facilitates using an Ant task to run a bunch of different test sets; eg using apply to pickup on all .js test files by naming convention or location convention</li>
</ol><li>Return the count of failures as our PhantomJS exit code</li>
<ol><li>This facilitates setting an Ant task to failonerror to detect unit test failures</li>
</ol></ol><div>So, without further ado, error handling, namespaces/packages, or any other cleanup here is a version that works in a manner very near to the desired final result:<br />
<b>try4.js</b></div><pre class="Javascript" name="code">function importJs(scriptName) {
console.log('Importing ' + scriptName);
phantom.injectJs(scriptName);
}
console.log('starting...');
//Arg1 should be QUnit
importJs(phantom.args[0]);
//Arg2 should be user tests
var usrTestScript = phantom.args[1];
importJs(usrTestScript);
//Run QUnit
var testsPassed = 0;
var testsFailed = 0;
//extend copied from QUnit.js
function extend(a, b) {
for ( var prop in b ) {
if ( b[prop] === undefined ) {
delete a[prop];
} else {
a[prop] = b[prop];
}
}
return a;
}
QUnit.begin({});
// Initialize the config, saving the execution queue
var oldconfig = extend({}, QUnit.config);
QUnit.init();
extend(QUnit.config, oldconfig);
QUnit.testDone = function(t) {
if (0 === t.failed)
testsPassed++;
else
testsFailed++;
console.log(t.name + ' completed: ' + (0 === t.failed ? 'pass' : 'FAIL'))
}
var running = true;
QUnit.done = function(i) {
console.log(testsPassed + ' of ' + (testsPassed + testsFailed) + ' tests successful');
console.log('TEST RUN COMPLETED (' + usrTestScript + '): ' + (0 === testsFailed ? 'SUCCESS' : 'FAIL'));
running = false;
}
//Instead of QUnit.start(); just directly exec; the timer stuff seems to invariably screw us up and we don't need it
QUnit.config.semaphore = 0;
while( QUnit.config.queue.length )
QUnit.config.queue.shift()();
//wait for completion
var ct = 0;
while ( running ) {
if (ct++ % 1000000 == 0) {
console.log('queue is at ' + QUnit.config.queue.length);
}
if (!QUnit.config.queue.length) {
QUnit.done();
}
}
//exit code is # of failed tests; this facilitates Ant failonerror. Alternately, 1 if testsFailed > 0.
phantom.exit(testsFailed);
</pre><div><br />
<b>try4-tests.js</b><br />
<pre class="javascript" name="code">test("This test should fail", function() {
ok( true, "this test is fine" );
var value = "hello";
equals( "hello", value, "We expect value to be hello" );
equals( "duck", value, "We expect value to be duck" );
});
test("This test should pass", function() {
equals( "hello", "hello", "We expect value to be hello" );
});
</pre><br />
This runs as follows:</div><pre>>phantomjs.exe try4.js qunit.js try4-tests.js
starting...
Importing qunit\qunit.js
Importing javascript\try4-tests.js
This test should fail completed: FAIL
This test should pass completed: pass
queue is at 0
1 of 2 tests successful
TEST RUN COMPLETED (try4-tests.js): FAIL
</pre><br />
Note that we have not modified qunit.js, and we have split our tests into their own file. This allows us to easily set the whole thing up to run from Ant:<br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project default="js-tests4">
<target name="js-tests4">
<property name="phantomjs.exe.file" value="phantomjs.exe" />
<property name="qunit.js.file" location="path/to/qunit.js" />
<apply executable="${phantomjs.exe.file}" failonerror="true">
<arg value="path/to/try4.js"/>
<arg value="${qunit.js.file}" />
<srcfile/>
<fileset dir="path/to/tests">
<include name="try4-tests.js" />
</fileset>
</apply>
</target>
</project>
</pre><br />
It even works run this way: <br />
<pre>js-tests4:
[apply] starting...
[apply] Importing path\to\qunit.js
[apply] Importing path\to\try4-tests.js
[apply] This test should fail completed: FAIL
[apply] This test should pass completed: pass
[apply] queue is at 0
[apply] 1 of 2 tests successful
[apply] TEST RUN COMPLETED (try4-tests.js): FAIL
BUILD FAILED
</pre><br />
Note that Ant has detected that a js unit test failed and failed the build, just as we intended.<br />
<br />
This leaves us with a proof of concept implementation that seems to prove that using PhantomJS to run QUnit based Javascript tests to run from a command line build is fundamentally possible. Doubtless if/when we try to use it "for real" additional problems will emerge ;)Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com318tag:blogger.com,1999:blog-4949964667779944881.post-56544293947094080622011-06-14T09:27:00.000-07:002011-06-14T14:30:53.572-07:00Beginning Scala: Building a project with Maven and EclipseLet us suppose that after many years languishing in the dark corridors of Java, frantically scrambling to avoid the various framework ghouls, one finally emerged into the light and saw a Scala frolicking in the sun on a much greener grassy field. The terse little Scala looks inviting to the verbose, ragged, framework beridden, somewhat aged Java developer ... but how to start? Well luckily there are a few nice introductions (<a href="http://www.codecommit.com/blog/scala/roundup-scala-for-java-refugees">Scala for Java Refugees</a>, <a href="http://www.ibm.com/developerworks/views/java/libraryview.jsp?sort_by=&show_abstract=true&show_all=&search_flag=&contentarea_by=Java+technology&search_by=The+busy+Java+developer%27s+guide+to+Scala&topic_by=-1&industry_by=-1&type_by=All+Types&ibm-search=Search">The busy Java developer's guide to Scala</a>) out there and the <a href="http://www.amazon.com/Programming-Scala-Comprehensive-Step-Step/dp/0981531644">staircase book</a> (by the language creator no less) is a must if one is serious about learning the language.<br />
<br />
Introductions are all well and good but as a Java developer I want some first-class tools and I want to run some code! Until recently first-class tools were <b>seriously</b> lacking. Luckily in recent months even the languages creator has acknowledged the tooling was lacking and has taken aggressive steps to correct the issue.<br />
<br />
At the time of writing I use Eclipse 3.6.2 Helios (Java Developer edition) with Scala-IDE 2.0.0.somethingOrOther-beta. For the sake of this example I will assume we wish to build the project using Maven. First step, download the tools:<br />
<ol><li>Eclipse Helios</li>
<ol><li>At the time of writing I favor the Java developer edition as it is somewhat less bloated than the EE version; http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/heliossr2</li>
</ol><li>Scala IDE 2</li>
<ol><li>The 2.0 release is a major modification and the first solid Eclipse IDE plugin for Scala; the 1.x version was prone to all sorts of "exciting" behaviors</li>
<li>Add to Eclipse using the update site from link on the front page of http://www.scala-ide.org/</li>
</ol><li>m2eclipse</li>
<ol><li>Add to Eclipse using the update site from http://m2eclipse.sonatype.org/installing-m2eclipse.html</li>
</ol><li>Maven 3</li>
<ol><li>Download from <a href="http://maven.apache.org/download.html">http://maven.apache.org/download.html</a></li>
<li>Make sure if you run <span class="Apple-style-span" style="color: blue;">mvn -v</span> in command prompt it prints the expected version</li>
</ol></ol><div>Now we have the tools, lets setup a project! First up, create a directory for your project and in that directory create a file called pom.xml similar to:</div><div><pre class="Xml" name="code"><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.blogspot.whileonefork</groupId>
<artifactId>mvn-scala-trial1</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>Sample Project for Blog</name>
<url>http://com.blogspot.whileonefork</url>
</project>
</pre></div><div><br />
Next, create a directory named src containing a directory named main containing a directory named scala. Your project should now contain the following:<br />
<pre>mvn-scala-project
|--pom.xml
`--src
`--main
`--scala
</pre></div><div><br />
</div><div>In the src/main/scala directory, create a new file HelloWorldJustLikeJava.scala with the simplest hello world implementation possible:<br />
<pre class="Scala" name="code">object HelloWorldJustLikeJava {
def main(args:Array[String]) = {
println("Hello, World");
}
}
</pre><br />
This, while arguably not the absolute simplest Scala hello world, is the definition closest to the one you would write in Java. Now to get it compiling in Maven. If you open a command prompt in the mvn-scala-project directory and run <span class="Apple-style-span" style="color: blue;">mvn clean install</span> it will succeed but you will get a warning that no content was marked for inclusion; this occurs because by default Maven doesn't include the plugin necessary to build Scala code.<br />
<br />
To compile Scala files we must update our pom.xml; we'll tell Maven what plugin to use to build Scala files and we'll advise it of the existence of a URL where it can find the plugin in question. Our updated pom.xml should look like this:<br />
<pre class="Xml" name="code"><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.blogspot.whileonefork</groupId>
<artifactId>mvn-scala-trial1</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>Sample Project for Blog</name>
<url>http://com.blogspot.whileonefork</url>
<!-- Notify Maven it can download things from here -->
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
</repositories>
<dependencies>
<!-- Scala version is very important. Luckily the plugin warns you if you don't specify:
[WARNING] you don't define org.scala-lang:scala-library as a dependency of the project -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.9.0-1</version>
</dependency>
</dependencies>
<build>
<!-- add the maven-scala-plugin to the toolchain -->
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.14.2</version>
</plugin>
</plugins>
</build>
</project>
</pre><br />
If you run mvn scala:compile in the mvn-scala-project directory the project should build successfully and in /target/classes you should find two .class files: HelloWorldJustLikeJava$.class and HelloWorldJustLikeJava.class. Scala (usually) compiles the .class files to run on the JVM, just like Java.<br />
<br />
All the usual Java tools can be used on these .class files; for example if we run <a href="http://download.oracle.com/javase/6/docs/technotes/tools/windows/javap.html">javap</a> on HelloWorldJustLikeJava$ we see the Scala source was compiled to Java; in this case fairly predictable Java:<br />
<pre class="Java" name="code">public final class HelloWorldJustLikeJava$ extends java.lang.Object implements scala.ScalaObject{
public static final HelloWorldJustLikeJava$ MODULE$;
public static {};
public void main(java.lang.String[]);
}
</pre>Digging a little deeper, using javap -verbose, we can see that our HelloWorldJustLikeJava class invokes HelloWorldJustLikeJava$:<br />
<pre>public static final void main(java.lang.String[]);
Code:
Stack=2, Locals=1, Args_size=1
0: getstatic #11; //Field HelloWorldJustLikeJava$.MODULE$:LHelloWorldJustLikeJava$;
3: aload_0
4: invokevirtual #13; //Method HelloWorldJustLikeJava$.main:([Ljava/lang/String;)V
7: return
</pre>HelloWorldJustLikeJava$ provides a simple implementation of main and some other odds and ends:<br />
<br />
<pre>public static final HelloWorldJustLikeJava$ MODULE$;
public static {};
Code:
Stack=1, Locals=0, Args_size=0
0: new #9; //class HelloWorldJustLikeJava$
3: invokespecial #12; //Method "<init>":()V
6: return
public void main(java.lang.String[]);
Code:
Stack=2, Locals=2, Args_size=2
0: getstatic #19; //Field scala/Predef$.MODULE$:Lscala/Predef$;
3: ldc #22; //String Hello, World
5: invokevirtual #26; //Method scala/Predef$.println:(Ljava/lang/Object;)V
8: return
LineNumberTable:
line 3: 0
LocalVariableTable:
Start Length Slot Name Signature
0 9 0 this LHelloWorldJustLikeJava$;
0 9 1 args [Ljava/lang/String;
</init></pre><br />
The practice of outputting additional classes beyond what you directly coded is VERY common in Scala. A class using closures and other features of Scala will often output an exciting wack of $whatever classes. </div><br />
Anyway, at this point the objective of making our project compile with Maven is achieved; we can now move on to getting it setup to edit in Eclipse. In the mvn-scala-project directory (the same one as pom.xml), create a .classpath file and a .project file:<br />
<br />
.classpath<br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry kind="src" path="src/main/scala"/>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
<classpathentry kind="con" path="org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER"/>
<classpathentry kind="con" path="org.scala-ide.sdt.launching.SCALA_CONTAINER"/>
<classpathentry kind="output" path="target/classes"/>
</classpath>
</pre><br />
.project<br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>mvn-scala-project</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.scala-ide.sdt.core.scalabuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>org.maven.ide.eclipse.maven2Builder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.maven.ide.eclipse.maven2Nature</nature>
<nature>org.scala-ide.sdt.core.scalanature</nature>
<nature>org.eclipse.jdt.core.javanature</nature>
</natures>
</projectDescription>
</pre><br />
At this point your project should contain the following:<br />
<pre>mvn-scala-project
|--.classpath
|--.project
|--pom.xml
`--src
`--main
`--scala
|--HelloWorldJustLikeJava.scala
`target
`--classes
|--HelloWorldJustLikeJava$.class
|--HelloWorldJustLikeJava.class
</pre>If you happened to run <span class="Apple-style-span" style="color: blue;">mvn clean</span> the target directory may be absent.<br />
<br />
In Package Explorer in Eclipse your project should look similar to:<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVi1y38j_aevFBmgqCNFCDl3dXHir4VVm2DZ1up6vZ9hqCYT5HlmduS4regTw3bEH6d9MontYOEPrqzrADijil68dJjxdzvMr4aM-iZboXEv_E8vtFqVKVa-1a85WHpLpGcaT-dFJJ6nqw/s1600/ScalaInIDE.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVi1y38j_aevFBmgqCNFCDl3dXHir4VVm2DZ1up6vZ9hqCYT5HlmduS4regTw3bEH6d9MontYOEPrqzrADijil68dJjxdzvMr4aM-iZboXEv_E8vtFqVKVa-1a85WHpLpGcaT-dFJJ6nqw/s1600/ScalaInIDE.png" /></a></div><br />
<br />
And now we can edit our Scala in Eclipse and compile with Maven, and can therefore easily load our project into just about any continuous integration server. Hooray!<br />
<br />
EDIT: Assuming you are using m2eclipse, you will want to make sure your Eclipse IDE boots using a JDK VM; this is accomplished by editing eclipse.ini and adding a -vm argument. Bumping memory up a bit usually helps too; the file I am currently using on a Windows dev box follows:<br />
<pre>-startup
plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.2.R36x_v20101222
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-vm
C:/Program Files/Java/jdk1.6.0_07/bin/javaw.exe
--launcher.defaultAction
openFile
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx768m
</pre>EDIT2: If you want the project to build a bit more gracefully in Eclipse consider binding Scala compile to run by default (eg when you run mvn clean install instead of the somewhat novel mvn scala:compile). Modify the plugin element to add the execution:<br />
<pre class="Xml" name="code"><build>
<!-- add the maven-scala-plugin to the toolchain -->
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<goals><goal>compile</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</pre>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com9tag:blogger.com,1999:blog-4949964667779944881.post-87937448533021928162011-06-03T12:49:00.000-07:002011-06-03T12:49:17.393-07:00When the UI designer doesn't use the UISuppose you wanted to provide a basic interface where users could set training goals, stuff like "Run 50km in June" perhaps. This seems semi-reasonable right? - well, at least as long as nobody ever wants to set a goal for two weeks.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEemUSUm8lryEdPFePR2WNz9n-z1O_e72SVdnOe9cdxb7R6nYjbrvK3xpiZl8H8kXvC3udefG89KKKso-XwpRo4-4EBhxuaIn_MwD4rHDttrt0js6rcyuMqqbh4H6XkliU4Y_dCdi8IpWi/s1600/AnnoyingUi1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEemUSUm8lryEdPFePR2WNz9n-z1O_e72SVdnOe9cdxb7R6nYjbrvK3xpiZl8H8kXvC3udefG89KKKso-XwpRo4-4EBhxuaIn_MwD4rHDttrt0js6rcyuMqqbh4H6XkliU4Y_dCdi8IpWi/s400/AnnoyingUi1.jpg" width="400" /></a></div><br />
So surely if I want a goal for one month I can just set Calendar month, pick a day in the month (or maybe it'll even flip to be a month dropdown?), and go, right?<br />
<br />
Not so fast bucko, UI has got to be validated and you jackasses aren't going to start putting goals for the past into our system!<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2sbBSVGCDEC1s-6NOEkJ1wGq63GV0rNagHwGmjPNCEh864ci8ilVxj71eGgkukaickm5_aBFWqM4Kc0Q6dC6WFZFNH59or8jZJeFEUfgek3PyhR1SINq6YYMSJmbaAPkUgp_1VlxEz5J0/s1600/AnnoyingUi2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2sbBSVGCDEC1s-6NOEkJ1wGq63GV0rNagHwGmjPNCEh864ci8ilVxj71eGgkukaickm5_aBFWqM4Kc0Q6dC6WFZFNH59or8jZJeFEUfgek3PyhR1SINq6YYMSJmbaAPkUgp_1VlxEz5J0/s400/AnnoyingUi2.jpg" width="338" /></a></div>Not to worry though, the "calendar month" relative to June 3rd is a slightly weird way to put it but it does make sense. Or does it?<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1DTJhdBa1c8Qzcmrr7waRKvQKzUTGG7IG7nAzSlJf5dFJi7n7bmdcdjgpltRtwObscdYTA4Gc2VUAwOd1HQXsMh833oOA6UB8I7L-oR7GCkPb8XhNbWk738Nf91e0cfhqn6YNzOgZp-_0/s1600/AnnoyingUi3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="91" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1DTJhdBa1c8Qzcmrr7waRKvQKzUTGG7IG7nAzSlJf5dFJi7n7bmdcdjgpltRtwObscdYTA4Gc2VUAwOd1HQXsMh833oOA6UB8I7L-oR7GCkPb8XhNbWk738Nf91e0cfhqn6YNzOgZp-_0/s400/AnnoyingUi3.jpg" width="400" /></a></div>So um ... I can't enter the first because it's in the past, and I must pick the first day of a calendar month ... so it's impossible to specify a goal for June? And you don't even give me an arbitrary start/end date option so I could at least set June 3rd-30th?<br />
<br />
Clearly the designer of this never had to actually use it! Arglefarg.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com3tag:blogger.com,1999:blog-4949964667779944881.post-60047434914256267792011-05-31T11:00:00.000-07:002011-05-31T11:00:43.531-07:00The magic of getOrElseFor the last month or so I've been coding in Scala. From time to time I use Option's; they seem pretty handy; it's much nicer to get an Option[T] back instead of a null. I semi-often find myself writing things like:<br />
<br />
<pre class="Scala" name="code">//where f is a function returning Option[Something]
val t = f(args) getOrElse 3
</pre><br />
Recently I realized (classic lightbulb moment :) that getOrElse actually takes a function as the argument:<br />
<br />
<pre class="Scala" name="code">def getOrElse [B >: A] (default: ⇒ B): B
//Returns the option's value if the option is nonempty, otherwise return the result of evaluating default.
</pre><br />
Why does this matter? It means that the code passed to getOrElse doesn't run at all (default is not evaluated) if the Option is defined! For example, this code:<br />
<br />
<pre class="Scala" name="code">object GetOrElseTest {
def main(args : Array[String]) : Unit = {
def a() = Some(3)
def b() = None
val aV = a getOrElse { println("nothing from a :("); 1 }
val bV = b getOrElse { println("nothing from b :("); 1 }
println("aV=%d, bV=%d".format(aV, bV))
0
}
}
</pre><br />
Will print this output (note that the println for getting a, which is defined, never runs):<br />
<br />
<pre>nothing from b :(
aV=3, bV=1
</pre><br />
That's awesome and really hard to accomplish in Java. Imagine if log4j could do this; suddenly we wouldn't have any issues with things log.debug("a" + objectThatIsExpensiveToToString.toString()) having a runtime cost even when debug isn't enabled because we wouldn't evaluate the function to create a message at all if debug is off. <br />
<br />
The new uniforms are pretty snappy eh first officer!Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com8tag:blogger.com,1999:blog-4949964667779944881.post-26567354801030786312011-05-30T09:05:00.000-07:002011-05-30T09:06:47.679-07:00How to print Ivy's cachepath one artifiact per line with AntMany moons ago one <a href="http://blog.andrewbeacock.com/">Andrew Beacock</a> shared how to use <a href="http://ant.apache.org/manual/Tasks/pathconvert.html"><pathconvert></pathconvert></a> to dump out a classpath in a human-readable fashion (see http://blog.andrewbeacock.com/2005/08/pretty-printing-java-classpaths-using.html). The amount of Ant pain this gem saved me back in the day is ... well a lot.<br />
<br />
Fast forward to 2011 and we're using Ant with Ivy as our dependency manager. We use Ivy's <a href="http://ant.apache.org/ivy/history/2.1.0/use/retrieve.html">retrieve</a> Ant task to get our artifacts but how do we print out a simple list of what Ivy got for us? Well, it turns out cachepath can get this for us, optionally filtered if we see fit:<br />
<br />
<pre class="Code" name="Xml"><!-- after a resolve; in our case this target depends on our resolving target -->
<ivy:cachepath pathid="ivy.cachepath" settingsRef="ivy.settings" />
<pathconvert pathsep="${line.separator} "
property="dependency.list"
refid="ivy.cachepath"/>
<echo>DEPENDENCY LIST</echo>
<echo> ${dependency.list}</echo>
</pre><br />
This will print something along the lines of:<br />
<br />
<pre>[echo] C:\...\.ivy2\cache\org.slf4j\slf4j-api\jars\slf4j-api-1.6.1.jar
[echo] C:\...\.ivy2\cache\org.slf4j\jcl-over-slf4j\jars\jcl-over-slf4j-1.6.1.jar
[echo] C:\...\.ivy2\cache\org.slf4j\log4j-over-slf4j\jars\log4j-over-slf4j-1.6.1.jar
[echo] C:\...\.ivy2\cache\ch.qos.logback\logback-classic\jars\logback-classic-0.9.28.jar
[echo] C:\...\.ivy2\cache\ch.qos.logback\logback-core\jars\logback-core-0.9.28.jar
</pre><br />
Once in a while this is a lifesaver as it makes it easy to see a simple list of our dependencies. Since we print it out line-by-line it's now nice and easy to grep through and find specific things. Like say what version of the components in such and such a group we are getting.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com1tag:blogger.com,1999:blog-4949964667779944881.post-18860058933656483142011-04-21T09:52:00.000-07:002011-04-21T09:53:00.227-07:00Deceiving yourself in ScalaI found a cool way to trip myself in Scala today. This is what I meant:<br />
<pre class="Scala" name="code">private val lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object]
...
locks.get(key)
</pre>This is what I actually typed:<br />
<pre class="Scala" name="code">private def lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object]
...
locks.get(key)
</pre>Because I put 'def lockByKey' instead of 'val lockByKey' I created myself a function that creates a new ConcurrentHashMap every time it gets called instead of a single instance. Scala allows leaving out brackets in many cases so code like lockByKey.get(...) compiled fine, but as every access was creating a new map things didn't work entirely correctly.<br />
<br />
Thankfully unit tests of basic functionality (eg ability to get the same value back twice) caught that something was awry and I was able to identfy that I had cleverly created a factory function rather than a static instance. Thank god for unit tests ;)Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com2tag:blogger.com,1999:blog-4949964667779944881.post-84256258621436470042011-04-21T09:28:00.000-07:002011-04-21T09:59:36.244-07:00Dynamic lock allocationSuppose you had some code that manipulated things in the filesystem and you wanted to ensure only one thread worked on any given file at a time. Various parts of the code may construct a File object and they then desire to synchronize access to it. However, each part of the code may have a different <i>instance</i> of the File so while the instances are equal (.equals would return true in Java) they are not the same instance (== would <b>not </b>return true in Java. Visually:<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiszNTqZROhDTuQV0CIY-qrwgf5bhd_YgFXYJXDxBlsL95tjOA-YX8bnH71fMpQUwzOuK2QU9C6Ze9wgMqVaSThMgcQrIGGhWuwIirs_YTyKuBHhOik8rkZASHaRfx7KXsFTXEk01A-JIF/s1600/lockon.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiszNTqZROhDTuQV0CIY-qrwgf5bhd_YgFXYJXDxBlsL95tjOA-YX8bnH71fMpQUwzOuK2QU9C6Ze9wgMqVaSThMgcQrIGGhWuwIirs_YTyKuBHhOik8rkZASHaRfx7KXsFTXEk01A-JIF/s1600/lockon.png" /></a></div>We could just declare a lock and synchronize on that, but that means that regardless of how many different resources we have only one thread can use any of them. It would be handy if we could say (pseudo-code):<br />
<br />
<pre class="Scala" name="code">val resource = new SomeResourceOrOther("""blah""")
val lock = lockFor(resource )
lock synchronized {
//do stuff w/exclusive access to resource
}
</pre><br />
By acquiring a different lock for each resource we ensure optimal concurrency. This is somewhat similar to how structures like java.util.concurrent.ConcurrentHashMap partition their data into many segments and allocate a key per segement, thus potentially allowing as many threads as there are segments to safely operate on the data structure concurrently. We can build such a structure fairly easily:<br />
<br />
<pre class="Scala" name="code">object Locks {
private val lockByKey = new java.util.concurrent.ConcurrentHashMap[Any, Object]
@tailrec
def lockFor(key: Any): Object = {
val lock = lockByKey.get(key)
if (lock == null) {
lockByKey.putIfAbsent(key, new Object())
lockFor(key)
} else {
lock
}
}
...
class ConsumerOfLocks {
val resource = new String("Maybe its a string?") //contrived means of insuring different instance!
Locks.lockFor(resource) synchronized {
//wherein I have exclusive access to resource
}
}
</pre><br />
<br />
Obviously the resource could be a file/directory or just about anything else. This structure can be particularly useful if you have multiple dynamic local resources you want to synchronize access to. If you move to a lock that is distributed in some manner then the same structure works across multiple nodes. For example, I have used a logically similar structure to ensure that only one node runs a given batch process.<br />
<br />
This structure works best if the same set of resources are used consistently. If over time we are continually creating and locking new resources then we have to introduce additional complexity to ensure the lock map doesn't simply grow forever.<br />
<br />
Source samples are for Scala 2.8.1.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com0tag:blogger.com,1999:blog-4949964667779944881.post-66104886364376936382011-03-10T13:44:00.000-08:002011-03-10T14:07:04.897-08:00100% - 30px: Mixed-unit calculation in Firefox 4Firefox 4 (see <a href="http://www.mozilla.com/en-US/firefox/4.0rc1/releasenotes/">release notes</a>, or <a href="https://developer.mozilla.org/En/Firefox_4_for_developers">dev edition</a> thereof) adds some pretty cool new features. I love the <a href="http://www.azarask.in/blog/post/designing-tab-candy/">tab candy</a>.<br />
<br />
A fairly big one that almost slipped under my radar is that they shipped an initial implementation of CSS <a href="http://www.w3.org/TR/css3-values/#calc">calc</a>: "<span class="Apple-style-span" style="color: #333333; font-family: 'Lucida Grande', 'Lucida Sans Unicode', Lucida, Arial, Helvetica, sans-serif; font-size: 14px; line-height: 22px;">Support for</span><span class="Apple-style-span" style="color: #333333; font-family: 'Lucida Grande', 'Lucida Sans Unicode', Lucida, Arial, Helvetica, sans-serif; font-size: 14px; line-height: 22px;"><code style="font-style: inherit; font-weight: inherit; font: normal normal normal 100%/normal 'Courier New', 'Andale Mono', monospace;"><span style="font-family: Verdana, Tahoma, sans-serif;"> <code style="font-style: inherit; font-weight: inherit; font: normal normal normal 100%/normal 'Courier New', 'Andale Mono', monospace;"><a href="https://developer.mozilla.org/en/CSS/-moz-calc" rel="custom nofollow" style="color: #336699; text-decoration: none;">-moz-calc</a></code> has been added. This lets you specify <code style="font-style: inherit; font-weight: inherit; font: normal normal normal 100%/normal 'Courier New', 'Andale Mono', monospace;"><a href="https://developer.mozilla.org/en/CSS/length" rel="custom nofollow" style="color: #336699; text-decoration: none;"><length></length></a></code> values as mathematical expressions.</span></code></span>" (see <a href="https://developer.mozilla.org/en/CSS/-moz-calc">https://developer.mozilla.org/en/CSS/-moz-calc</a>).<br />
<br />
Support for calc is kind of a big deal as it allows you to do things that are annoying as hell to accomplish today. For example, perhaps you'd like to have three columns, left and right fixed width and center resized to fill available space:<br />
<br />
<div style="text-align: center;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">[200px][100% - 400px, minimum 250px][200px]</span></div><br />
The problematic bit is the middle column - it's actually fairly arcane to setup mixed unit calculations presently. calc makes it easy:<br />
<pre class="Xml" name="code"><!DOCTYPE html>
<html>
<head>
<title>-moz-calc</title>
<style type="text/css">
body { margin: 0px }
#container {
min-width: 650px;
width: 100%;
}
#left {
width: 200px;
background-color: #F0F0F0;
float: left;
}
#center {
min-width: 250px;
width: -moz-calc(100% - 400px);
background-color: #A0A0A0;
float: left;
}
#right {
width: 200px;
background-color: #F0F0F0;
float: left;
}
</style>
</head>
<body>
<div id="container">
<div id="left">left!</div>
<div id="center">center!!</div>
<div id="right">righter!</div>
</div>
</body>
</html>
</pre>This snippet produces fixed-width left and right columns with a center column whose width is dynamically calculated using a mixture (% and px) of length units. The container div is used to prevent the left/center/right divs from dropping to the next line if the browser is sized down far enough the min-width on the center kicks in (we don't want it infinitely narrow!). Images below show the result before/after resizing the browser window:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1ConLql4LPXs5IfZ0__Xq55GSfg1aLVlwXQjrRqKOh8TaWOB-WhRTBitpRvnviGCQnILIY5n19Fq522HoJ3ESXZCtvlSi_6EbQ-hwF9wxM6CcZmNhMyda8AmcJbmaeY-K1ds5gYDieqjg/s1600/mozcalc1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="106" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1ConLql4LPXs5IfZ0__Xq55GSfg1aLVlwXQjrRqKOh8TaWOB-WhRTBitpRvnviGCQnILIY5n19Fq522HoJ3ESXZCtvlSi_6EbQ-hwF9wxM6CcZmNhMyda8AmcJbmaeY-K1ds5gYDieqjg/s400/mozcalc1.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3W3q9RUZVnarFIudR62V3-KtVDtOp9z53P9Ie0JdW-QZFLK9rsUK5VSSK4MF1ECyTOIHGiiWXzzt7r3OWC2cKHqG8lwDlAvW-M7YX_03kLtwBde4RlAv7GI5xeV8DNJ4EyPn9fHUzZIuQ/s1600/mozcalc2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="134" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3W3q9RUZVnarFIudR62V3-KtVDtOp9z53P9Ie0JdW-QZFLK9rsUK5VSSK4MF1ECyTOIHGiiWXzzt7r3OWC2cKHqG8lwDlAvW-M7YX_03kLtwBde4RlAv7GI5xeV8DNJ4EyPn9fHUzZIuQ/s320/mozcalc2.png" width="320" /></a></div><div class="separator" style="clear: both; text-align: left;"><br />
</div><div class="separator" style="clear: both; text-align: left;">The ability to easily (the ease being the key here; it was possible to pull off similar results before but not <i>easily</i>) create page structures using mixed-unit calculated sizes is an AWESOME improvement. Now all we need is a full implementation in Firefox and the other browsers :) Oh and user adoption sufficient to justify use on significant sites.</div>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com1tag:blogger.com,1999:blog-4949964667779944881.post-38367710584811690562011-03-07T18:03:00.000-08:002011-03-07T18:03:46.890-08:00C# using is the loan pattern in ScalaOne of the remarkably handy little features of C# is the using statement (<a href="http://msdn.microsoft.com/en-us/library/yh598w02.aspx">http://msdn.microsoft.com/en-us/library/yh598w02.aspx</a>). I miss it so in Java! However, Scala will let me make my own, complete with some nice additions. Using relies on <a href="http://msdn.microsoft.com/en-us/library/system.idisposable.aspx">IDisposeable</a> for cleanup; for example purposes we'll use Java's <a href="http://download.oracle.com/javase/6/docs/api/java/io/Closeable.html">Closeable</a>. Our goal is to be able to write code similar to:<br />
<pre class="Scala" name="code">...
using(new FileWriter(file)) { fw => fw append code }
...
</pre>Simple but pretty cool. Our FileWriter could be any Closeable. This code breaks down like this:<br />
<div class="separator" style="clear: both; text-align: left;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAlsE5CAg52nSHW1b1cZ_I6msz0gHSnfS8RmX6OdJf9N5ocUkPO3ZgamV2PpMNhD6HaJTHG2zz5-x9KPu-4Mn8LRp-9ZE0TUT2q5GvhdJ6n8bt2Wg4m-jDkSlNYOe0OCjcXLWFRrUlg3Ky/s1600/fw.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAlsE5CAg52nSHW1b1cZ_I6msz0gHSnfS8RmX6OdJf9N5ocUkPO3ZgamV2PpMNhD6HaJTHG2zz5-x9KPu-4Mn8LRp-9ZE0TUT2q5GvhdJ6n8bt2Wg4m-jDkSlNYOe0OCjcXLWFRrUlg3Ky/s1600/fw.png" /></a></div>using is just a Scala function:<br />
<pre class="Scala" name="code">package com.active.scala.util
import java.io.Closeable
object Loans {
def using[T <: Closeable, R](c: T)(action: T => R): R = {
try {
action(c)
} finally {
if (null != c) c.close
}
}
}
</pre>And again with some notes:<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFzaqpfX1nkVQ-GJPWIu-ARanzSFYwS_OEM4V3uZ0x6KjcS9g9W5RQd5SLH-CkdiwTXFKe13DpBT-5N28mdPCLtQ2Tfn4-c-dRnFeXVXDCYcQozWmaOnWnWeAbD6EOgQWbh7iGYAJ60rNr/s1600/fw.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: left;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFzaqpfX1nkVQ-GJPWIu-ARanzSFYwS_OEM4V3uZ0x6KjcS9g9W5RQd5SLH-CkdiwTXFKe13DpBT-5N28mdPCLtQ2Tfn4-c-dRnFeXVXDCYcQozWmaOnWnWeAbD6EOgQWbh7iGYAJ60rNr/s1600/fw.png" /></a></div>Note that we can return a value from the Scala using if we wish. This is a simple example of the <a href="http://scala.sygneca.com/patterns/loan">loan pattern</a> in Scala. We can apply this to any resource we would typically have to use in a try { ... } finally { cleanup my resource } structure in Java.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com29tag:blogger.com,1999:blog-4949964667779944881.post-56959944786975483982011-03-07T09:50:00.000-08:002011-03-07T09:50:20.233-08:00How jQuery.ready worksFor some time I have been meaning to poke a little at how jQuery works. Line references and code samples are from the dev version of jQuery 1.5.1 - <a href="http://code.jquery.com/jquery-1.5.1.js">http://code.jquery.com/jquery-1.5.1.js</a>.<br />
<br />
First up, the ready function (http://api.jquery.com/ready/).<br />
<pre class="Xml" name="code"><!DOCTYPE html>
<html>
<head>
<title>How the heck does jQuery work?</title>
<script type="text/javascript" src="http://code.jquery.com/jquery-1.5.1.js" ></script>
<script type="text/javascript">
$(onStart); //short-hand for $(document).ready(onStart);
function onStart($) {
alert('hello');
}
</script>
</head>
<body>
</body>
</html>
</pre><br />
The $(onStart) call requests that our function be run when the DOM is available; normally if we just called onStart here the DOM probably wouldn't be loaded and we thus be unable to do cool stuff like attach event handlers or manipulate our page here because none of the objects have been created yet. The stack at time of loading is very short: We are at the very bottom!<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzrAb4GfuFYz3GKOgSRMQSyJEEPjP5YNGZ_BF9CiNJ5nHwwI0FEzIF9zOSbXenomHncbxI58snPHzOj-UgNWwHsngJ0j-NibteP0v4QI65-JfWmVVHtBho-RUqinl8gqJmRDJKzpcudQzO/s1600/jQueryStartup.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzrAb4GfuFYz3GKOgSRMQSyJEEPjP5YNGZ_BF9CiNJ5nHwwI0FEzIF9zOSbXenomHncbxI58snPHzOj-UgNWwHsngJ0j-NibteP0v4QI65-JfWmVVHtBho-RUqinl8gqJmRDJKzpcudQzO/s1600/jQueryStartup.png" /></a></div><br />
Somewhat later our onStart function will be run via callback from jQuery. Presumably jQuery runs the ready function(s) we submit in response to some event so it seems natural to wonder what the callstack looks like when our onStart function actually runs. It turns out we are called from a stack of jQuery functions, rooted on DOMContentLoaded():<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYe-SChjIiqlCQcQRYKqs0_5t3lQktf0lPOV74fKUvhQ6LYnL8kTtyBcmdNJQj6_6ZPU4kpg0yVN7745Q8EjZjSejve4NL0IenAJje6wrPVlfK_A1bMfr3DXVUoKY9tk05H2ecdc6bAIoN/s1600/jQueryStartup2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYe-SChjIiqlCQcQRYKqs0_5t3lQktf0lPOV74fKUvhQ6LYnL8kTtyBcmdNJQj6_6ZPU4kpg0yVN7745Q8EjZjSejve4NL0IenAJje6wrPVlfK_A1bMfr3DXVUoKY9tk05H2ecdc6bAIoN/s1600/jQueryStartup2.png" /></a></div>The handler function for DOMContentLoaded is setup differently depending on what browser capabilities are available:<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5J2oPiHplz98OJwmatuqdANnj3J5qHS7D3qykiekuAfWiRUE0YF5dWN3AmFFu8AjgB8-0xZ6t3DMrcZ4Jyvfez-61SgGc_wUD7lcbXNhK9NkcNgkd-mzP7aTGRDJIe6JtVtk2YIxcmmD5/s1600/jQueryStartup3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5J2oPiHplz98OJwmatuqdANnj3J5qHS7D3qykiekuAfWiRUE0YF5dWN3AmFFu8AjgB8-0xZ6t3DMrcZ4Jyvfez-61SgGc_wUD7lcbXNhK9NkcNgkd-mzP7aTGRDJIe6JtVtk2YIxcmmD5/s1600/jQueryStartup3.png" /></a></div><br />
jQuery uses the presence of addEventListener as a signal that DOMContentLoaded event is available (line 1052 above). In the past it has been fairly normal to use addEventListener as a way to detect a Mozilla based browser (eg http://dean.edwards.name/weblog/2005/09/busted/). DOMContentLoaded is ideal for ready() as it fires after the DOM is loaded but before things like large images have necessarily downloaded (M$ demo of this: http://ie.microsoft.com/testdrive/HTML5/DOMContentLoaded/Default.html). The "load" event on the other hand may wait for resources to load. Unfortunately, IE8 and lower don't support addEventListener (or DOMContentLoaded) so the failover to attachEvent is required.<br />
<br />
IE9 will support addEventListener (http://www.davidflanagan.com/2010/03/ie9-will-have-a.html) and DOMContentLoaded (http://blogs.msdn.com/b/ie/archive/2010/03/26/dom-level-3-events-support-in-ie9.aspx) ... Yay!<br />
<br />
The code at line ~1050 (above) merely sets up a DOMContentLoaded function based on the capabilities that appear to be available; something still needs to actually call it. The callbacks are attached to events by bindReady, at line 429.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHNuZdzJWRaC-7kz8VyYaBTTTW7tmC_iZ5F7RJouCLoEDGu4z4LYxHL8NMqYPooEvILbgDIjXKzMVmYtAtKgrOLWeWZxSHi2Atu11PAyK1byCjEMZNFvo-lSRvXCpybyFPLCT8-xYFwFuh/s1600/jQueryStartup6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHNuZdzJWRaC-7kz8VyYaBTTTW7tmC_iZ5F7RJouCLoEDGu4z4LYxHL8NMqYPooEvILbgDIjXKzMVmYtAtKgrOLWeWZxSHi2Atu11PAyK1byCjEMZNFvo-lSRvXCpybyFPLCT8-xYFwFuh/s1600/jQueryStartup6.png" /></a></div><br />
Multiple events are setup to callback (eg both DOMContentLoaded and load) to jQuery.ready to ensure that <i>something </i>actually runs it; exactly which ones are setup varies based on the browser capabilities available. Taking this type of nonsense away from most developers and having it "just work" across browsers is a large part of why jQuery is awesome - it takes FOREVER to figure out how to make this stuff work reliably.<br />
<br />
When DOMContentLoaded ultimately fires it calls jQuery.ready. Ready does some work to try to confirm it's really time to run, double-checks the document is actually ready (line 407, again shielding us from browser weirdness), then executes the list of functions that have been listed to run on ready via resolveWith:<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTcH_TjICl0TYndM_mJcdh4hSmjlO0AMaKuTlx1ukUyNmwNo_yDxN8kzqVGdkvYCumGGsDeA1900Camn_C8Uu0k-fjC1IRDIU77-zTNI_T3QCgq4yHDdeb8F9oOBUfYTgfnxyRsvQZqHVc/s1600/jQuery-ready.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTcH_TjICl0TYndM_mJcdh4hSmjlO0AMaKuTlx1ukUyNmwNo_yDxN8kzqVGdkvYCumGGsDeA1900Camn_C8Uu0k-fjC1IRDIU77-zTNI_T3QCgq4yHDdeb8F9oOBUfYTgfnxyRsvQZqHVc/s1600/jQuery-ready.png" /></a></div><br />
resolveWith does some gymnastics to try to avoid running repeatedly, then runs each of our callbacks:<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9K1riYCNdeKq5-qq8O7edIM2sRNQGqubCRPbZhpKi57Gu0VeygxMeDZ6jzYVNa0J8vJWmwivBCy1KrLiSwo1hwEnS_gssWQH2HRj5fWT06odmS9iMxF8BdyikyvYdwR8crEVV8x0NAC8_/s1600/jQueryStartup4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9K1riYCNdeKq5-qq8O7edIM2sRNQGqubCRPbZhpKi57Gu0VeygxMeDZ6jzYVNa0J8vJWmwivBCy1KrLiSwo1hwEnS_gssWQH2HRj5fWT06odmS9iMxF8BdyikyvYdwR8crEVV8x0NAC8_/s1600/jQueryStartup4.png" /></a></div><br />
That was kind of cool but it is sort of hard to understand merged into all the other jQuery code. Perhaps what is called for is to build our own :) Let us suppose we wanted our library to focus on running functions when something happened. We want usage to be something like this:<br />
<pre class="Javascript" name="code">//SAMPLE USE OF OUR LIBRARY
when.DOMLoaded(onDomLoad);
function onDomLoad() {
alert('hello');
}
</pre>For this to work we'll need to declare 'when' and expose DOMLoaded. Something basic should suffice:<br />
<pre class="Javascript" name="code">//OUR LIBRARY
function When() {
this.DOMLoaded = function(userFn) {
}
}
var when = new When();
</pre>Spiffy! All we have to do now is implement it. Luckily jQuery already did so we can rip off much of the relevant part of their implementation (simplified somewhat to try to keep the example clear):<br />
<pre class="Javascript" name="code">//OUR LIBRARY
function When() {
var domLoadedHandlers = [];
var handleReady = function() {
while (domLoadedHandlers.length > 0) {
domLoadedHandlers.shift()();
}
}
// Mozilla, Opera and webkit nightlies currently support this event
if ( document.addEventListener ) {
// Use the handy event callback
document.addEventListener( "DOMContentLoaded", handleReady, false );
// A fallback to window.onload, that will always work
window.addEventListener( "load", handleReady, false );
} //else IE event model is used ... etc
this.DOMLoaded = function(userFn) {
domLoadedHandlers.push(userFn);
}
}
var when = new When();
//SAMPLE USE OF OUR LIBRARY
when.DOMLoaded(onDomLoad);
</pre><br />
The implementation above has plenty of limitations: It only works on some browsers, it doesn't handle submission of a ready function after the loaded event has fired, errors from a ready function abort all processing, and so on.<br />
<br />
At this point we can start to really appreciate the hard work jQuery is doing for us. Even seemingly trivial scripts - do something when the page is ready - are fairly ugly, with excitingly frequent browser-specific corner cases and hacks (IE-specific ready detection with special casing based on whether or not we are in a frame...). To make matters worse, just when you think you have it all correct a new version of the browser comes out and muddles everything up again. Thank god for jQuery!Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com4tag:blogger.com,1999:blog-4949964667779944881.post-40007278744874915832011-02-25T10:52:00.000-08:002011-02-28T08:30:07.652-08:00Scala Permutations 2My second attempt at permutations in Scala (see <a href="http://whileonefork.blogspot.com/2011/02/scala-vs-erlang-vs-java-permutations.html">first attempt</a>): <br />
<br />
<pre class="Scala" name="code">def perms3[T](L: List[T]):List[List[T]] = L match {
//permutations of a single element is simply a list containing a list with that element
case head :: Nil => List(List(head))
case head :: tail => {
(List[List[T]]() /: L)((result, head) => (List[List[T]]() /: perms3(L-head))((accum, tailperm) => (head :: tailperm) :: accum) ::: result)
}
case Nil => List[List[T]]()
}
</pre><br />
I actually wound up with three versions (perms, perms2, perms3, all shown below) while trying to get rid of the mutable variable from the first version:<br />
<br />
<pre class="Scala" name="code">object Perm {
def main(args : Array[String]) : Unit = {
tryPerms ("A" :: Nil)
tryPerms ("A" :: "B" :: Nil)
tryPerms ("A" :: "B" :: "C" :: Nil)
}
def tryPerms(input: List[String]) {
println ("Permutations of " + input.mkString(",") + ", v1")
for (perm <- perms (input))
println (perm.mkString(","))
println ("Permutations of " + input.mkString(",") + ", v2")
for (perm <- perms2 (input))
println (perm.mkString(","))
println ("Permutations of " + input.mkString(",") + ", v3")
for (perm <- perms3 (input))
println (perm.mkString(","))
println
}
def perms[T](L: List[T]):List[List[T]] = L match {
//permutations of a single element is simply a list containing a list with that element
case head :: Nil => List(List(head))
case head :: tail => {
var result = List[List[T]]()
for (head <- L; tail <- perms(L-head))
result = (head :: tail) :: result
result
}
case Nil => List[List[T]]()
}
def perms2[T](L: List[T]):List[List[T]] = L match {
//permutations of a single element is simply a list containing a list with that element
case head :: Nil => List(List(head))
case head :: tail => {
L.foldLeft(List[List[T]]())((result, current) => {
perms2(L-current).foldLeft(List[List[T]]())((r, c) => {
(current :: c) :: r
}) ::: result
})
}
case Nil => List[List[T]]()
}
def perms3[T](L: List[T]):List[List[T]] = L match {
//permutations of a single element is simply a list containing a list with that element
case head :: Nil => List(List(head))
case head :: tail => {
(List[List[T]]() /: L)((result, head) => (List[List[T]]() /: perms3(L-head))((accum, tailperm) => (head :: tailperm) :: accum) ::: result)
}
case Nil => List[List[T]]()
}
}
</pre><br />
Version two gets rid of the offensive mutable result variable from version one. Version three re-expresses version two more concisely (perhaps to the point of mild illegibility!).<br />
<br />
Output is:<br />
<pre>Permutations of A, v1
A
Permutations of A, v2
A
Permutations of A, v3
A
Permutations of A,B, v1
B,A
A,B
Permutations of A,B, v2
B,A
A,B
Permutations of A,B, v3
B,A
A,B
Permutations of A,B,C, v1
C,A,B
C,B,A
B,A,C
B,C,A
A,B,C
A,C,B
Permutations of A,B,C, v2
C,A,B
C,B,A
B,A,C
B,C,A
A,B,C
A,C,B
Permutations of A,B,C, v3
C,A,B
C,B,A
B,A,C
B,C,A
A,B,C
A,C,B
</pre>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com3tag:blogger.com,1999:blog-4949964667779944881.post-2538158222082581192011-02-24T17:07:00.000-08:002011-02-24T17:07:50.667-08:00Scala vs Erlang vs Java permutationsErlang permutations is ... terse:<br />
<pre class="Erlang" name="code">perms([]) -> [[]];
perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])].
</pre>I have entire post about this terse little gem, including the Java equivalent (rather less terse!); see <a href="http://whileonefork.blogspot.com/2010/11/erlang-so-awesome-it-makes-my-brain.html">Erlang: so awesome it makes my brain bleed</a>.<br />
<br />
It seemed rather reasonable to try this in Scala. It's a functional programming language with list comprehensions so surely it will be pretty terse! My first draft as a Scala newb:<br />
<pre class="Scala" name="code">def perms[T](L: List[T]):List[List[T]] = L match {
//permutations of a single element is simply a list containing a list with that element
case head :: Nil => List(List(head))
case head :: tail => {
//not quite sure how to dodge use of the result var to accumulate results :(
var result = List[List[T]]()
for (head <- L; tail <- perms(L-head))
result = (head :: tail) :: result
result
}
case Nil => List[List[T]]()
}
</pre>Much nicer than the Java version but it doesn't hold a candle to Erlang for terse! It does however have the marked benefit of being vastly easier to read and comprehend ;) I wish I knew how to get rid of the result var :(<br />
<br />
For reference, the Java equivalent (written to clearly express what is happening more than to be as lean as possible):<br />
<pre class="Java" name="code">import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class Perms {
public static void main(String[] argv) {
System.out.println(perms(Arrays.asList("A", "B", "C")));
}
public static List<List<String>> perms(List<String> input) {
List<List<String>> output = new ArrayList<List<String>>();
for (String H : input) {
//L--[H]
List<String> LminusH = new ArrayList<String>(input);
LminusH.remove(H);
if (LminusH.isEmpty()) {
//[H|T] when T is empty
output.add(new ArrayList<String>(Arrays.asList(H)));
} else {
for (List<String> T : perms(LminusH)) {
//a list made up of [H|T]
List<String> HT = new ArrayList<String>();
HT.add(H);
HT.addAll(T);
output.add(HT);
}
}
}
return output;
}
}
</pre>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com3tag:blogger.com,1999:blog-4949964667779944881.post-47715245955399264912011-02-22T12:17:00.000-08:002011-02-22T12:17:38.398-08:00JUnit 4 ScalaHaving just tried the very basics of getting ScalaIDE up and running for Scala I naturally immediately want unit tests because like any good Java developer I believe my code doesn't work until proven otherwise.<br />
<br />
The internets turned up a number of fairly ugly sounding solutions for getting unit testing going in Scala so I thought rather than try that, why not just see if I can use ye olde JUnit "just like Java" (does that make me a bad Scala user?). Scala compiles to normal .class files so provided I get both Scala and JUnit on classpath it should work just fine. And it does!<br />
<br />
First I expressed my dependence on JUnit. As I'm using a simple Eclipse test project I just right-clicked the project, selected Build Path>Add Libraries, picked JUnit and then selected to use JUnit 4. This added the JUnit 4 entry to my .classpath for me:<br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry kind="src" path="src"/>
<classpathentry kind="con" path="org.scala-ide.sdt.launching.SCALA_CONTAINER"/>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.6"/>
<classpathentry kind="con" path="org.eclipse.jdt.junit.JUNIT_CONTAINER/4"/>
<classpathentry kind="output" path="bin"/>
</classpath>
</pre><br />
Next I created a test class, imported the bits of JUnit I needed, slapped @Test onto a few methods, held my breath, and hit Alt+Shift+X, T to run as JUnit test. And like magic it worked perfectly!<br />
<br />
The code:<br />
<pre class="Scala" name="code">import org.junit.Assert._
import org.junit.Test
class FizzBuzzTestCase {
@Test
def meFail() = assertEquals(1, 2)
@Test
def meSucceed() = assertEquals(1, 1)
}
</pre>The outcome:<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicy_npwHpYx8yNX7kfEyrZZSYHq4cOXf9379HmEWSYtfPYN8rJsPRqY6KNr5w57jfAwuRe_LiIRdBUa_fK2P7y9Sys2rrQlEcz8NqF3KpIvNa8seSS2tJwNys557mluVMvNNjDwCcSOqaL/s1600/ScalaUnit.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="65" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicy_npwHpYx8yNX7kfEyrZZSYHq4cOXf9379HmEWSYtfPYN8rJsPRqY6KNr5w57jfAwuRe_LiIRdBUa_fK2P7y9Sys2rrQlEcz8NqF3KpIvNa8seSS2tJwNys557mluVMvNNjDwCcSOqaL/s320/ScalaUnit.jpg" width="320" /></a></div>Hooray! Super simple, familiar, and effective :)Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com2tag:blogger.com,1999:blog-4949964667779944881.post-23753736101755321722011-02-22T11:24:00.000-08:002011-02-24T14:16:12.221-08:00ScalaIDE vs MeI like Eclipse, I like functional programming languages, and I hear great things about Scala so I thought I'd try it out by way of the <a href="http://www.scala-ide.org/">ScalaIDE</a>.<br />
<br />
Initial installation was a little painful. The (experimental) Eclipse 3.6 Helios update site wouldn't work so I grabbed the latest Eclipse 3.5 Galileo (Java Dev, SR2) and tried to use the update site for that. Sadly this just gave me a "we hate your certificate" message with no option to trust anyway and aborted the install process. Arglefarg. Downloading the zip file of the update site and pointing Eclipse at the provided content.jar also produced a "we hate your certificate" message but this time there was an option to trust anyway and finally the damn thing got installed. Finally!<br />
<br />
Having the IDE installed adds New Project>Scala Wizards>Scala Project. This sets up the .classpath and .project files appropriately for Scala. I like editing these files manually at times so I tend to inspect them whenever I try a new project type.<br />
<br />
<b>.classpath</b><br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry kind="src" path="src"/>
<classpathentry kind="con" path="org.scala-ide.sdt.launching.SCALA_CONTAINER"/>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.6"/>
<classpathentry kind="output" path="bin"/>
</classpath>
</pre><b>.project</b><br />
<pre class="Xml" name="code"><?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>helloscala</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.scala-ide.sdt.core.scalabuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.scala-ide.sdt.core.scalanature</nature>
<nature>org.eclipse.jdt.core.javanature</nature>
</natures>
</projectDescription>
</pre>Using the commandline interpreter hello world looks like this:<br />
<pre class="Scala" name="code">println "hello, world"
</pre>However, ScalaIDE doesn't like that; it demands a proper object declaration:<br />
<pre class="Scala" name="code">object helloapp {
def main(args : Array[String]) = println("Hello, World")
}
</pre>ScalaIDE installs the expected keyboard shortcuts (eg Alt+Shift+X, S to run a Scala app) but sadly it doesn't actually work; the IDE just spits out no class def if you try to run:<br />
<pre>java.lang.NoClassDefFoundError: helloapp
Caused by: java.lang.ClassNotFoundException: helloapp
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
Exception in thread "main"
</pre>A little poking around revealed that the projects /bin directory had nothing in it. As this is where it looks for .class files the NoClassDef issue makes sense ...except that the file I tried to run was valid and should have compiled into /bin damnit!<br />
<br />
After playing around a bit I finally figured out that the first file I created (the minimal hello world that the IDE doesn't like) doesn't compile in the IDE and apparently that prevented other files added to the project from being compiled either! WTF! In any case, once the offending file was fixed and the Problems window reports no Scala issues the expected .class files show up in /bin and it becomes possible to run as Scala App from within Eclipse.<br />
<br />
In painful detail:<br />
<ol><li>Create a few valid Scala files in Eclipse, with Project>Build Automatically turned on</li>
<ol><li>.class files show up in the /bin dir, we can run as Scala App just fine<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3VxTBYvFcqRVMcqJvjNCMDIdJTqsmgUtYWkQAsJq8-IF3JRJ-1vUiCG0jOqXnBWk-rspbVJ2oxYXSyOVFHXxYGoqZm6HkoI6WpoMq7ELG4tf8SPlg2YjmNNnzfoX15VKf8tuWbmC_6vPs/s1600/HappyScalaIDE.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="191" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3VxTBYvFcqRVMcqJvjNCMDIdJTqsmgUtYWkQAsJq8-IF3JRJ-1vUiCG0jOqXnBWk-rspbVJ2oxYXSyOVFHXxYGoqZm6HkoI6WpoMq7ELG4tf8SPlg2YjmNNnzfoX15VKf8tuWbmC_6vPs/s320/HappyScalaIDE.jpg" width="320" /></a></div></li>
</ol><li>Add a Scala file with a problem</li>
<ol><li>The IDE reports it in Problems; we can run the other files as Scala App just fine (because their .class files are in /bin)<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEho2QDIWzxI6JkkBwP60056O-hQXB61wtBQrDRndba1XC6aaQs_UFKMlogm4d7ql2GzocXKkQ1adeDX5NJeaAjYBHvfRjl4nLRFIyLRlbE8mbuoPtuTg8hprEgjqs_I4whIHCkQagffAdPG/s1600/Unhappy1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="164" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEho2QDIWzxI6JkkBwP60056O-hQXB61wtBQrDRndba1XC6aaQs_UFKMlogm4d7ql2GzocXKkQ1adeDX5NJeaAjYBHvfRjl4nLRFIyLRlbE8mbuoPtuTg8hprEgjqs_I4whIHCkQagffAdPG/s320/Unhappy1.jpg" width="320" /></a></div></li>
<li>Other (valid) files can be run OK ... unless you do a Project>Clean, in which case all your .class files are wiped out and none are replaced because one bad file = no class files are created for any file</li>
<li>Additionally, if you modify a valid file (in a valid way) it's .class file is removed but not replaced, again because apparently one bad file = no class files for all<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcVRSPKNAzGj9But4GY7UA5ZB4HAT9t-j3XkuO2WwwahGZDCbqFHWiG6FfWH9yGvDzCUQEfTRWmM0WqxCOeH8gSuRrvnbm2qt0c2Xl9lITKBWGaF9TF3dkQlD7A-eDN41sLTxg0e2ayBZg/s1600/Unhappy2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcVRSPKNAzGj9But4GY7UA5ZB4HAT9t-j3XkuO2WwwahGZDCbqFHWiG6FfWH9yGvDzCUQEfTRWmM0WqxCOeH8gSuRrvnbm2qt0c2Xl9lITKBWGaF9TF3dkQlD7A-eDN41sLTxg0e2ayBZg/s320/Unhappy2.jpg" width="320" /></a></div></li>
</ol></ol><div>It's not that big of a deal as I usually keep my files in a compilable state, but it definitely gives the impression that the tool (ScalaIDE, not Scala itself) is still pretty immature. Apparently such problems plague others as well - <a href="http://www.assembla.com/spaces/scala-ide/tickets/1000073-java-lang-noclassdeffounderror-when-launching-application">http://www.assembla.com/spaces/scala-ide/tickets/1000073-java-lang-noclassdeffounderror-when-launching-application</a>. Amusingly other features that one might think of as "advanced", such as refactor support for extract variable and the like are working for me, it's just the basic compilation management that seems weak.<br />
<br />
On the plus side Scala itself is awesome. Anytime a language lets me pass a function it makes me happy ;)<br />
<pre class="Scala" name="code">object fizzbuzz {
def main(args : Array[String]) = {
//OMG I can pass a function like a first class citizen!
//Damnit Java ... let me use functions this way
(1 to 100).foreach(fizzbuzz)
}
def fizzbuzz(i : Int) = {
val isFizz = i % 3 == 0
val isBuzz = i % 5 == 0
if (isFizz) print ("Fizz")
if (isBuzz) print ("Buzz")
if (!isFizz && !isBuzz) print (i)
println
}
}
</pre>I hope the ScalaIDE stabilizes a bit because so far Scala looks awesome and I am fond of working in Eclipse.</div>Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com2tag:blogger.com,1999:blog-4949964667779944881.post-69419815884628895442011-02-21T18:34:00.000-08:002011-02-21T18:34:31.607-08:00Java multi-monitor screenshotsRandom snippets I might want again one day ;)<br />
<br />
Create a "C:\\CanDelete" directory before running any of this, or change the path building.<br />
<br />
Take a screenshot of each monitor individually:<br />
<pre class="Java" name="code">package com.blogspot.whileonefork.screencapture;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
public class EachMonitor {
public static void main(String[] argv) throws Exception {
//Take a screenshot of every monitor
GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice[] screens = ge.getScreenDevices();
for (GraphicsDevice screen : screens) {
Robot robotForScreen = new Robot(screen);
Rectangle screenBounds = screen.getDefaultConfiguration().getBounds();
//The screen bounds have an annoying tendency to have an offset x/y; we want (0,0) => (width, height)
screenBounds.x = 0;
screenBounds.y = 0;
BufferedImage screenShot = robotForScreen.createScreenCapture(screenBounds);
String ext = "jpg";
String outputFile = "C:\\CanDelete\\screenshot_"
+ screen.getIDstring().replaceAll("[^A-Za-z0-9]", "_")
+ "."
+ ext;
ImageIO.write(screenShot, ext, new File(outputFile));
System.out.println("Saved " + outputFile
+ " of " + screen.getIDstring()
+ " xy=(" + screenBounds.x + "," + screenBounds.y + ")"
+ " bounds=("+ screenBounds.width + "," + screenBounds.height + ")");
}
}
}
</pre><br />
Take a single big screenshot of all monitors:<br />
<br />
<pre class="Java" name="code">package com.blogspot.whileonefork.screencapture;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
public class AllMonitors {
public static void main(String[] argv) throws Exception {
/**
* Take one big screenshot of the entire UI.
* On my Windows box monitors seem to act like they are side by side on X, extending on Y.
* Seems, at least on windows, to ignore any offset config setup in display properties.
*/
GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice[] screens = ge.getScreenDevices();
Rectangle allScreenBounds = new Rectangle();
for (GraphicsDevice screen : screens) {
Rectangle screenBounds = screen.getDefaultConfiguration().getBounds();
allScreenBounds.width += screenBounds.width;
allScreenBounds.height = Math.max(allScreenBounds.height, screenBounds.height);
}
Robot robot = new Robot();
BufferedImage screenShot = robot.createScreenCapture(allScreenBounds);
String ext = "jpg";
String outputFile = "C:\\CanDelete\\AllMonitors"
+ "."
+ ext;
ImageIO.write(screenShot, ext, new File(outputFile));
System.out.println("Saved " + outputFile
+ " of all monitors "
+ " xy=(" + allScreenBounds.x + "," + allScreenBounds.y + ")"
+ " bounds=("+ allScreenBounds.width + "," + allScreenBounds.height + ")");
}
}
</pre><br />
All this has been tested exactly once on a Windows box with two monitors of substantially different size; ymmv :)Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com50tag:blogger.com,1999:blog-4949964667779944881.post-1607733296959139142011-02-17T10:39:00.000-08:002011-02-17T10:39:01.482-08:00Bash delete directories except that oneRan into a scenario where I needed to delete all subdirectories of target except target/keepme/. So I want <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">rm -rf target/subdir1 target/subdir2 ... target/subdirN</span> with one directory omitted. I want to use grep to remove the directory I want to keep from the list so something like this should work:<br />
<br />
<br />
<ol><li>Print the list of directories <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">echo target/*/</span></li>
<li>Flip space separator to newline <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">tr " " "\n"</span></li>
<li>Remove the line for the directory to keep <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">grep -v keepme</span></li>
<li>Flip newline back to space <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">tr "\n" " "</span></li>
<li>Run rm -rf on the results from steps 1-4</li>
</ol><br />
As a 1-liner:<br />
<br />
<pre class="Bash" name="code">rm -rf $(echo target/*/ | tr " " "\n" | grep -v keepme | tr "\n" " ")
</pre><br />
Noting it in case I need to do it again someday ;)Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com6tag:blogger.com,1999:blog-4949964667779944881.post-62953343373710600802011-02-10T18:06:00.000-08:002011-02-14T15:06:08.977-08:00Terse vs Verbose conditional execution in BashAs today is apparently my day for remedial Bash (see <a href="http://whileonefork.blogspot.com/2011/02/bash-for-each-line-1-liners.html">previous</a> <a href="http://whileonefork.blogspot.com/2011/02/for-directories.html">posts</a> about <a href="http://whileonefork.blogspot.com/2011/02/do-or-die.html">it</a>) I found myself with a complete script (not published) that I felt could be much more terse. This led me to start looking into ways to compress things. In the <a href="http://whileonefork.blogspot.com/2011/02/do-or-die.html">do or die</a> post we saw one such example, a few more follow.<br />
<br />
My examples for this post are largely from the excellent <a href="http://www.ibm.com/developerworks/library/l-bash3.html">Bash by example, part 3</a> post on IBM. Refer to The Advanced Bash Scripting Guide sections <a href="http://tldp.org/LDP/abs/html/testconstructs.html">7.1 Test Constructs</a> and <a href="http://tldp.org/LDP/abs/html/comparison-ops.html">7.3 Other Comparison Operators</a> for details on the comparisons used. The examples shown were tested on Solaris 5.10.<br />
<br />
<b>Correct number of arguments</b><br />
Verbose:<br />
<pre class="Bash" name="code">if [ $# -ne 2 ]
then
echo "Please specify ebuild file and unpack, compile or all"
exit 1
fi
</pre>Terse:<br />
<pre class="Bash" name="code">[ $# == 2 ] || {echo "Please specify ebuild file and unpack, compile or all"; exit 1;}
</pre><br />
This works because Bash <a href="http://en.wikipedia.org/wiki/Short-circuit_evaluation">short-circuits</a>. That is for <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">A || B</span> we only execute B if A is true, if A is false the result is already known to be false and we don't have to run B at all.<br />
<br />
<b>NOTE: </b>Example above was corrected to {} instead of (); this is important as you do NOT want a subshell or else exit won't actually break out of your script. { commands; } doesn't create a subshell and thus works as expected.<br />
<br />
<b>Set a variable to a default value if unset</b><br />
Verbose:<br />
<pre class="Bash" name="code">if [ -z "$DISTDIR" ]
then
# set DISTDIR to /usr/src/distfiles if not already set
DISTDIR=/usr/src/distfiles
fi
</pre>Terse:<br />
<pre class="Bash" name="code"># set DISTDIR to /usr/src/distfiles if not already set
[ -z "$DISTDIR" ] && DISTDIR=/usr/src/distfiles
</pre>This time we use short-circuited &&: if DISTDIR is null the first part is true and the second part executes.<br />
<br />
Terse option 2:<br />
<pre class="Bash" name="code">[ -n "$DISTDIR" ] || DISTDIR=/usr/src/distfiles
</pre>If DISTDIR is NOT null the first part is true and we skip the second. If DISTDIR is null the first part is false and we have to execute the second part.<br />
<br />
<b>Make sure we have a clean directory to work in</b><br />
Verbose:<br />
<pre class="Bash" name="code">if [ -d ${WORKDIR} ]
then
rm -rf ${WORKDIR}
fi
mkdir ${WORKDIR}
</pre>Terse:<br />
<pre class="Bash" name="code">[ ! -d ${WORKDIR} ] || rm -rf ${WORKDIR}
mkdir ${WORKDIR}
</pre>Note the addition of ! in the terse version: we want the first part to check false if the directory exists to ensure the second part executes, removing the directory.<br />
<br />
<b>Closing Notes</b><br />
We can obviously continue with just about every if statement that only has one or two short actions.<br />
<br />
A huge advantage of the verbose versions is that people are more used to them. Terse is nice in some ways but I strongly suspect more developers understand <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">if [ blah ] then echo exit fi</span> than <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">[ something ] || (echo; exit)</span>. There may be additional ramifications to the reduction; I'm not Bash guru enough to say they are exact equivalents.<br />
<br />
Another possible advantage to the short-form is it makes error checking easy enough to validate commands we usually just assume to work. For example, don't just <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">tar xzf whatever</span> and assume it unpacked, actually check it worked rather than detecting the failure later when some subsequent step failed.<br />
<br />
Anyway, I'm far from ready to suggest that the terse form should be used everywhere but it is a useful tool to have for one-liners in the terminal and possibly to shorten scripts that have piles of dead simple if/then checks.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com3tag:blogger.com,1999:blog-4949964667779944881.post-25133735583510671152011-02-10T14:54:00.000-08:002011-02-10T14:54:38.135-08:00do or die()Apparently today is my day to learn Bash tidbits (see <a href="http://whileonefork.blogspot.com/2011/02/bash-for-each-line-1-liners.html">prev</a> <a href="http://whileonefork.blogspot.com/2011/02/for-directories.html">posts</a>). I've always been fond of concise ways to script <i>do x or die with an informative message</i>. Eg PHPs <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">something() or die("even though I can enter a helpful message I won't damnit");</span>.<br />
<br />
So how do we do this in Bash? Well this type of thing pops up fairly regularly:<br />
<pre class="Bash" name="code">mycommandhere
if [ "$?" -ne "0" ]; then
echo $cmd " failed :("
exit 1;
fi
</pre>You can use <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">A || B</span> to chain things on failure (if A succeeds B doesn't execute) ... so why don't people do this:<br />
<pre class="Bash" name="code">mycommandthatmightfail || (echo "informativeness!"; exit 1)
</pre>Note that as we used ; to separate the commands to run if mycommandthatmightfail does indeed fail they will all run even if some of them fail.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com0tag:blogger.com,1999:blog-4949964667779944881.post-37558615255956443802011-02-10T14:27:00.000-08:002011-02-10T14:27:28.664-08:00*/ for directoriesWant to do something to only the directories but not the files? Just use Bash parameter expansion: <span class="Apple-style-span" style="color: blue; font-family: 'Courier New', Courier, monospace;">*/</span> is "everything whose name ends with a /", eg directories.<br />
<br />
For example:<br />
<pre class="Bash" name="code">#print all the directories
echo */
#also print all the directories (without -d it recurses)
ls -d */
#zip all the directories in target
zip myoutputfile.zip target/*/
</pre>Kinda cool.Rodhttp://www.blogger.com/profile/01080301889766047685noreply@blogger.com1