Solr Highlighting

I have used Lucene a few years ago when highlighting was not even feature. Now I am using Solr to perform my searches and highlighting is very much something we would like to use. At first, it was not very clear how to get it working and nor are the docs very helpful. So I've written up a short blurb on what was needed.

If you've configured your schema correctly where the fields to be highlighted must have "stored=true", then most of the work is in your constructed query. You need to add the following to your query:

• hl=true
• hl.fl=*

Here is an example query:

http://localhost:8989/solr/select?q=britta&hl=true&hl.fl=*

And an example response might be:

<response>
−
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">8</int>
−
<lst name="params">
<str name="q">britta</str>
<str name="hl.fl">*</str>
<str name="hl">true</str>
</lst>
</lst>
−
<result name="response" numFound="1" start="0">
−
<doc>
<str name="description">britta is awesome</str>
<str name="id">poddObject:67</str>
</doc>
</result>
−
<lst name="highlighting">
−
<lst name="poddObject:67">
−
<arr name="description">
<str><em>britta</em> is awesome</str>
</arr>
</lst>
</lst>
</response>

If it's still not working for you, here is another reference that provides additional things you should look for:
http://www.mail-archive.com/solr-user@lucene.apache.org/msg11031.html

Phil

Loading configuration file from Classpath

InputStream is = getClass().getClassLoader().getResourceAsStream(INDEX_CONFIG_FILE);

JUnit Hibernate Lazy initialization

To configure your JUnit testcases for testing data access through Hibernate with entity mappings using lazy initialization requires minor changes to the setUp() and tearDown() methods as follows:

    /**
     * The application context used in this test case.
     */
    protected ApplicationContext ctx;
    protected StrainDao dao;
    // keep reference to session factory to handle lazy-initialization
    protected SessionFactory sessionFactory;
    protected Session session;
    
    public PhenomeBankTestCase(final String name) {
        super(name);

        // Override the default definition for dataSource
        /*
        final String[] files = {                    
            "file:C:/workspace/mouse/src/main/webapp/WEB-INF/applicationContext-hibernate.xml",
            "classpath:applicationContextDataSource.xml",
            "file:C:/workspace/mouse/src/main/webapp/WEB-INF/phenomeBank-service.xml",             
        };*/
        
        final String[] files = {                    
                "file:./src/main/webapp/WEB-INF/applicationContext-hibernate.xml",
                "classpath:applicationContextDataSource.xml",
                "file:./src/main/webapp/WEB-INF/phenomeBank-service.xml",             
            };        

        ctx = new ClassPathXmlApplicationContext(files);
        sessionFactory = (SessionFactory)ctx.getBean(BEAN_ID_SESSION_FACTORY);
    }
    
    /**
     * Sets up the test fixture.
     */
    @Override
    public void setUp() {
        dao = (StrainDao) ctx.getBean(BEAN_ID_STRAIN_DAO);         

        session = SessionFactoryUtils.getSession(this.sessionFactory, true);
        TransactionSynchronizationManager.bindResource(this.sessionFactory, new SessionHolder(session));        
    }

    @Override
    protected void tearDown() throws Exception {
        TransactionSynchronizationManager.unbindResource(this.sessionFactory);
        SessionFactoryUtils.releaseSession(this.session, this.sessionFactory);
    }

Reference: http://note19.com/2007/08/26/lazy-initialization-with-hibernate-and-spring/

Hudson and missing maven repository artifacts

When attempting to make a build in Hudson for the first time, it complained of missing artifacts in the maven repo. I found that I had to configure the MAVEN_HOME/conf/settings.xml file to point to the maven repo by editing the localRepository element

Here's a reference for more details:
http://stackoverflow.com/questions/44144/hudson-cant-build-my-maven-2-project-because-it-says-artifacts-are-missing-from

Once the initial build was working, I had to make additional changes to make maven do a 'clean package'
and perform a build everytime a change gets submitted to SVN as shown below:

SFTP: Received message too long 1936094320

The problem:

Recently the server running SFTP was updated with some security patches applied. Soon afterwards, one of the users couldn't connect via SFTP with the following error:

$ sftp broken_user@server

Received message too long 1936094320

 I spent quite a bit of time searching for solutions and most talked about an echo statement in one of the .bashrc or .bash_profile scripts. However, upon inspection I couldn't find any echo statements.

However, I noticed that when I used the root user it worked fine:

$ sftp root@server 

 The solution:

I did manage to find a workaround, although I am still unable to explain why it works.

I ended up editing the /etc/passwd file and changed a line from:

broken_user:x:503:503::/var/websites/my_website/./:/bin/sftpsh

to

broken_user:x:503:503::/var/websites/my_website/./:/bin/bash

Extending an existing Web application WAR

Problem:

Extend an existing web application, reusing as much as possible including not just the source code, but the web resources such as JSPs, images, CSS styling, Javascript etc...

The simple way would be to branch or copy the existing application and import it into a new project, but this has a major drawback. What if there are bug fixes or changes made to the base web application? If we were to branch, then that would mean having to merge all the changes from the base web application with the new web application! I say GAH!

Solution:

If you're already using Maven as your build, then it's Maven WAR overlays to the rescue!

An article about Maven WAR overlays can be found here:
Reference: http://maven.apache.org/plugins/maven-war-plugin/examples/war-overlay.html

To automatically include all dependencies from the base WAR, I use the warpath plugin found here:
http://static.appfuse.org/maven-warpath-plugin/usage.html

You are effectively creating a WAR project that has a dependency on the base WAR project found in your maven repo. Thus, any changes made to the base WAR application can easily be picked up by simply synching up with your SVN code base and rebuilding your new WAR.

In your new web app project, when you make a build, maven will automatically merge the 2 WAR files (base WAR and new app WAR) together to create a new WAR file.

Below is a sample of the pom.xml file to enable this merging:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>au.org.australianphenomics</groupId>
  <artifactId>podd-apn-webapp</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>podd-apn-webapp</name>
  <url>http://maven.apache.org</url>
  
  <developers>
      <developer>
          <name>Philip Wu</name>
          <email>philip.wu@anu.edu.au</email>
      </developer>
  </developers>  
  

  
  
  <!-- ================ 
      Build 
   ==================== -->  
  <build>
      <finalName>podd-apn</finalName>
    <plugins>
         
     
        <plugin>
          <groupId>org.appfuse</groupId>
          <artifactId>maven-warpath-plugin</artifactId>
          <version>2.0.2</version>
          <extensions>true</extensions>
          <executions>
            <execution>
              <goals>
                <goal>add-classes</goal>
              </goals>
            </execution>
          </executions>
        </plugin>     
     
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-war-plugin</artifactId>
            <version>2.0</version>
            <configuration>           
                <dependentWarIncludes>**</dependentWarIncludes>   
            </configuration>
        </plugin>
        
    </plugins>  
         
    
  </build>
  
  <!-- ================ 
      Dependencies  
   ==================== -->  

      <dependencies>
       
        <dependency>
            <groupId>au.edu.uq.itee.maenad</groupId>
            <artifactId>podd-webapp</artifactId>
            <version>1.0</version>     
            <type>war</type>
        </dependency>
     
        <dependency>
            <groupId>au.edu.uq.itee.maenad</groupId>
            <artifactId>podd-webapp</artifactId>
            <version>1.0</version> 
            <type>warpath</type>        
        </dependency>          
  
    </dependencies>

  
  
  
  
</project>

As an 'Overlay' the web.xml files from the two WARs are not merged.
If you're looking to do just that, then you might want to take a look at:
http://cargo.codehaus.org/Merging+WAR+files

Java String.trim() not trimming Whitespace!

The problem:

I was using String.trim() to remove leading and trailing whitespace characters from an imported string.
However it turns out, that the trim() method only checks for a whitespace characters defined in ASCII.
That is, any non-ASCII whitespace characters will still persist!

Reference: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4080617

Diagnosis:

Upon inspecting each character of the string, the whitespace character returned a value of 160
The code to inspect is below:

        for (int i=0; i < string160.length(); i++) {                    
            char c = string160.charAt(i);
            int intC = (int) c;
            System.out.println(intC);            
        }

Solution:

The solution was to remove all instances of this troublesome character with the following code:

string160 = string160.replaceAll("[\\u00A0]", "");