Tuesday, February 21, 2017

Fineuploader with Grails

Fineuploader is an excellent frontend javascript library for supporting full-featured uploading capabilities such as concurrent file chunking and file resume. Our use case involves uploading extremely large files such as genomic DNA sequencing data including FASTQs, BAMs and VCFs. But the javascript library does run on it's own out of the box. You will still need to implement some server-side code to handle the file uploading.

There's already a Github repository with many examples of server-side implementations for the popular programming languages like Java, Python, PHP, node.js etc... which can be found here:


However, I could not find any examples for a Grails implementation, so once again, I rolled up my own solution which can be found below. For this implementation I've only focused on file concurrent chunking and file resume. Other features such as file deletion we're omitted on purpose, but that's not to say you couldn't modify it to support the remaining features of fineuploader.

And for the GSP I have something like this:

Monday, February 6, 2017

Gradle intellij

Commands used to get intellij to recognize the Gradle project

gradle idea

From intellij GUI, File -> Invalidate Caches/Restart

Import project from Gradle

Monday, January 23, 2017

Grails 3 assets duplicated

For some reason, in Grails 3, the assets (javascripts, css ) declarations we're being duplicated HTML source as shown below:

Generated HTML:
   <script type="text/javascript" src="/assets/jquery-2.2.0.min.js?compile=false" ></script>  
   <script type="text/javascript" src="/assets/jquery-ui.min.js?compile=false" ></script>  
   <link rel="stylesheet" href="/assets/jquery-ui.min.css?compile=false" />  
   <link rel="stylesheet" href="/assets/jquery-ui.theme.min.css?compile=false" />  
   <link rel="stylesheet" href="/assets/bootstrap.css?compile=false" />  
 <link rel="stylesheet" href="/assets/grails.css?compile=false" />  
 <link rel="stylesheet" href="/assets/main.css?compile=false" />  
 <link rel="stylesheet" href="/assets/mobile.css?compile=false" />  
 <link rel="stylesheet" href="/assets/application.css?compile=false" />  
   <script type="text/javascript" src="/assets/igv-1.0.6.js?compile=false" ></script>  
   <link rel="stylesheet" href="/assets/igv-1.0.6.css?compile=false" />  
   <script type="text/javascript" src="/assets/jquery-2.2.0.min.js?compile=false" ></script>  
 <script type="text/javascript" src="/assets/bootstrap.js?compile=false" ></script>  
 <script type="text/javascript" src="/assets/igv-1.0.6.js?compile=false" ></script>  
 <script type="text/javascript" src="/assets/jquery-ui.min.js?compile=false" ></script>  
 <script type="text/javascript" src="/assets/application.js?compile=false" ></script>  

As you can see, several assets are being defined twice, for example jquery-2.2.0.min.js, jquery-ui.min.js and igv-1.0.6.js!

My following GSP code looked like this:

   <asset:javascript src="jquery-2.2.0.min.js"/>  
   <asset:javascript src="jquery-ui.min.js"/>  
   <asset:stylesheet src="jquery-ui.min.css" />  
   <asset:stylesheet src="jquery-ui.theme.min.css" />  
   <asset:stylesheet src="application.css"/>  
   <asset:javascript src="igv-1.0.6.js" />  
   <asset:stylesheet src="igv-1.0.6.css"/>  
   <asset:javascript src="application.js"/>  

Strangely enough, if I remove the following line, then the duplicates are removed:

 <asset:javascript src="application.js"/>  

Another problem I had was that in production( but worked fine development mode), my javascript files were not being minified even after I set the configuration to use ES6 in my gradle.build file:

 assets {  
   minifyJs = false  
   minifyCss = true  
   enableSourceMaps = true  
   minifyOptions = [  
       languageMode: 'ES6',  
       targetLanguage: 'ES6'  

To work around the issue, I set the minifyJs = false as shown above.

At the moment, the asset pipeline just feels buggy and unstable and it's probably better to just disable some of the features in the configuration to get some basic things working.

Not sure if I've misunderstood how assets were meant to be used, but if somebody can explain this, please enlighten me!

Tuesday, January 17, 2017

Myrepublic broadband - How to get connected from a customer perspective

So I've finally made the switch to Myrepublic broadband, but it was a bumpy road and I wanted to share with everyone how I managed to get it all working to save yourself some headaches.

Previously, I was signed up with iiNet for my broadband connection at around $80/month for NBN 25Mbs downloads and 3 MBs uploads, unlimited usage. It was a no brainer to switch to myrepublic for just $60 uncapped speeds and unlimited usage.

When you sign up through their online process, they will tell you that they will contact you when it's been activated. I waited probably almost 2 months before I finally got an email saying my broadband has been activated, without any further instruction on how to actually start using it.

Here is a copy of the email:

Congratulations, your MyRepublic service is now active.

To access your service follow the Quick Start Guide that came with your Wi-Fi Hub+ to connect your modem to the MyRepublic network.

If you have ordered a Home Phone service, you will be sent a separate email of when your service will be active.

Our Frequently Asked Questions provide support and information to get you connected and other helpful advice.

The Team at MyRepublic

Initially I thought they would automatically cancel my iiNet service and a simple modem swap with the existing connections would be it. But I waited and waiting and monitored my iiNet account, and my broadband was still going through iiNet. So I rang up Myrepublic support to see what was going on. Turns out, they activated my NBN on a secondary port, UNI-D 2, in the box housing the modem. So I really had 2 broadband connections running to my home simultaneously activated. Once I plugged the modem into the secondary port, UNI-D 2, then I was connected to Myrepublic.

I did a speed test and got around 90 MBps downloads and 38 MBps uploads, a significant boost compared to iiNet with a much lower price.

The other issue I had with Myrepublic was the VOIP connection. I waited and waited for many months still no connection. But I received my first bill which already included the cost of the phone.
I called them up again, and they've decided to credit me back for the phone, while they try and get me connected with a new number.

Myrepublic is new to australia and is still getting their feet wet with rolling out these services. In particular, they need to improve on how they process connections, then communicate that back to the user with instructions on how to hook it up and on what ports, then improve on their billing to be more accurate based on the services connected. On the flipside, their australian based support team was very helpful, better than any offshore call centres I've dealt with from other service providers.

In the end, it's definitely worth the pain, or at least wait a few months for them to get their act together before making the switch.

Monday, January 16, 2017

RestBuilder client large file downloads

I have a Grails 2.x project using the RestBuilder plugin, and I had a use case which required downloading of large data files as client of a remote RESTful webservice. However the documentation for supporting binary downloads was scarce and painstakingly took long hours of trolling through API code of both Spring and RestBuilder. So I'd like to share with others what I did to get this working:

      def void download(String url, OutputStream outputStream) {            
           RequestCallback requestCallback = new RequestCallback() {  
                void doWithRequest(ClientHttpRequest clientRequest) throws IOException {  
                     clientRequest.headers.add('accept', 'application/octet-stream')             
           ResponseExtractor responseExtractor = new ResponseExtractor() {  
                Object extractData(ClientHttpResponse clientResponse) throws IOException {  
                     System.out.println('Headers: '+clientResponse.headers)  
                     outputStream << clientResponse.body  
           RestBuilder rest = new RestBuilder()  
           RestTemplate restTemplate = rest.getRestTemplate()  
           restTemplate.execute(url, HttpMethod.GET, requestCallback, responseExtractor)                                

I had to fallback to using Spring's RestTemplate class and barely used any of the RestBuilder plugin to implement the solution.

It's the ResponseExtractor anonymous class that does the most important job of routing the InputStream from the response to an OutputStream.

Reference: https://stackoverflow.com/questions/32988370/download-large-file-from-server-using-rest-template-java-spring-mvc/38664475#38664475

Thursday, January 12, 2017

Grails automated build number

Grails version 2.4.4

In Grails projects you can write some scripts that will automatically assign a build number.
For example, if you want to set the current date as the build number, you can add a file called _Events.groovy to your scripts folder:

 import java.text.SimpleDateFormat  
 eventCompileStart = { arg ->  
      Date date = new Date()  
      SimpleDateFormat dateFormatter = new SimpleDateFormat("yyyy.MM.dd")  
      String buildNumber = dateFormatter.format(date)  
      metadata.'app.buildNumber' = buildNumber  
      println "**** Compiling with build number: ${buildNumber}"  

And if you want to display the build number on screen, then in your GSP file you can use the following:

  ${grailsApplication.metadata['app.buildNumber'] }  

You will notice that in your applications.properties file the app.buildNumber is automatically populated.

Tuesday, December 13, 2016

Grails 3.x "Execution failed for task ':bootRun'"

When trying to run the grails application using the command "grails run-app", I encountered the following error:

 FAILURE: Build failed with an exception.  
 * What went wrong:  
 Execution failed for task ':bootRun'.  
 > Process 'command '/usr/java/jdk1.8.0_101/bin/java'' finished with non-zero exit value 1  

Even after looking at the stacktrace, it wasn't clear what was causing this error. And it seems from many google searches, that there are multiple reasons for getting such an error. But none of the solutions helped me.

As it turns out, it seems Grails 3.x doesn't like having Loggers declared in domain classes.

For example, I had the following class definition:

 class MyDomainClass {  
   Logger logger = Logger.getLogger(MyDomainClass.class)  

Once I removed the Logger declaration, then my application ran successfully. I know from experience, that previous versions of Grails 2.x allowed loggers in domain classes, but it seems it's not supported anymore?

I hope this helps somebody else from painful hours of debugging!