Spring MVC and Swagger. Generating documentation for your RESTful web services - PART 2

Automating your RESTful web services documentation Using Spring MVC with Swagger for generating it

Part 2 - Implementing some business logic and configuring Swagger

1.- Configuring swagger and swagger-ui

1.1.- What is swagger ?

Swagger is a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services as stated by their official website

Personally I find Swagger to be a pretty nice way to generate my web services documentation instead of having to rewrite everything again (I already comment my code so it should be enough!)

However at the moment swagger does not support Spring MVC natively but there is an implementation over at github here

As stated by the developer not all the Swagger annotations are currently supported but the implementation works mainly with the Spring MVC annotations(@Controller, @RequestMapping , etc.) which I just what I needed.

1.2.- Integrating swagger into spring

First thing to do is to create a configuration file needed by the swagger-spring-mvc implementation. So go ahead and create a swagger.properties files under src/main/resources/config

#this is the base url for your applications (basically the root url for all your webservices)
documentation.services.basePath=http://localhost:9090/spring-mvc-swagger-tutorial
documentation.services.version=1.0

head up to your spring-web.xml configuration file and add the following lines :

1.3.- Swagger UI

In order to display the web services documentation swaggers it's necessary to download and add the swagger-ui libraries to your project

To my knowledge the Swagger UI dependency cannot be found in the maven repository, so you need to downloaded manually from here :
swagger-ui download

Once you've downloaded it unzip it in your maven webbap folder. you should end up with a tree similar to this one :

Note : It's equally possible to clone the GIT repository instead of downloading the files if you prefer

You will now need to edit the index.html file and change the discoveryUrl parameter to reflect your application

From :

   "discoveryUrl": "http://petstore.swagger.wordnik.com/api/api-docs.json",  

To your application base url (eg : http://localhost:9090/spring-mvc-swagger-tutorial/api-docs.json) :

   "discoveryUrl":"http://localhost:9090/spring-mvc-swagger-tutorial/api-docs.json", 

2.- Writing the business logic

The future controller will be using a simple DAO implementation to recover data from the database

Below is the DAO interface that will be used by the controller. To save some space I will not be posting the DAO implementation here but you can get it from GitHub here

package com.ufasoli.tutorial.swagger.springmvc.core.dao;

import com.ufasoli.tutorial.swagger.springmvc.core.status.OperationResult;
import java.util.List;

public interface DAO<T, U> {

    public OperationResult create(T object);  
    public OperationResult<T> update(U id, T object);  
    public OperationResult delete(U id);  
    public T findOne(U id);  
    public List<T> findAll();
}

I realize the interface and the implementation are far from perfect there is no proper exception handling but since that is not the main goal of this tutorial please bear with me :)

3.- Writing the Spring controllers and swagger "magic"

Finally we are almost there one last thing before we can begin writing some spring controllers and adding some swagger "magic"

There are a few swagger annotations allowing you to customize what it will be printed out by the swagger-ui interface. We'll be using the following :

• @ApiOperation : describes an API operation or method
• @ApiParam : describes an api method parameter and it's eventual constraints (ie : required)
• @ApiError : describes a possible error (HTTP Status) and the error's cause

Bellow is a sample spring controller annotated with the appropriate swagger annotations

package com.ufasoli.tutorial.swagger.springmvc.web.services;
import com.ufasoli.tutorial.swagger.springmvc.core.dao.DAO;
import com.ufasoli.tutorial.swagger.springmvc.core.model.Book;
import com.ufasoli.tutorial.swagger.springmvc.core.status.OperationResult;
import com.wordnik.swagger.annotations.ApiOperation;
import com.wordnik.swagger.annotations.ApiParam;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;
import java.util.List;


@Controller
@RequestMapping(value = "/books", produces = MediaType.APPLICATION_JSON_VALUE)
public class BookService {

    @Autowired
    private DAO<Book, String> bookDAO;


    @ApiOperation(value = "Creates a Book")
    @RequestMapping(method = RequestMethod.POST)
    public  @ResponseBody void create(@ApiParam(required = true, name = "book", value = "The book object that needs to be created")@RequestBody Book book){

        return bookDAO.create(book);
    }

    
    @ApiOperation(value = "Method to update a book")
    @RequestMapping(method = RequestMethod.PUT, value = "/{bookId}")
    public  @ResponseBody OperationResult<Book> update(
            @ApiParam(required = true, value = "The id of the book that should be updated", name = "bookId")@PathVariable("bookId") String bookId,
                    @ApiParam(required = true, name = "book", value = "The book object that needs to be updated")@RequestBody Book book){

        return bookDAO.update(bookId, book);
    }

    
    @RequestMapping(method = RequestMethod.GET)
    @ApiOperation(value = "Lists all the books in the database")
    public  @ResponseBody List<Book> list(){
        return bookDAO.findAll();
    }

    
    @ApiOperation(value = "Retrieves a book based on their id")
    @ApiErrors(value = {@ApiError(code=404, reason = "No book corresponding to the id was found")})
    @RequestMapping(method = RequestMethod.GET, value = "/{bookId}")
    public  @ResponseBody Book view(@ApiParam(name = "bookId" , required = true, value = "The id of the book that needs to be retrieved")@PathVariable("bookId") String bookId){
         
      Book book =  bookDAO.findOne(bookId);

        if(book == null){

            response.setStatus(404);
        }
        return book;
    }

    
    @ApiOperation(value = "Deletes a book based on their id")
    @RequestMapping(method = RequestMethod.DELETE, value = "/{bookId}")
    public  @ResponseBody OperationResult delete(@ApiParam(name = "bookId", value = "The id of the book to be deleted", required = true)@PathVariable("bookId") String bookId){
      return bookDAO.delete(bookId);

    }

}

4.- Time to check-out the generated documentation

Once again fire up your Jetty server by running the maven goal

  mvn jetty:run

Once your server is started head up to the following URL using your inseparable favorite browser :

http://localhost:9090/spring-mvc-swagger-tutorial/

If Murphy's law has not applied here we should see swagger-ui's homepage

Go ahead and click on the show/hide link right next to the books element. You should see a pretty nice ui show you the different api operations available along with the required params and HTTP methods :

What's even better swagger-ui offers you a light REST client allowing you to directly test your API operations by simply selecting an operation and filling the required parameters.

You can play with the different annotations and options and observe the different results that they produce

5.- Wrapping things up

So that's it, I hope this tutorial was useful to you, even though you will no longer have any excuse to not having nice up-to-date documentation for your RESTful applications.

You can find this sample application over at github :

Swagger - Spring MVC tutorial sample application code

Spring MVC and Swagger. Generating documentation for your RESTful web services - PART 1

Automating your RESTful web services documentation Using Spring MVC with Swagger for generating it

Part 1 - Setting up the WAR and spring configuration

1.- Introduction

Through this simple tutorial I will be developing a simple web application using the following technologies :

• spring (mvc, core, jdbc) : Dependency Injection, Database access, RESTful web services implementation
• hsqldb : On memory database
• swagger : Documentation framework
• swagger-spring-mvc : Swagger integration with spring mvc
• maven : build lifecycle
• jetty : web application server

The main goal of this tutorial is to illustrate how the swagger documentation framework can be integrated into a spring mvc application.

2.- The Database Model

In this tutorial we will be using a very simple model consisting of 1 table :

3.- Creating the maven project structure

There are a few of Spring archetypes when creating java web applications but I find them oversized for this simple tutorial so I will use here the org.codehaus.mojo.archetypes.webapp-javaee6 archetype :

mvn archetype:generate -DarchetypeGroupId=org.codehaus.mojo.archetypes -DarchetypeArtifactId=webapp-javaee6 -DpackageName=com.ufasoli.tutorial.swagger.springmvc -DgroupId=com.ufasoli.tutorial -DartifactId=swagger-spring-mvc -Dversion=1.0

This should generate a pretty standard maven WAR project structure

Note : You should adapt your groupId and artifactId according to your config.

4.- Dependencies

In your pom.xml file add the following dependencies that we are going to need for our project :

...
 
        ${project.build.directory}
        UTF-8
        3.2.1.RELEASE
        1.8.0.10
    

    
        
            javax
            javaee-web-api
            6.0
            provided
        

        
            com.mangofactory
            swagger-springmvc
            0.5.2
        

        
            org.springframework
            spring-core
            ${version.spring}
        

        
            org.springframework
            spring-jdbc
            ${version.spring}
        
        
            org.springframework
            spring-expression
            ${version.spring}
        
        
            org.springframework
            spring-web
            ${version.spring}
        
        
            org.springframework
            spring-webmvc
            ${version.spring}
        

        
            org.springframework
            spring-beans
            ${version.spring}
        
        
            org.hsqldb
            hsqldb
            ${version.hsqldb}
        

    

...

5.- Spring Configuration

There are many schools when it comes to creating spring context files
I usually create a spring-web.xml file under the src/main/webapp/WEB-INF folder that I will use as my web application context and an applicationContext.xml file under src/main/resources/META-INF/spring that will be my application context.

5.1.- Configuring the Application Context (applicationContext.xml)

5.1.1- Setting up the classpath scanner

5.1.2- Setting up the database

Spring has a pretty nifty way to step up an in-memory database.You can provide a list of SQL scripts that will be executed when the database is started

I have written 2 SQL scripts one for creating the database and another one to have some sample data inserted in the database

This scripts can be found under src/main/resources/sql :

CREATE TABLE BOOKS (id VARCHAR NOT NULL, title VARCHAR NOT NULL, author VARCHAR NOT NULL, publicationYear INTEGER, comment VARCHAR, PRIMARY KEY(id))

INSERT INTO BOOKS VALUES ('1', 'A Game of Thrones', 'George R. R. Martin', 1996, 'n.c.');
INSERT INTO BOOKS VALUES ('2', 'A Clash of Kings', 'George R. R. Martin', 1998, 'n.c.');
INSERT INTO BOOKS VALUES ('3', 'A Storm of Swords', 'George R. R. Martin', 2000, 'n.c.');
INSERT INTO BOOKS VALUES ('4', 'A Feast for Crows ', 'George R. R. Martin', 2005, 'n.c.');
INSERT INTO BOOKS VALUES ('5', 'A Dance with Dragons', 'George R. R. Martin', 2011, 'It was about time!');
INSERT INTO BOOKS VALUES ('6', 'The Winds of Winter', 'George R. R. Martin', 2014, 'If we are lucky!');
INSERT INTO BOOKS VALUES ('7', 'A Dream of Spring', 'George R. R. Martin', 2100, 'oh c`mon');

We will now configure the applicationContext.xml file using a HSQL in-memory database and instruct spring to execute the SQL scripts in a predefined order as well as bootstrap Spring's JDBC template class which is an utility class that simplifies the use of JDBC

Below is how the applicationContext.xml file should look like :

With this configuration every time the application is started the database will be bootstraped and some sample data will be inserted

5.2.- Configuring the Web Application Context (spring-web.xml)

The Web Application context extended Application Context which is designed for work with the standard javax.servlet.ServletContext so it's able to communicate with the container.

5.2.1- Setting up the classpath scanner & the Spring MVC configuration

I will not go into the details of Spring but to sumarize the classpath scanner for the web application context will essentially pick up the controllers and register them in the context whereas the Spring MVC configuration will define how controllers should by discovered and how URLs should be handled

5.3.- Configuring the web.xml descriptor

It's now time to tell our web app to use spring-mvc. For this to happen we need to edit the web.xml file under src/main/webapp/WEB-INF and add the spring configuration :

     

    Spring MVC Swagger integration

      
        
            index.html
        

    
    
        contextConfigLocation
        classpath:/META-INF/spring/applicationContext.xml
    

    
    
        org.springframework.web.context.ContextLoaderListener
    

    
    
        spring-servlet
        org.springframework.web.servlet.DispatcherServlet
        
        
            contextConfigLocation
            /WEB-INF/spring-web.xml
        
        1
    

    
    
        spring-servlet
        /
    

6.- Taking it for a spin

If everything goes according to plan we should be able to take our application for a spin with a simple hello world!

6.1.- Creating the HelloWorld Controller

Create a HelloWorldService.java file under src/main/java/ and the package of your choice

package com.ufasoli.tutorial.swagger.springmvc.web.services;

import org.springframework.http.MediaType;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import javax.servlet.http.HttpServletResponse;

@Controller
@RequestMapping(value = "/hello", produces = MediaType.TEXT_PLAIN_VALUE)
public class HelloWorldService {

    @RequestMapping( produces = MediaType.TEXT_PLAIN_VALUE, method = RequestMethod.GET)
    public @ResponseBody String sayHelloWorld(HttpServletResponse response){

        return "Hello World!";
    }
}

With this configuration spring will register your hello-world controller under :

  ${context.root}/hello

6.2.- Configuring jetty and running the app

We are almost there, the last thing we need to do is to configure our embedded Jetty server

Head to your pom.xml file

...


  
    ...
     
      org.mortbay.jetty
      maven-jetty-plugin
      6.1.26
      
        
          /spring-mvc-swagger-tutorial
        
        0
        
          
             9090
             60000
           
          
       
     
     ...


...

This configuration will start a jetty server under the port 9090 and deploy my application under the /spring-mvc-swagger-tutorial context-root

6.3.- Running the app

Head to your command line or favorite IDE and run the following maven goal

   mvn jetty:run

Wait for your server to start, you should see a message similar to:

...
  2013-06-06 15:23:50.668:INFO::jetty-6.1.26
2013-06-06 15:23:50.825:INFO::No Transaction manager found - if your webapp requires one, please configure one.
2013-06-06 15:23:51.135:INFO:/spring-mvc-swagger-tutorial:Initializing Spring root WebApplicationContext
log4j:WARN No appenders could be found for logger (org.springframework.web.context.ContextLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2013-06-06 15:23:51.957:INFO:/spring-mvc-swagger-tutorial:Initializing Spring FrameworkServlet 'spring-servlet'
2013-06-06 15:23:53.511:INFO::Started SelectChannelConnector@0.0.0.0:9090
[INFO] Started Jetty Server

Now crack open your favorite browser and head to :

http://localhost:9090/spring-mvc-swagger-tutorial/hello

And if the programming Gods are good you should get your well deserved greeting

6.- Wrapping-up part 1

So that's it for part 1 of the tutorial. On the second part we will create a more complex controller and see how to integrate the Swagger framework into our project

You can find this sample application over at github :

Swagger - Spring MVC tutorial sample application code

Managing system scoped jars with maven and the addjars plugin

When using some third party dependencies in a maven project you realize that sometimes some of jars are not available in any of the public maven repositories available (e.g. the oracle jdbc driver that is not available for copyright reasons)

When developing a team project and that you are required to use a jar of this type you have mainly 3 options :

1. Using Maven repository repository manager

There are a few maven repository managers out there which allow mainly to mirror distant maven repositories such as maven central, by storing the remote jars locally and making them available inside the company

This is where you can store your companies jars and wars and some other jar not available in the public repositories

I will not go into the details of the different repository managers but there are mainly 3

• Sonatype nexus repository
• JFrog Artifactory
• Archiva

I never used Archiva but I really like Nexus. There is a nice comparison matrix over here so you can make and educated choice

Anyhow if you or your company have a repository manager you can easily install the required JAR on your repository.

Once the JAR is installed in the repository it will be available to anyone that uses the repository as long as your settings.xml file or pom.xml file are properly configured.

The main advantage if this approach is that once the JAR downloaded and installed in the repository manager it will be available for all the users and future projects

The main inconvenient is the fact that you need to install a repository manager

2. Installing the JAR in the maven local repository

Installing a JAR in the user's maven local repo is quite straightforward

a) Download the JAR to the local machine

b) Install the JAR in the local repository

    mvn install:install-file -Dfile=/data/downloads/ojdbc6.jar -DgroupId=com.oracle -DartifactId=ojdbc6 -Dversion=11.2.0.3

c) Declare the JAR as a maven dependency

   

       .....

      
          
              =com.oracle
              ojdbc6
              11.2.0.3
         
     

  

This is the quickest and simplest approach to implement

The major inconvenient is that every new developer will need to install the JAR in their local repository

3. Using the addjars shade plugin and system as dependency scope

Maven allows you to easily defined dependencies with a system scope. But there is a problem with system scoped jars. When packaging the application (WAR, JAR, etc.) the system scoped jars will not be included in the packaged result. Here is where the addjar plugins comes in handy, this plugin allows you to include system scoped jars when packaging the application

Below is a quick example of a simple java web app with system scoped jar (ojdbc6.jar) :

my-app
|-- pom.xml
`-- src
    |-- main
    |   `-- java
    |   `-- webapp
    |-- external
    |   `-- ojdbc6.jar   

As stated before if I run a mvn package on this project I will get a WAR file but the ojdbc6.jar JAR will not be included under WEB-INF/lib

Thankfully the addjars plugins comes to the rescue here this plugin will include your third party jar in your project's classpath when packaging

You can see below a sample configuration for the plugin in the pom.xml file :

   
...

  
      
       com.googlecode.addjars-maven-plugin
       addjars-maven-plugin
       1.0.5
          
             
                
                  add-jars
                
                
                  
                    
                      ${basedir}/external/
                    
                 
              
           
        
      
    
  
...

The main advantage is that once the pom.xml configured with the dependency and the plugin and the JAR committed into the source control the building becomes transparent for the users

The main inconvenient comes from the fact that you will need to commit the JAR file in your source control repository and handle JAR versions manually

It's good to note that the add-jars plugin requires maven 3.0.3+

Reconfiguring a mongodb replicaset after a "loading local.system.replset config (LOADINGCONFIG)" error

Weird thing happen today when I came to work my MongoDB replicaset was not working anymore after a system update

There was a mongodb instance running but when I checked for the replicaset status I had this error message :

> rs.status()

{
        "startupStatus" : 1,
        "ok" : 0,
        "errmsg" : "loading local.system.replset config (LOADINGCONFIG)"
}

Now I have no idea what caused the problem but I could see (from the error message) that it was related to the replicaset configuration

The problem was actually pretty weird since when I printed the replicaset config it showed an old configuration

> rs.config()
{
        "_id" : "rps",
        "version" : 2,
        "members" : [
                {
                        "_id" : 0,
                        "host" : "192.168.13.200:27017"
                },
                {
                        "_id" : 1,
                        "host" : "192.168.13.201:27017"
                }
        ]
}

Since this is a development environment I didn't have a database backup so I really needed the replicaset to start running again

Thankfully MongoDB (since version 2.0) offers a relatively easy way to reconfigure a replicaset that is the rs.reconfig() command

In my case what I had to to was to reconfigure my replicaset with the only surviving member (itself) since the second one was under heavy maintenance.

This operation can be done in 6 steps :

1. Get the current replicaset configuration into a variable

> var cfg = rs.config()

2. Overwrite the members property with the remaining replicaset nodes

> cfg.members = [{"_id" : 3, "host"  : "192.168.1.100"}]
[ { "_id" : 3, "host" : "192.168.1.100" } ]

3. Reconfigure the replicaset with the new configuration

> rs.reconfig(cfg , {force : true})
{
        "msg" : "will try this config momentarily, try running rs.conf() again in a few seconds",
        "ok" : 1
}

4. Wait for the configuration to be applied

Give it a few seconds and run again the conf command to see if the new configuration is properly applied

> rs.conf()
{
        "_id" : "rps",
        "version" : 77682,
        "members" : [
                {
                        "_id" : 3,
                        "host" : "192.168.1.100:27017"
                }
        ]
}

5. Restart your mondod instance

/etc/init.d/mongod restart

6. Check that everything is running properly

rps:PRIMARY> rs.status()
{
        "set" : "rps",
        "date" : ISODate("2013-05-23T07:54:47Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 3,
                        "name" : "192.168.1.100:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 8,
                        "optime" : {
                                "t" : 1352365425,
                                "i" : 1
                        },
                        "optimeDate" : ISODate("2012-11-08T09:03:45Z"),
                        "self" : true
                }
        ],
        "ok" : 1
}

Sources :

Reconfigure a Replica Set with Unavailable Members

JSF 2 performance improvement on latest release (Mojarra)

According to this blog the latest version of Mojarra JSF (2.1.22) performance should improve dramatically when handling pages with a large quantity of objects (1000+).

jsf-performance-mojarra-improves-dramatically

Always acording to the blog author with the previous version of Mojarra, rendering a page with 1000+ components would take up to 5 times more compared to the other JSF implementation Apache MyFaces.

This is great news and comes as a welcomed improvement!

This improvement comes mainly from the resolution of the bug :

JAVASERVERFACES-2494

Custom Database function using JPA JPQL and eclipselink

Sometimes it's useful to exploit some of the native functions provided by database vendors while using JPA and JPQL.

And although it's possible to execute native queries, you'll loose some of the JPA magic in the process.

Thankfully Eclipselink (and probably Hibernate) allow some degree of flexibility through special operators.

According to the official documentation the FUNC operator allows for a database function to be call from JPQL. It allows calling any database functions not supported directly in JPQL, and calling user or library specific functions.

@Entity
public class Employee implements Serializable{
   
   @Id
   private Long id;   
   private String firstname;
   private String lastname;

  // Setters - Getters

}

Below is a quick example on how to use FUNC to call Oracle's TO_CHAR function and convert all the ids in the query to a String.

Let's say that for an obscure reason we would like to get a list of all the employees id's but not as a list of java.lang.Long but as a list of java.lang.String.

We could write the query like so :

List employeesId = 
     em.createQuery("SELECT FUNC('TO_CHAR', e.id) FROM Employee e ").getResultList();

The usage is pretty straightforward the first argument is the name of the native function followed by the function arguments

More information about this can be found in Eclipselink's official documentation here

It's good no note that FUNC will be changed to FUNCTION in JPA 2.1

JPA Eclipselink optimizing query performance on a large resultset

Recently I had to look for ways of optimizing a very slow database request on a very large ResultSet (more than 5000 results with several @OneToMany associations).

While reading EclipseLink's documentation I stumbled upon an interesting piece of information regarding handling large ResultSets. EclipseLink recommends using Streamed Cursors when handling large resultsets

According to the documentation : Cursored streams provide the ability to read back a query result set from the database in manageable subsets, and to scroll through the result set stream.

The initial query would load practically the entire database into memory in one request (all the associations are marked with QueryHint.LEFT_FETCH in order to eagerly fetch the associations upon the initial request).

I will not go into details regarding the process since it's irrelevant to this post not the goal of this post, but basically the program would query an Oracle Database transform every JPA Entity and store it in a somewhat different object in a Mongo Database

The process was really long (more than 1 hour between recovering all the objects from the Oracle Database and processing them into the Mongo Database

The DAO class

public CursoredStream getMyLargeResultset(){

   Query programsQuery = em.createNamedQuery("MyNamedQuery");
   // tell the EntityManager to return a StreamedCursor
   programsQuery.setHint("eclipselink.cursor", true);    
   return (CursoredStream) programsQuery.getSingleResult();
}

The client

public void synchronizeDatabases(){


    final int PAGE_SIZE = 50;

    //get the cursor
    CursoredStream cursor= myDao.getMyLargeResultset();
      
    // iterate through curor 
    cursor(!cursor.atEnd()){ 
 
        // get the next batch of objects and
        // cast it to the target entity 
        List myEntities = (List)(List)cursor.next(PAGE_SIZE);
 processEntities(myEntities);
      }

    cursor.close();

}

By using this technique I was able to reduce by a factor of 10 the process time of this operation!

PS: I realize that the double casting on the List is not very pretty and I could use a different approach by using the next() method without arguments but the existent method processEntities() accepted a List as a parameter and I wasn't allowed to modify that method

Documentation sources

• EclipseLink Advanced Query API
• EclipseLink Advanced Query API (Cursors)