IoC : Ulises Fasoli's blog: January 2014

SyntaxHighlighter and blogger dynamic templates

Recently I decided to switch my blogger template to use one of Google's dynamic templates since I really like the possibility to switch the presentation of the blog posts

Even though I thought this would be pretty straight-forward as it turns out I run into a problem. I use Alex Gorbatche's SyntaxHighlighter JavaScript plugin to "prettify" my code, and as it turns out Dynamic Templates and the Syntax Highlighter do not play well together (apparently the content of the posts is loaded after the script has been bootstrapped so you end up with ugly unreadable bits of code.

After goggling here and there I found a few workarounds some worked, others not.. (I'm sorry I do not have the links now, but they are easy to found). Those who worked basically consisted in altering my previous posts to give an id to each of the pre that I had used before which was unacceptable for me. So here is the solution that I came up with by inspiring myself of the different workarounds out there .

Basically what my workaround does is to delay the moment where the Highlighting will be performed to after the posts content has been loaded and then iterate through all the appropriate html tags in your code and apply highlighting

1.- Add SyntaxHighlighter's CSS resources in the template's head

<head>
     
      <link href='http://alexgorbatchev.com/pub/sh/3.0.83/styles/shCore.css' rel='stylesheet' type='text/css'/> 
      <link href='http://alexgorbatchev.com/pub/sh/3.0.83/styles/shThemeMidnight.css' rel='stylesheet' type='text/css'/>

2.- Add Javascript resouces and snippet to delay the script's bootstrapping at the bottom of the page (before the body closing tag



     

      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shCore.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushJava.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushSql.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushXml.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushXml.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushBash.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushJScript.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushPerl.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushPlain.js' type='text/javascript'/>
      <script src='http://alexgorbatchev.com/pub/sh/3.0.83/scripts/shBrushScala.js' type='text/javascript'/>

     <script language='javascript' type='text/javascript'>
       
          // define usual options
           SyntaxHighlighter.config.bloggerMode = true;  
           SyntaxHighlighter.config.clipboardSwf = 'http://alexgorbatchev.com/pub/sh/2.1.382/scripts/clipboard.swf';
        
            function doHighlights(){
           
   
              if(jQuery("pre").length > 0){

                     if(hasConsole()){
                        console.log("About to highlight fields");           
                     }

                     jQuery("pre").each(function( index ) {  SyntaxHighlighter.highlight(undefined, jQuery( this ).get(0));   });

              }
          }

      

          function hasConsole(){

         return typeof console != "undefined";
         } 
          

         setInterval(doHighlights, 500);
  
  </script>
</body>

Please note a few things :

I tried using jQuery's document ready which I find more elegant than the JavaScript timeout but it didn't work probably because it was still executed too early
Here running the highlight function every 500 milliseconts which I know it's not great...
Finally please note that the above code snippet will apply to all the pre tags in your code (which is OK for me since I only use them to display code) but it might affect your blog layout if you use them for other purposes; in that case just modify the script to check for CSS classes corresponding to those used by SyntaxHighlighter

Testing email sending functionality with a free mock smtp server

Intro

Often web applications include some email sending functionality that you have to develop/and test and there are certainly a lot of way to do this, use a fake-email recipient, use a dummy email recipient, spam your colleagues, etc. and there are some advantages and drawbacks to each of these approaches.

Recently I had to do some maintenance on an application that would send a massive amount of emails (mailing list) and I needed to test if everything was still working properly after the code modifications and since I had to change the application to allow sending multi-part messages with attachments there was a possibility that something was broken in the process.

Now for obvious reasons I could not test this in the production environment as it will have an impact on the company's SMTP server and the poor souls receiving an indecent amount of emails so I needed a mock or fake SMTP server to emulate this...

A solution

I present you Fake SMTP server. This great little Java program allows you to easily test your email sending functionality without side-effects (like spamming all your colleagues) by mimicking a real SMTP server.

Once the server is started and listening it will provide you with a list of received emails as well as SMTP logs and RAW email data and it works for both "simple" and MultiPart emails with attachments

How to use it ?

Just download it, unzip it and run it:

[/home/ufasoli/tools/smtp ~] java -jar fakeSMTP.jar

this will start the GUI but not the server for that you will need to click on the "Start server" button

There are also a few command line options that you can provide when starting the server like for example to start the server automatically :

-o : Change the output directory
-p : Change the listening port
-s : Start the server automatically when you run the GUI

As indicated by the author on the websites there are a few alternatives :

DevNullSmtp : Written in Java 1.4
SMTP4dev : Written in .Net so not cross-platform

Requirements

The only thing you need to run this server is JVM 1.6+. Please also note that if you run this on a Unix/Linux environment you will need root privileges to listen on the port 25

Keeping alive a SSH connection or in my case a multi-hop ssh tunnel to prevent timeout

A few months ago I wrote an article on how to create a multi-hop SSH tunnel between 3 machines

Since then I have been using this a lot for one of the projects I'm working on but I was annoyed that whenever the connection went idle for more than 2 minutes (like if I was reading an article or answering an email) the connection will be dropped abruptly and I will have to start it all over again...

After playing a bit with SSH options and reading ssh man I finally found my salvation, that is the -o flag with 2 options :

TCPKeepAlive: Specifies whether the system should send TCP keepalive messages to the other side. If they are sent, death of the connection or crash of one of the machines will be properly noticed. However, this means that connections will die if the route is down temporarily, and some people find it annoying. On the other hand,if TCP keepalives are not sent, sessions may hang indefinitely on he server, leaving host users and consuming server resources. The default is "yes" (to send TCP keepalive messages), and the server will notice if the network goes down or the client host crashes. This avoids infinitely hanging sessions. To disable TCP keepalive messages, the value should be set to "no".
ServerAliveInterval: Sets a timeout interval in seconds after which if no data has been received from the server, ssh will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server. This option applies to protocol version 2 only.

Below is the line of code that I use to create my multi-hop SSH tunnel and to prevent it from getting disconnected when idle :

ssh -o TCPKeepAlive=no -o ServerAliveInterval=15 -v -L38080:localhost:38080 ufasoli@host1 -t ssh -o TCPKeepAlive=no -o ServerAliveInterval=15 -v -L38080:localhost:38080 ufasoli@host2 -t ssh -o TCPKeepAlive=no -o ServerAliveInterval=15 -v -L38080:localhost:8080 ufasoli@host3

Here I'm disabling the TCPKeepAlive option on all the SSH chained connections and sending a message through the tunnel every 15 seconds to keep the data flowing

Please note that disabling the TCPKeepAlive option might be frowned upon by your IT Linux Guru / System administrator since as stated in the manual can keep alive dead connections (like if you forget to close your connection) so be careful when using these options and do not forget to properly close your SSH connection / tunnel. Also as I always say, I'm no Linux Guru so I'm not aware of the possible side-effects of this options so use them at your own risk !

MongoDB using MapReduce to filter fields by text content

Let's say that we want to filter a list of documents and extract only those in which the text of one of the fields matches a given constraint, for example a word that is repeated in the text

For this example I will be doing it using MongoDB's MapReduce


   {
      "_id" : "507f191e810c19729de860ea",
      "txtField" : "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus mauris arcu, lacinia a pharetra id, rhoncus in lorem. Morbi tempor consequat ante, vel porta tellus lacinia id. Phasellus vel nisi vitae velit pulvinar tincidunt non id massa. "
}

Below is the Map/Reduce functions that I will be applying to filter my results the results :

1.- Map Function

   
   function Map() {
 
     // count the number of times the regexp is matched in the text for the field txtField
     var count = this.txtField.match(/lorem/gi);

    // emit the result when the pattern has been matched at least 2 times
    if( count != null && count.length > 1){ 
       emit(this._id,count);
   }    
 
}

2.- Reduce Function


   function Reduce(key, values) { 
      return values[0];
   }

3.- Full query

  db.runCommand({ 
       mapreduce: "MY_DOCS", 
       map : function Map() {
            var count = this.txtField.match(/lorem/gi);
    
           if( count != null && count.length > 1){ 
             emit(this._id,count);
           }    
    
 
       },
      reduce : function Reduce(key, values) {
         return values[0];
      },
      query : { "txtField" : { "$exists" : true } },
      out : { inline : 1 }
  });

You can find additional info on MapReduce operations this nice article

Custom number formatting in JSP with JSTL for a given locale

So this one was pretty painful to resolve and I spent a few hours struggling with it.. especially since I haven't work with JSP for A WHILE...

Usually when you need to format a given number (in my case a BigDecimal) you can use the fmt:formatNumber tag if you have setted your locale properly it should use the appropriate decimal separator according to the language.

However this time it wasn't working and I'm really not sure why, other fields in which I was using the spring tag spring:bind the output was properly formatted

I needed to output a large number using grouping separators (by groups of 3) and at least 2 decimals

What I was trying to accomplish was to have a number with Swiss grouping and decimal separators like this 2'300'120.40 but I was getting this 2 300 120,40

I tried a lot of things, like setting the locale for the page using the setLocaletag

Forcing the pattern when calling the formatNumber function

And different combinations of values for the different attributes of the formatNumber tag, such as type="currency" nothing worked!! and the spring:bind just stood there gloating !!

It would have been great if the formatNumbertag would allow you to force the grouping and decimal character that would be used, but it's just not possible.

Luckily however this can be done in Java using the java.text.DecimalFormat ; and java.text.DecimalFormatSymbols

Since I was running out of time the workaround that I found was :

Create a Util class in Java with a static method that would do the formatting and conversion how I wanted it to be done
Declare this static method as a custom JSTL tag
Call this function with my number as argument to obtain the formatted String
Go enjoy a coffee :)

Below are the code snippets I used

The Java util method :


  public static String bigDecimalToFormattedString(BigDecimal bd)
  {
    String formattedString = null;
    try
    {
      DecimalFormatSymbols symbols = new DecimalFormatSymbols();
      symbols.setDecimalSeparator('.');
      symbols.setGroupingSeparator('\'');
      
      String strange = "###,###.##";
      
      DecimalFormat chCurrencyFormat = new DecimalFormat(strange, symbols);
      chCurrencyFormat.setGroupingSize(3);
      chCurrencyFormat.setMinimumFractionDigits(2);
      
      formattedString = chCurrencyFormat.format(bd);
    }
    catch (Exception e)
    {
     logger.error(String.format("Unable to convert BigDecimal : %s to String. error : %s", new Object[] { bd, e.getMessage() }));
    }
    return formattedString;
  }

You will need then to declare your custom functions in a .tld file that needs to be placed under the WEB-INF :




    Custom Functions    
    1.0
    http://java.sun.com/xml/ns/javaee:cf

    
      Converts a given BigDecimal to a formatted String with decimal and grouping separators
        bd2s
        com.ufasoli.StringUtils
        String bigDecimalToFormattedString(java.math.BigDecimal)

Once this is done you can use your custom functions by simply importing the taglib in a JSP file


<%@taglib prefix="cf" uri="http://java.sun.com/xml/ns/javaee:cf"  %>
.....

   

......

Now all of this can certainly be improved like by allowing the user to override the pattern, or the grouping separator, but in my case the output would be the same formatting throughout the application so I didn't need them. Regarding the hyper generic I know it's a really bad practice and should not be used but in this case we needed a quick and dirty solution that will not cause the app to fail if an exception occurred when formatting a given number