Running 32bit applications on 64bit Ubuntu

After upgrading to 64bit Ubuntu 11.04, I wasn’t able to get some applications to work and was getting a very cryptic error

nullin@null-in:/work$ p4
bash: /build/bin/p4: No such file or directory

Even though /build/bin/p4 exists, on running the command, the system was complaining that the file doesn’t exist. I soon realized these were 32 bit applications and to fix this, one just needs to install ia32-libs:

sudo apt-get install ia32-libs

After doing this, things starting working properly again:

nullin@null-in:/work$ p4
Perforce -- the Fast Software Configuration Management System.
p4 is Perforce's client tool for the command line.  Try:
...
...

VMware Workstation 7.1.4, Ubuntu 11.04 and Unity

Three days back I installed Ubuntu 11.04 Beta onto my work machine. I tried getting Workstation to work but it kept crashing. In Unity session, it would start up and even start the VMs but would keep crashing on trying to switch to the VM console. In gnome session workstation would startup, switch to console session but crash while trying to move files in and out of the VM.

Version 7.1.3 has known issue with running in the new Unity session and v7.1.4 was supposed to fix this. Going by some comments online, it has seemed to fix it in many cases to. For me it just wasnt working. After looking around for a while I found a workaround on VMware community forums. For making VMware Workstation 7.1.4 with work with Ubuntu 11.04 in Unity Session, just start it from cli as follows:

$ export LD_PRELOAD=/usr/lib/vmware/lib/libglib-2.0.so.0/libglib-2.0.so.0
$ vmware

This fixes the glib related errors that were crashing the software and you can start using your VMs. See this forum thread for more information.

 

HemingwayEx v1.8 released

I played around a bit with WP 3.0 Menus and was able to incorporate it into HemingwayEx. Also, with this theme, users can decide to have a single column homepage instead of have the two columns.

Setting up WP Menus was quite straightforward actually, based on information in Nicholas’s article. Then there was some work needed to show drop down menus and for that I chose to use the Superfish JQuery Plugin. The site’s got everything you need to get the Superfish menus working. I just needed to make a few tweaks to the arrow images to change the color from white to black and had to fix the CSS.

One thing that caused me considerable amount of headache was invoking the superfish js using jQuery when document loads. Because other parts of the theme were also using jQuery and other JavaScripts, I had to use the JQuery NoConfict mode. I had to change the following:

 

to

And then everything worked perfectly. Hope you enjoy this release.

I have started hosting HemingwayEx code and downloads on Github now. v1.8 is available for download right now.

Logging tests to separate files

We have been using TestNG to run our tests for a while now. When we initially started, our code was written using Log4J and we were able to configure it to log all the tests to a single file. This made the logs very unwieldy and looking for issues very difficult. Looking at 100+ MB of logs isn’t an easy task. Changing this to rolling files only made the matters worse. So, I started looking for ways to log each test case into a separate file (meaning a test named foo, would be logged to foo.log).

My first attempt was to do this using log4j itself. Log4J doesn’t provide an easy way of doing this. It’s possible if you follow certain conventions in declaring your loggers and how you use MDC, but I wasn’t able to get it working with all my code. On looking a bit further, I found Logback. And SiftingAppender in Logback is just what I needed.

So, using slf4j, logback, MDC and a few simple coding conventions, I was able to get per test log files working. Here’s what you need to do:

  1. Start using SLF4J for your logging. If you are using Log4J, moving from L0g4J to SLF4J is pretty simple
  2. Write functions to set MDC at start of a test and unset it at the end of a test
  3. Update the test cases, such that the MDC set/unset functions are invoked
  4. Configure logback.xml to use the SiftingAppender

I’ll expand a bit more on the above.

Using SLF4J

Starting to use SLF4J or moving to SLF4J from other logging frameworks is pretty straightforward. I didn’t bother using the java application at http://www.slf4j.org/migrator.html and found it much easier to just to regex to do the job. There were some places where log.xxx(object) had to be changed to log.xxx(object.toString()) but that wasn’t a whole lot of pain.

Code to set/unset MDC

Logback’s documentation on MDC is very extensive and explains the concept quite clearly. Essentially using MDC would allow us to share a key/value across a thread hierarchy. Using MDC is as simple as putting a key/value into a Map. This value will be accessible anytime in the thread and any children of that thread. At the beginning of the test, we’ll put <"testname", $test_case_name> as the key/value into the map. At the end of the test, we’ll remove this key. This MDC value would be used by the sifting appender to create logs at runtime.  I wrote a simple class to encapsulate this functionality:

package com.nm.examples;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;

/**
 * Class to handle setting/removing MDC on per test case basis. This helps
 * us log each test case into it's own log file. Please see
 * {@link http://logback.qos.ch/manual/appenders.html#SiftingAppender}
 * and {@link http://logback.qos.ch/manual/mdc.html}
 * @author nullin
 */
public class TestLogHelper
{
  private static final Logger log = LoggerFactory.getLogger(TestLogHelper.class);
  public static final String TEST_NAME = "testname";

  /**
   * Adds the test name to MDC so that sift appender can use it and log the new
   * log events to a different file
   * @param name name of the new log file
   * @throws Exception
   */
  public static void startTestLogging(String name) throws Exception {
    MDC.put(TEST_NAME, name);
  }

  /**
   * Removes the key (log file name) from MDC
   * @return name of the log file, if one existed in MDC
   */
  public static String stopTestLogging() {
    String name = MDC.get(TEST_NAME);
    MDC.remove(TEST_NAME);
    return name;
  }
}

Update test cases

Now, you just need to make sure that you invoke TestLogHelper.startTestLogging(String testname) as early as possible during the test execution. Eventually, by the end of the test you should invoke TestLogHelper.stopTestLogging() to ensure that no extra logs get logged into this log file.

As we are using TestNG, for us it was a very simple matter of creating a method annotated @BeforeClass or @BeforeMethod as per requirements and put this code in there. For example, I use the following two methods in the base class for all our tests:

   @BeforeClass
   public void testSetUp() throws Exception {
      //start logging to test specific log file
      TestLogHelper.startTestLogging(getTestId());

      //
      //Do some setup specific stuff here
      //
   }

   @AfterClass
   public void testCleanUp() throws Exception {
      try {
         //
         //Do some cleanup specific stuff here
         //
      } finally {
         //stop test logging to test specific file
         TestLogHelper.stopTestLogging();
      }
      return true;
   }

   public String getTestId() {
      return ((testId == null) ? (this.getClass().getName()) : testId);
   }

Configure logback.xml with SiftingAppender

The last part is to configure the logback.xml to use a SiftingAppender. You can look at the documentation for SiftingAppender to see the different examples. Following is a snippet of the configuration that I am using:

   <appender name="RootSiftAppender" class="ch.qos.logback.classic.sift.SiftingAppender">
      <discriminator>
         <Key>testname</Key>
         <DefaultValue>testrun</DefaultValue>
      </discriminator>
      <sift>
         <appender name="FILE-${testname}" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <File>${testname}.log</File>
            <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
               <FileNamePattern>${testname}.%i.log</FileNamePattern>
               <MinIndex>1</MinIndex>
               <MaxIndex>100</MaxIndex>
            </rollingPolicy>
            <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
               <MaxFileSize>5MB</MaxFileSize>
            </triggeringPolicy>
            <layout class="ch.qos.logback.classic.PatternLayout">
               <Pattern>%d{ISO8601} %-5level %C{1} [%M:%L] [%thread] - %msg%n</Pattern>
            </layout>
         </appender>
      </sift>
   </appender>

This configuration says that the test cases would specify the MDC value using a key “testname” and if logback finds no such key, it’ll just log the statements into testrun.log.

And that’s it. You should be good to go.

HemingwayEx v1.7 released

I finally found some time yesterday to fix a bug and made HemingwayEx compatible with WP3.0. It’s been a long time since the last official release. A lot of work has gone into this theme between the last few revisions and most of it was done by db0. He spent a great amount of time enhancing the theme and adding support for many plugins.

I have moved the source of HemingwayEx to Github. It’s a great place to code socially because it makes it very easy for anyone to create a patch and submit it for addition to the main project.

Anyways, you can get the latest version of HemingwayEx at github.

Hudson CI for Testing

We are heavily using Hudson for testing a product we are working on. Other than maintenance and some instability issues, it’s been a great experience working with Hudson. We started out with just a single machine running Hudson server and then soon moved to a master-slave configuration where we have setup one master connected to four slaves. We are currently running about 10-15 jobs on this setup at one point or another during the day and use it to run pre-checkins (well in a way *post*-checkins), BATs and regression runs.

Here are a couple of screenshots that I took  when we were just getting started and were stabilizing Hudson and the product.

We have the following kinds of jobs running on this system:

  1. Pre-checkins: We provide development team with a set of tests that they are supposed to run before they commit any code. To ensure that the development team was doing this, we have a job that runs pre-checkins tests on every commit. This helps us ensure that things remain green though out the day and we come to know of failures asap.
  2. BATs: Basic Acceptance Tests as another category of tests that help us ascertain the daily quality of builds. Before the QA picks up a build for testing, we run the BATs against the daily official build and make sure that all tests pass.
  3. Regression runs: We also use hudson to run regression tests. At time of taking this screenshots, we had only about a 1000 tests and hence had a single job which would run all the tests and would take about 6-7 hours to complete. Now, we have split up that job into multiple jobs by using multiple build stages that Hudson supports. I will talk more about this in a later post.
  4. Helper jobs: We have parameterized jobs setup that help any one (dev or QA) to start any kind of test run against an official or sandbox build.
  5. CI jobs: This is how Hudson was originally intended to be used. For each branch that we work on, we poll the SCM every 5 minutes to check for checkins and build the dev and QA code. This helps us ensure that the code any checkin hasn’t broken the QA build and that users have access to the latest artifacts/distributions at all times.

For the project, we had invested time into providing infrastructure to pick up official/sandbox builds and then using these builds to setup environment, configure tests and then execute the tests. We just had to put in some more time to change or add features to make it all work easier with Hudson. Now, instead of letting everyone fend for themselves when it comes to setting up environments for testing and then reporting the results, we use Hudson to automate that process to a very large extent and make these tasks more manageable and the product quality more visible.

My Toyota Camry’s mileage

Tracked the mileage of my car for about 3 years. Most of my driving was city driving and about 15-25 miles a day. Took a few road trips as well, longest being to Las Vegas, Grand Canyon and back to bay area. Some of the numbers are missing and some are skewed because of getting different amounts of gas filled but the average mileage does come out to about 20-22 mpg.

Date Miles Gallons Amount Price Mileage
10/30/06 355 15.092 $35.00 $2.319 19.613
11/13/06 651 14.332 $34.10 $2.379 20.514
11/23/06 945 13.942 $34.56 $2.479 25.463
11/29/06 1,300 15.001 $36.14 $2.409 21.465
12/19/06 1,622 15.480 $41.01 $2.649 20.672
01/09/07 1,942 14.368 $38.35 $2.669 21.297
01/27/07 2,248 14.732 $37.55 $2.549 22.061
02/14/07 2,573 15.288 $42.79 $2.799 22.109
03/02/07 2,911 16.074 $48.05 $2.989 18.290
03/10/07 3,205 12.090 $38.68 $3.199 28.040
03/24/07 3,544 15.365 $49.15 $3.199 21.282
04/12/07 3,871 15.209 $50.78 $3.339 24.328
04/20/07 4,241 15.647 $52.71 $3.369 21.474
05/13/07 4,577 16.001 $55.51 $3.469 16.061
05/25/07 4,834 10.738 $36.93 $3.439
05/26/07 5,229 14.077 $47.85 $3.399 22.661
05/27/07 5,548 7.023 $25.14 $3.580
05/31/07 5,758 14.522 $49.65 $3.419 21.898
06/22/07 6,076 15.975 $52.70 $3.299 22.160
07/05/07 6,430 15.987 $50.18 $3.139 21.517
07/14/07 6,774 14.929 $47.61 $3.189 22.373
08/07/07 7,108 15.436 $46.91 $3.039 21.379
08/25/07 7,438 14.856 $42.18 $2.839 22.281
09/07/07 7,769 15.579 $45.01 $2.889 24.584
09/24/07 8,152 16.550 $49.47 $2.989 23.142
10/12/07 8,535 15.955 $49.76 $3.119 21.811
10/26/07 8,883 15.678 $50.47 $3.219 23.536
11/11/07 9,252 16.274 $55.64 $3.419 10.630
11/18/07 9,425 7.864 $27.36 $3.479 50.483
11/26/07 9,822 15.581 $53.89 $3.459 20.538
01/06/08 10,142 14.811 $50.34 $3.399 23.496
01/18/08 10,490 14.921 $49.08 $3.289 21.781
02/01/08 10,815 15.337 $48.45 $3.159 23.342
02/18/08 11,173 15.779 $51.74 $3.279 21.801
03/03/08 11,517 15.932 $56.38 $3.539 22.219
03/13/08 11,871 15.953 $58.05 $3.639 22.692
03/29/08 12,233 5.497 $20.00 $3.638 29.471
04/01/08 12,395 15.965 $58.74 $3.679 22.863
04/19/08 12,760 15.558 $60.04 $3.859 24.296
05/04/08 13,138 15.697 $61.67 $3.929
05/22/08 15.948 $65.06 $4.080
06/08/08 13,810 10.372 $45.00 $4.339 35.866
06/21/08 14,182 16.487 $75.00 $4.549 22.442
07/07/08 14,552 15.003 $67.95 $4.529 26.728
07/26/08 14,953 15.224 $66.77 $4.386 23.975
08/15/08 15,318 15.094 $62.78 $4.159 23.122
08/27/08 15,667 14.551 $57.90 $3.979 25.222
09/15/08 16,034 15.227 $59.07 $3.879 23.248
10/01/08 16,388 14.544 $54.38 $3.739 24.684
10/17/08 16,747 15.093 $52.21 $3.459 27.430
10/29/08 17,161 7.671 $25.00 $3.259 27.115
11/03/08 17,369 16.543 $48.95 $2.959 21.943
11/23/08 17,732 15.059 $32.51 $2.159 24.172
01/12/09 18,096 15.338 $31.22 $2.035
01/25/09 16.237 $34.08 $2.099
02/13/09 14.644 $33.08 $2.259
03/03/09 12.829 $28.47 $2.219
03/21/09 14.982 $32.05 $2.139
04/13/09 19,889 15.671 $36.03 $2.299 24.759
05/07/09 20,277 14.845 $36.06 $2.429 19.333
05/22/09 20,564 11.333 $29.91 $2.639 18.265
05/22/09 20,771 6.846 $18.20 $2.658 51.855
05/23/09 21,126 12.561 $30.64 $2.439 14.887
05/24/09 21,313 6.978 $18.83 $2.698 61.479
05/25/09 21,742 14.330 $38.82 $2.709 24.703
05/25/09 22,096 12.850 $35.71 $2.779 32.840
06/11/09 22,518 15.301 $46.19 $3.019
07/04/09 14.119 $42.34 $2.999
07/20/09 23,303 14.826 $42.09 $2.839
08/16/09 15.099 $46.79 $3.099
09/24/09 15.670 $50.13 $3.199
10/11/09 14.757 $46.62 $3.159
11/12/09 14.591 $43.76 $2.999
12/10/09 15.355 $44.82 $2.919
02/28/10 14.865 $44.58 $2.999

Here are some graphs from OpenOffice to visualize this information.

Hopefully this information is helpful to someone.

Running single test or class using TestNG and Ant

I have been using TestNG for a while now in the latest project that I am working on. We have developed a whole suite of tests for an application and TestNG has really served us well.

When it comes to running tests, we find that people (developer, testers and others) usually are hesitant to use something that involves much learning or is different from how they are used to doing things currently. So, we found it very useful to provide utilities that made it easy for anyone to be able to pick up test artifacts and with-in a few steps be able to run tests to reproduce bugs. Because we are building our project using ant, it was easy to provide a build.xml with the test distribution that will able to run the TestNG tests. This keeps things simple and uniform.

The TestNG ant task uses TestNG suites defined in XML files to run tests. It doesn’t provide an easy way to run a single test method or all the test methods in a single class. Basically, anytime someone has to run a test using this task, they need to create an XML file specifying the class and method for that test in an XML file and then execute the ant task. This can be simplified by letting ant targets take care of setting up the XML file with the require test and then executing it.

It’s quite simple. You just need to create a TestNG suite XML template, modify it with the parameters that the user passes in on command line and then execute the TestNG ant task. Here’s how this can be achieved. First, create a template XML file like the following at ${testng.templates}/testfn.xml:

< !DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

   
      
         
            
               
               
            
         
      
   

Now, you can add the following target to the build file:


   
      
         
         
      
   
   
   
      
         
         
      
   
   
      
      
   

The target picks the testfn.xml file, replaces the tokens with the specified input and copies it to the temp location. Now, the TestNG task can use this updated XML file to run tests. This target is invoked as:

ant run-single-test -Dclass.name=com.nalinmakar.testng.ant.Demo -Dtest.name=Test1

for the following class:

package com.nalinmakar.testng.ant;
public class Demo
{
   @Test
   public void Test1()
   {
      //do something
   }
   @Test
   public void Test2()
   {
      //do something
   }
}

Similarly, you can also create a template and another ant target for running all the test methods in a class:

< !DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

   
      
         
       
   


   
      
   
   
   
      
         
      
   
   
      
      
   

and can invoke this as

ant run-single-test -Dclass.name=com.nalinmakar.testng.ant.Demo

Fix “500 Internal Error” in WordPress

I recently upgraded my WordPress installation to the latest version (2.8.6) and started running into issues with uploading images. Every time I tried using the Flash Uploader I would see

HTTP Error.

On trying to use the non-flash browser based uploader, I saw the following:

Error 500 – Internal server error
An internal server error has occured!
Please try again later.

I googled quite a bit and finally found a solution. This error can happen because of quite a few different reasons:

  1. Incompatible plugins
  2. Incorrect .htaccess file contents
  3. PHP running out of memory
  4. Incorrect file permissions
  5. Not using PHP5

Incompatible plugins

Easiest way to find if this is the issue is to deactivate all plug-ins and see if the issue still persists. If it goes away you can activate each plug-in one by one to find the culprit. You can read more on managing plug-ins here.

Incorrect .htaccess file contents

For some people, the issue went away after fixing the .htaccess file contents. You need to ensure that the .htaccess file at root of your website has the following contents:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

You can read more about it here.

PHP running out of Memory

This can happen if you are using some memory hungry plug-ins. You can fix this issue by adding a php.ini file in <wordpresss_root>/wp-admin folder with the following contents:

memory=20MB

You can read more about this at codedifferent.com, where this is explained in more details.

Incorrect file permissions

WordPress can start acting up if the permissions on directories and files in WordPress installation aren’t set up properly. You can ensure that this is fixed by logging into your hosting provider and making sure all files and directories have permissions set to 755.

Another issue that might lead to errors while uploading is certain security issues with files used for uploading. This can be fixed by using Image Upload HTTP Error Fix plug-in. Otherwise, you could edit .htaccess to add the following lines:

#BEGIN Image Upload HTTP Error Fix
<IfModule mod_security.c>
<Files async-upload.php>
SecFilterEngine Off
SecFilterScanPOST Off
</Files>
</IfModule>
<IfModule security_module>
<Files async-upload.php>
SecFilterEngine Off
SecFilterScanPOST Off
</Files>
</IfModule>
<IfModule security2_module>
<Files async-upload.php>
SecFilterEngine Off
SecFilterScanPOST Off
</Files>
</IfModule>
#END Image Upload HTTP Error Fix

Not using PHP5

Even after doing all the above, things weren’t working for me. That is when I came across this post, which explains that the issue could be because PHP5 wasn’t enabled. According to a help article on 1and1.com, fix is to just add the following line in .htaccess file:

#enable php5
AddType x-mapp-php5 .php

This fixed the issue for me!

Hope this helps.