Quantcast
RSS Entries RSS
RSS Subscribe by Email

Archive for January, 2011

Commons Math vs. ojAlgo

There are numerous math libraries for Java.   This is frustrating as a user because it’s hard to decide which to use.   Sometimes an algorithm is implemented in one library, but not another, which means you must marshal your data between proprietary formats.  I was working on solving systems of linear equations and there was no good Java-only solution available, so I had to write my own.  I decided to contribute the SimplexSolver to Commons Math because it used a friendly license and because it already had significant mindshare.  After doing so, I was informed that ojAlgo has a LinearSolver as well.  Today I decided to test it out to see whether I’d wasted my time by writing my own implementation.  It turns out that the ojAlgo implementation is buggy as shown by the unit test below which I created.

package com.benmccann.test;

import static org.ojalgo.constant.BigMath.EIGHT;
import static org.ojalgo.constant.BigMath.FIVE;
import static org.ojalgo.constant.BigMath.FOUR;
import static org.ojalgo.constant.BigMath.ONE;
import static org.ojalgo.constant.BigMath.SEVEN;
import static org.ojalgo.constant.BigMath.SIX;
import static org.ojalgo.constant.BigMath.TEN;
import static org.ojalgo.constant.BigMath.TENTH;
import static org.ojalgo.constant.BigMath.THREE;
import static org.ojalgo.constant.BigMath.TWO;
import static org.ojalgo.constant.BigMath.ZERO;

import java.math.BigDecimal;
import java.util.List;

import org.junit.Assert;
import org.junit.Test;
import org.ojalgo.matrix.BasicMatrix;
import org.ojalgo.matrix.store.PhysicalStore;
import org.ojalgo.matrix.store.PrimitiveDenseStore;
import org.ojalgo.optimisation.OptimisationSolver;
import org.ojalgo.optimisation.Variable;
import org.ojalgo.optimisation.OptimisationSolver.Result;
import org.ojalgo.optimisation.linear.LinearExpressionsModel;

public class SolverTest {

  @Test
  public void testMath286() {

    Variable[] objective = new Variable[] {
        new Variable("X1").weight(TENTH.multiply(EIGHT)),
        new Variable("X2").weight(TENTH.multiply(TWO)),
        new Variable("X3").weight(TENTH.multiply(SEVEN)),
        new Variable("X4").weight(TENTH.multiply(THREE)),
        new Variable("X5").weight(TENTH.multiply(SIX)),
        new Variable("X6").weight(TENTH.multiply(FOUR))};

    LinearExpressionsModel model = new LinearExpressionsModel(objective);
    model.setMaximisation(true);

    model.addWeightExpression("C1",
            new BigDecimal[] { ONE, ZERO, ONE, ZERO, ONE, ZERO }
        ).level(new BigDecimal(23));
    model.addWeightExpression("C2",
            new BigDecimal[] { ZERO, ONE, ZERO, ONE, ZERO, ONE }
        ).level(new BigDecimal(23));
    model.addWeightExpression("C3",
            new BigDecimal[] { ONE, ZERO, ZERO, ZERO, ZERO, ZERO }
        ).lower(TEN);
    model.addWeightExpression("C4",
            new BigDecimal[] { ZERO, ZERO, ONE, ZERO, ZERO, ZERO }
        ).lower(EIGHT);
    model.addWeightExpression("C5",
            new BigDecimal[] { ZERO, ZERO, ZERO, ZERO, ONE, ZERO }
        ).lower(FIVE);

    Result result = model.getDefaultSolver().solve();
    List solution = result.getSolution()
        .getRows(new int[] { 0, 1, 2, 3, 4, 5 })
        .toPrimitiveStore().asList();

    // A valid solution of 25.8 can be produced with:
    //     X1=10, X2=0, X3=8, X4=0, X5=5, X6=23
    // However, ojAlgo returns 21.7
    Assert.assertEquals(25.8, solution.get(0) * .8 + solution.get(1) * .2
        + solution.get(2) * .7 + solution.get(3) * .3
        + solution.get(4) * .6 + solution.get(5) * .4, .1);
  }

}

After releasing the Common Math SimplexSolver, I received numerous bug reports, which has been the main benefit of open sourcing it.  The code is now more robust as a result.  The test above was just one of many cases that Commons Math initially had trouble with, so I don’t fault the ojAlgo developers for getting it wrong the first time around – I did too.  However, I’m glad I chose to contribute to a well-recognized project because it led to flushing out many of these problems.  A lesser known project such as ojAlgo doesn’t have that advantage.  I’m not sure why so many people are still writing their own smaller libraries instead of contributing to make the larger players better.  Hopefully ojAlgo will consider recommitting its efforts towards Commons Math or another of the larger projects at some point.  It would be good for the community to see some consolidation.  At one point, matrix-toolkits-java was talking about combining with Commons Math, which would be a great move towards consolidated APIs.

Comments (7)

MySQL Commands Cheatsheet

Install SQL Buddy
sudo apt-get install apache2 libapache2-mod-php5 php5-mysql
sudo /etc/init.d/apache2 restart

Download SQL Buddy, unzip, and move to /var/www

Allow connections from a remote server:
  • Set the bind address in /etc/mysql/my.cnf to your ip address
  • Open the port in the firewall by running “/sbin/iptables -A INPUT -i eth0 -p tcp –destination-port 3306 -j ACCEPT” followed by “iptables-save”
  • Grant privileges on remote host as shown below
Login via MySql client:
$ mysql -u XXXXX -pXXXXX db_name
Create a new user:
CREATE USER 'user1'@'localhost' IDENTIFIED BY 'pass1';
GRANT ALL ON *.* TO 'user1'@'localhost';
CREATE USER 'user1'@'%' IDENTIFIED BY 'pass1';
GRANT ALL ON *.* TO 'user1'@'%';
List users:
select host, user, password, Create_priv from mysql.user;
List databases and tables:
show databases;
show tables;
Rename column:
ALTER TABLE Article change article_id id bigint(20);
Make column non-null:
ALTER TABLE Person MODIFY firstName varchar(255) NOT NULL;
Make column unique:
ALTER TABLE Person ADD UNIQUE INDEX(memberId);
Calculate the database size:
SELECT table_schema "DB Name", sum(data_length + index_length) / 1024 / 1024  "DB size in MB" FROM information_schema.TABLES GROUP BY table_schema;

Comments

Embedded Jetty

One of the coolest things about Jetty is the ability to run it in embedded mode.  This means that you can write a Java app with a main method that will launch a Jetty server.  This has some really nice benefits.  For example, it makes it really easy to deploy a server to EC2 because all you need to do is transfer your jar file there and hit run – no need to setup a servlet container, etc.  Also, it makes it possible to dynamically set servlet parameters such as Struts 2 devMode.  You can set a flag on the command line, parse it using a library such as JCommander, and then run in development mode depending upon what was requested by the flag passed in.

You first need to get the relevant jars included in your project as dependencies. All of the dependencies are available in Maven repositories. I use Gradle, so you may not be familiar with the syntax below, but you can take the parts between the colons and enter them as the groupId, artifactId, and version if you’re using Maven.

compile 'org.eclipse.jetty:jetty-io:8.0.0.M3'
compile 'org.eclipse.jetty:jetty-server:8.0.0.M3'
compile 'org.eclipse.jetty:jetty-servlet:8.0.0.M3'
compile 'org.eclipse.jetty:jetty-util:8.0.0.M3'
compile 'org.eclipse.jetty:jetty-webapp:8.0.0.M3'
compile 'org.mortbay.jetty:jsp-2.1-glassfish:2.1.v20100127'

Here’s an example of running a web app in Jetty’s embedded mode:

package com.benmccann.example.server;

import java.net.URL;

import org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.ErrorHandler;
import org.eclipse.jetty.server.nio.SelectChannelConnector;
import org.eclipse.jetty.servlet.ErrorPageErrorHandler;
import org.eclipse.jetty.servlet.FilterHolder;
import org.eclipse.jetty.servlet.FilterMapping;
import org.eclipse.jetty.servlet.ServletHandler;
import org.eclipse.jetty.webapp.WebAppContext;

import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.servlet.GuiceFilter;

/**
 * @author benmccann.com
 */
public class WebServer {

  private final Server server;

  public WebServer() {
    this.server = createNewServer();
  }

  private Server createNewServer() {
    Server server = new Server();

    SelectChannelConnector connector = new SelectChannelConnector();
    connector.setPort(8080);
    server.addConnector(connector);

    WebAppContext webApp = new WebAppContext(getBaseUrl(), "/");
    webApp.addEventListener(new GuiceListener());

    ServletHandler handler = createServletHandler();
    webApp.setServletHandler(handler);
    webApp.setErrorHandler(createErrorHandler());
    server.setHandler(webApp);

    return server;
  }

  private ErrorHandler createErrorHandler() {
    ErrorPageErrorHandler errorHandler = new ErrorPageErrorHandler();
    errorHandler.addErrorPage(500, "/error.html");
    return errorHandler;
  }

  private ServletHandler createServletHandler() {
    ServletHandler servletHandler = new ServletHandler();

    FilterHolder guiceFilterHolder = createGuiceFilterHolder();
    servletHandler.addFilter(guiceFilterHolder,
        createFilterMapping("/*", guiceFilterHolder));

    FilterHolder strutsFilterHolder = createStrutsFilterHolder();
    servletHandler.addFilter(strutsFilterHolder,
        createFilterMapping("/*", strutsFilterHolder));    

    return servletHandler;
  }

  private FilterHolder createGuiceFilterHolder() {
    FilterHolder filterHolder = new FilterHolder(GuiceFilter.class);
    filterHolder.setName("guice");
    return filterHolder;
  }

  private FilterHolder createStrutsFilterHolder() {
    FilterHolder filterHolder
        = new FilterHolder(StrutsPrepareAndExecuteFilter.class);
    filterHolder.setName("struts2");
    filterHolder.setInitParameter("struts.devMode", "true");
    return filterHolder;
  }

  private FilterMapping createFilterMapping(
      String pathSpec, FilterHolder filterHolder) {
    FilterMapping filterMapping = new FilterMapping();
    filterMapping.setPathSpec(pathSpec);
    filterMapping.setFilterName(filterHolder.getName());
    return filterMapping;
  }

  public void run() throws Exception {
    server.start();
    server.join();
  }

  private String getBaseUrl() {
    URL webInfUrl = WebServer.class.getClassLoader().getResource("WEB-INF");
    String webInfUrlString = webInfUrl.toExternalForm();
    return webInfUrlString.substring(0, webInfUrlString.lastIndexOf('/') + 1);
  }

  public static void main(String[] args) throws Exception {
    WebServer server = new WebServer();
    server.run();
  }

}

This web app used the Struts 2 filter and the Guice filter.  However, we specified them in code instead of in the web.xml.  We hardcoded struts.devMode to true, but you can easily see how we could make it true or false at runtime whereas that is impossible using a web.xml file.

You can also run Tomcat in embedded mode.

Comments (5)