Google AppEngine translate bot

Google provided a few translation bots recently.
Since I do a lot of Dutch -> French translation, I quicly whipped up my own and deployed it to Google AppEngine:

For translating I used the unofficial google-api-translate-java jar file.
Here’s what you need to do:

Place the jar in war\WEB-INF\lib

war\WEB-INF\web.xml

<?xml version="1.0" encoding="utf-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
	<servlet>
		<servlet-name>Translate_Bot</servlet-name>
		<servlet-class>be.bartv.translatebot.Translate_BotServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>Translate_Bot</servlet-name>
		<url-pattern>/translate_bot</url-pattern>
	</servlet-mapping>
	<servlet>
		<servlet-name>xmppreceiver</servlet-name>
		<servlet-class>be.bartv.translatebot.XMPPReceiverServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>xmppreceiver</servlet-name>
		<url-pattern>/_ah/xmpp/message/chat/</url-pattern>
	</servlet-mapping>
	<welcome-file-list>
		<welcome-file>index.html</welcome-file>
	</welcome-file-list>
</web-app>

war\WEB-INF\appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
	<application>translate-bot</application>
	<version>1</version>
 
	<!-- Configure java.util.logging -->
	<system-properties>
		<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
	</system-properties>
	<inbound-services>
    	<service>xmpp_message</service>
  	</inbound-services>
 
</appengine-web-app>

src\be.bartv.translatebot.XMPPReceiverServlet.java

package be.bartv.translatebot;
 
import java.io.IOException;
 
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
import com.google.api.GoogleAPI;
import com.google.api.translate.Language;
import com.google.api.translate.Translate;
import com.google.appengine.api.xmpp.JID;
import com.google.appengine.api.xmpp.Message;
import com.google.appengine.api.xmpp.MessageBuilder;
import com.google.appengine.api.xmpp.XMPPService;
import com.google.appengine.api.xmpp.XMPPServiceFactory;
 
public class XMPPReceiverServlet extends HttpServlet{
	private static final long serialVersionUID = 2212159648921332999L;
	public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException
	{
		XMPPService service = XMPPServiceFactory.getXMPPService();
		Message message = service.parseMessage(req);
		JID jid = message.getFromJid();
		String content 	= message.getBody();
 
		String reply = "Hmmz. I Should return a translation now";
		try {
			GoogleAPI.setHttpReferrer("http://notes.bartv.be/");
			reply = Translate.execute(content, Language.DUTCH, Language.FRENCH);
		} catch (Exception e) {
			reply = "Error occurred: "+e.getMessage();
			e.printStackTrace();
		}
		service.sendMessage(new MessageBuilder().withBody(reply).withRecipientJids(jid).build());
	}
}

Sometimes, I do get a error saying I violate Google’s Terms and conditions. No idea why tho …

Force AJAX calls no-cache in Java. The clean way

Recently I discovered Internet Explorer caches some AJAX calls.

I was using jQuery to make some AJAX calls in a web-admin interface I’m building. I noticed none of the data changed as I tried to refresh (using an AJAX call). conclusion:: IE caches AJAX calls… very annoying.

You could go around and alter every method in your Struts/Spring/… application to force no-cache. But that would take some time. Instead, I wrote a Filter.

Hold on though, you don’t want every page to get the no-cache headers, that would seriously decrease your site performance (all pages would be force-reloaded instead of browser-cached). So we’ll only filter out AJAX calls.

Luckily, jQuery passes a header argument: X-Requested-With: XmlHttpRequest

X-Requested-With

The Code!

import java.io.IOException;
 
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
public class AjaxCacheFilter implements Filter{
 
	@Override
	public void destroy() {
	}
 
	@Override
	public void doFilter(ServletRequest request, ServletResponse response,FilterChain chain) throws IOException, ServletException {
		if ("XMLHttpRequest".equals(((HttpServletRequest) request).getHeader("x-requested-with"))) {
			((HttpServletResponse)response).setDateHeader("Expires", 0);
			((HttpServletResponse)response).addHeader("Cache-Control", "no-cache");
			((HttpServletResponse)response).addHeader("Pragma", "No-Cache");
 
		}
		chain.doFilter(request, response);
	}
 
	@Override
	public void init(FilterConfig arg0) throws ServletException {
	}
}

And add it as the last filter in your web.xml

<filter>
	<filter-name>ajaxCache</filter-name>
	<filter-class>
		com.yoursite.web.filters.AjaxCacheFilter
	</filter-class>
</filter>
<filter-mapping>
	<filter-name>ajaxCache</filter-name>
	<url-pattern>/*</url-pattern>
 	<dispatcher>FORWARD</dispatcher>
		<dispatcher>REQUEST</dispatcher>
</filter-mapping>

Hope it helps somebody! If you used it, let me know in comments. I’d love to know!

SOLR server with basic authentication in Spring

About SOLR

SOLR is a great search server, written under the Apache license.
This means 2 things: it’s free and supported by a great community of experts!

It’s important to know that SOLR consists of 2 parts:

• The solr server: a standalone Java application that handles data management and searching
• The solr web applicaton: provides web services and handles requests. Deploy this to Tomcat, or use the integrated Jetty

Also, solr provides a Java interface called SOLRj.
SOLRj provides 2 ways of connecting to your SOLR server:

• EmbeddedSolrServer connects directly to your SOLR server core and omits the web application
• CommonsHttpSolrServer connects to the SOLR web app

Now I hear you thinking: why would I go via the Web App when I can connect directly to the core?
The answer is simple: distributed servers. Imagine your web application is divided among multiple servers in order to balance the load, you can’t connect to the SOLRCore, unless you use multiple solr instances (which I don’t at the moment).

The Problem

So we’re using the SOLR web app on 1 server, and we use it’s webservices.
However, we don’t really want to leave it nice and open so any scriptkiddie can just connect to http://yourprettysolrserver.com/solr/admin/. So we use Basic HTTP authentication (I know, still not very secure. But the communication will only happen between the 2 virtual servers, so no chance of a Man in the Middle attack).
On notes how to secure your SOLR via basic auth, visit the SolrSecurity Wiki page.

The Solution

SolrJ uses Apache’s
So doing it in Java should go in a jiffy(as described here).

My solution was a bit quick, and maybe a bit dirty, but it works!
I made a subclass of HttpClient:

public class BasicAuthHttpClient extends HttpClient {
	public BasicAuthHttpClient()
	{
		super();
	}
	public BasicAuthHttpClient(String username, String pass)
	{
		super();
		Credentials creds = new UsernamePasswordCredentials(username,pass);
		this.getState().setCredentials(AuthScope.ANY, creds);
		this.getParams().setAuthenticationPreemptive(true);
	}
}

There’s a constructor that simply takes a username and password as argument.
In my ApplicationContext, I defined the following:

<bean id="solrHttpClient" class="com.example.system.utility.BasicAuthHttpClient">
	<constructor-arg value="${solrserver_user}" />
	<constructor-arg value="${solrserver_pass}" />
</bean>
<bean id="httpSolrServer" class="org.apache.solr.client.solrj.impl.CommonsHttpSolrServer">
	<constructor-arg value="${solrserver_path}" />
	<constructor-arg ref="solrHttpClient" />
</bean>

Works like a charm here! And should be thread safe.
Hope it helps somebody

SCJD URLyBird v1.1.1 Assignment: first notes

After successfully completing my SCJP certification (score of 81% thank you very much), it’s time to start my SCJD certification.

This certificate is achieved in two parts:

• An assignment which requires you to develop an application
• An essay you must write to prove you were the one who developed the application and not your supersmart neighbour

I recieved my assignment this week and was surprised to find how little descriptive it was. I was expecting at least 5 use-cases, a GUI design, 3 domain classes that are interlinked and connection to an Oracle database or something.
What I received however was a 6 page HTML  file that describes about what it should do, an interface you must implement and a .db file with serialized data.

Today I dove into reading that file extensively. The only thing I can say is: there isn’t one word too much or too little on that page.
I spent about an hour reading, underlining and taking notes and I recommend you to do the same if you’re going to take your SCJD.

Also, I found this great class written by Roberto Perillo that will surely help you along if you’re having trouble figuring out your database structure.

Development Notes

These are the notes I took on my first read.
I will most likely append to these notes as I develop the application along the way as this will server as my checklist for development.

Structure

The application consists of 3 parts as I see it:

• Client application
• Server application
• Database layer

The data is always retrieved from the database layer. However, the client application can either read and write directly to the database file, or do this via a server application
The server application must be able to allow multiple connections (hello multithreading).

User Interface

• The user interface must be built using solely Java Swing components
• The user interface must provide functionality to give a lists of all the bookings
• The user interface must provide functionality to update/add bookings
• The user interface must provide search functionality on all fields of the database
• Results of search/booking listings must be presented in a JTable
• Settings and preferences of the UI must be saved in a file called suncertify.properties

Database

The database is from a given file. The structure of this file must be adhered.

Structure

The structure of the .db file consists of 3 important parts:

• Information
  • 4 bytes: Magic cookie that identifies this file as being a data file of a certain type (there are different data files for different assignments)
  • 4 bytes that tell you how long each record will be
  • 2 bytes that tell you how many fields each record
• Structure, repeated for each field
  • 2 bytes that tell you the size of the fieldname (e.g. ’4′ for field ‘name’)
  • n bytes: field name (n being defined in the precious record)
  • 2 bytes: field length
• Data: structure defined by previous section

The overall structure is given in the assignment, for me it was:

• Hotel name: name(64)
• Hotel location: location(64)
• Room size: size(4)
• Is it a smoking room: room(1)   (valid values: Y/N)
• Price: rate(8)   (including currency. E.g $200.00)
• Date of booking: date(10)  (yyyy/mm/dd)
• Owner of the booking: owner(8)   (numeric ID for customer of null for still available)

Requirements

The database layer must provide :

• record locking. This does not mean file locking!
• Searching on all fields
• Listing all records
• Adding and updating records

Server

The server provides network access so multiple clients can work on the same database.

This server can work via Sockets or RMI. When using RMI, you must use JRMP.
I think I will use RMI as it looks like the cleanest solution.

Footnote

These are all my notes for now. I’ll worry about documentation and delivering when I acutally start developping.