Was mich bewegt Meine Meinung zu Verschiedenem

Roolastic - Part Four

Part One

Part Two

Part Three

Since the last part I've added Spring security to Roolastic (it's not really used yet) but it helped me understand a bit more about Roo. So while it is still early days, I'd like to come up with some general notes and opinions on Roo and ElasticSearch.

Notes

Database

I've chosen to use H2 as it comes with a handy little web console. If you start that (as I do here) you can then point your browser to http://localhost:8082/and have a look at and modify what's in your db - neat!

IDE

I've been using STS and it's nice enough even though a bit lacking in the responsiveness department. If you use 'plain' Eclipse make sure you have a least m2eclipse installed (and you probably want to have the AspectJ plugin as well). Also if you use any of those - do yourself a favor and rightclick on the 'target' folder in Navigator view and check the Derived box in Properties to prevent files in target from popping up when you do an 'Open Resource...'

Serialization

I've been using Jackson to serialise my entities to JSON which I then feed into ElasticSearch and I really like it. What's a bit annoying is that Jackson serialises some fields named 'ajc$interField$...' which AspectJ is responsible for and which are not needed really. The only way to turn that off that I found was to specify every single of these fields in the @JsonIgnoreProperties annotation which is a bit painful - some pattern matching would be helpful here

Opinion

Spring Roo

What I like

• You must love the speed with which you can setup a fully functional webapp and start toying with your ideas. And once you accept the AspectJ stuff it is actually all fairly transparent
• The scaffolding is cool. I think it was one of the most envied feature of Ruby on Rails in the Java community. While I think it isn't really that important in a project in the long run (because you will deviate from the generated stuff) it is really useful at the beginning of a new project.
• I really like the whole 'shell/console' idea. It comes natural (to me at least) and feels very lightweight
• I know that Maven had to take quite a lot of criticism recently, but I like it and I like the fact that it standardises the project layout and so I'm quite happy that Roo makes use of Maven as well (even though there is talk of switching to ivy)
• I'm also very happy with Tiles being used in the view layer. I think it's quite a clean and clever way to keep your views mean and lean

What I'm not sure about

• My main gripe is with JSP actually. I never really got into JSPs and have always preferred Velocity and Freemarker - so getting Freemarker as an alternative view technology would be perfection. I'm also at odds with the way javascript is woven into the scaffolded views. The fact that the script tags get sprinkled all across the file rather than in one place in some sort of document.ready listener is not to my liking (even though I can imagine the generation would get more complicated that way)
• I also have the feeling that the aspectJ weaving adds a couple of seconds to the app startup time (having embedded database, JMS and ElasticSearch doesn't help there either) and a couple of seconds for every restart quickly add up over a days work - but I guess that's the price you pay (and it can serve as an excuse to get a new MacBook)

Overall Roo is growing on me quickly and I think it will become a permanent part of my toolbox. It gives you part of that RAD feeling that Rails and Grails are all about while letting you stay in beloved Java land and also feeling more transparent (to me at least - I was frightened by Grails' stacktraces)

ElasticSearch

While I haven't spent as many words on ES as on Roo, I really think ES rocks! Even though it's such a new project, it already feels really very useful. The flexibility it gives you is great, the effortless clustering is a boon and the JSON driven interface makes it easy to intergrate with almost anything (modern) you care to think of. The fact that you don't need to define a schema really makes it so easy to get started with (unlike Solr - which I never the less also really like). And it's getting new features almost on a daily basis. I will keep an eye on it and hopefully find the time to advance my little project over the coming months while blogging about my findings. (I especially need to research the facetting capabilities)

Posted at 05:06PM Mrz 25, 2010 by joerg in Allgemein  |  Kommentare[0]

Roolastic - Part Three

Part One

Part Two

Part Four

Now that we have our little webapp running and we can add images to it and they get indexed, we need a way to query the index.
So we quickly create a SearchController by running

controller class --class ~.web.SearchController

1. I put a JSON expression into the search field to give you a head start
2. I implemented two different controller methods linked to the two different search buttons. One just stuffs the json coming back from elasticsearch into the model and one deserialises the returned results back into entity instances and stuffs those into the model

Posted at 01:35PM Mrz 25, 2010 by joerg in Allgemein  |  Kommentare[0]

Roolastic - Part Two

Part One

Part Three

Part Four

Now that we have the basics running let's have a look at some interesting bits of code (have a look at the roo.log to see how I build what is there so far using roo). Now the only domain object we have so far is Image. This is the object that holds the data we want to persist for each uploaded image and it's simple enough to understand. The two interesting properties of this class' source are that there are no bean style getters and other boilerplate code, as Roo has generated that and 'outsourced' it into AspectJ .aj files (like this). That is one of Roos main design decisions. The other notable thing here is the @JsonIgnoreProperties annotation - but we get to that later
CRUD operation for this object are handled by the ImageController which I created by first running

controller scaffold --class ~.web.ImageController --entity ~.model.Image

<property name="hibernate.ejb.interceptor" value="de.woerd.blogs.roolastic.persistence.HibernateInterceptor"/>

jms setup --provider ACTIVEMQ_IN_MEMORY

jms listener class --class ~.search.IndexerSlave

Posted at 05:31PM Mrz 24, 2010 by joerg in Allgemein  |  Kommentare[5]

Roolastic - Part One

Part Two

Part Three

Part Four

Roolastic is a little research project by me. It is a project using Spring Roo and integrating ElasticSearch for full text search. It is motivated by my both in web frameworks and in everything that is happening in the full text search and especially in the Lucene world. While it is interesting to read about emerging technologies, nothing beats hands-on experience with a framework or project, so I came up with Roolastic. I hope it will serve me as a playground for both Roo experiments and getting to grips with ElasticSearch.

Objectives

A webapp that lets users upload images and search for images by their user provided properties and extracted metadata. This is admittedly rather primitive but I think it's useful enough to try out quite a lot of things and get a feeling for the quality, usefulness and 'taste' of the tools

Prerequisites

You need

• a recent version of Java obviously
• a working version of maven
• a Roo Installation or Spring Tools Suite (which includes Roo)
• ElasticSearch

Get up and running

It would probably be useful to read up on Roo as I don't intend to make this yet another Roo tutorial - my aim is more to review it.
You should then get a recent copy of ElasticSearch (I'm working with the latest development sources, checking them out with git). Once you have got the sources, did your first successful build with gradle and verified that it's working, do a

./gradlew elasticsearch:install

cd 'directory where your elasticsearch sources are'
cd build/distributions/exploded
bin/elasticsearch -f

mvn compile
mvn tomcat:run (OR) mvn jetty:run

Posted at 04:31PM Mrz 24, 2010 by joerg in Allgemein  |  Kommentare[2]

ElasticSearch

Shay Banon (@kimchy) of Compass fame has done it again and created a cool search project: ElasticSearch. It's a server component on top of Lucene which you interact with via JSON/REST. Unlike Solr it is schema less and focuses on the easiness of transparent multi node setups. It will be interesting to see how it compares to Solr in terms of search features like facetting and range searches. Here is what Shay has to say on the ElasticSearch vs. Solr topic. It's still early days but I will keep an eye on it.

Posted at 10:25AM Feb 16, 2010 by joerg in Allgemein  |  Kommentare[0]

Some git resources

I've been using git a bit more recently and have stumbled across a couple of nice tips and tools.

Installation

I use macports to get a recent version of git on my mac

Setup

Well you should at least specify your name and email on a global level, like so

$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

git config --global github.user username
git config --global github.token xyzxyzxyzxyzxyzxyzxyzxyz

Documentation

There are at least 3 online books

• http://learn.github.com/
• http://book.git-scm.com/
• http://progit.org/book/

(GUI-)Tools

If you installed git with macports you might as well install tig, which is quite a nice CLI git browser. I also like gitX and there is SmartGit, which is also free for non-commercial use. When using git you obviously want to have a look at github; if you are not developing open source projects trac might be a 'poor mans github alternative' with the git plugin. I also really quite like the Git Bundle for TextMate. EGit/JGit are clearly worth having a look out for for those of you, who earn there money in Java land. You can find information on how to setup FileMerge.app for diffing and merging here and here

Posted at 05:23PM Feb 09, 2010 by joerg in Allgemein  |  Kommentare[0]

why I like groovy

I use groovy more and more often. See this little gem for transforming the charset of a batch of textfiles. Couldn't get a lot simpler. It's basically 6 lines of code:

if(args.length != 4) {
	println "Usage: specify these arguments: input dir, input encoding, output dir, output encoding"
	return
}

def inputDir = new File(args[0])
def inputEncoding = args[1]
def outputDir = new File(args[2])
def outputEncoding = args[3]

assert inputDir.exists(), "${inputDir} must exist"
assert inputDir.canRead(), "${inputDir} must be readable"
assert outputDir.exists(), "${outputDir} must exist"
assert outputDir.canWrite(), "${outputDir} must be writable"

inputDir.eachFileMatch(~/.+\.txt/) { file ->
	def reader = file.newReader(inputEncoding)
	
	def outputFile = new File(outputDir, file.name)
	outputFile.withWriter(outputEncoding) { writer ->
		writer << reader
	}
}

Posted at 04:51PM Feb 09, 2010 by joerg in Allgemein  |  Kommentare[0]

A sample of Spring AOP usage

In the last couple of years I've come across this problem several times: you've created a business application for a client with various business / domain objects, all is well and then the client decides they need to somehow be able to group various of those objects together and then they need to be able to set global filters to make sure you only see and work with objects belonging to a certain grouping. Now there is two problems here:

• The grouping isn't restricted to one of your domain objects. You need to be able to throw all your various domain objects into those groups, i.e you can create a group that consists of people and things and emails and boxes and …
  Unfortunately that isn't a trivial thing to model in your classical rdbms/hibernate persistence application. Relational databases aren't particularly well suited to store relations from one thing (table) to all kinds of other things (tables) while still keeping it easy to query those relationships and also maintaining relational integrity.
  Well, I haven't really solved this problem. I recently implemented a simple solution that is good enough for my project albeit not very elegant I think - if someone has solved this in a better way, please let me know or point me to a good resource. Anyway I have solved it by creating link objects for each thing that needs to be grouped. The link objects form a class hierarchy that ist mapped onto a single table. That allows me to quite easily query those relationships at the cost of the link table having to have quite a few nullable columns - which C.J.Date wouldn't approve of I guess
• The second problem is that you then depending on some contextual information in the user session of your application you need to filter all listings of your domain objects. You could of course have some if or switch statements in your controller logic but that smells. It also means that you need to touch potentially lots and lots of places in your code if anything with your grouping changes.

<aop:aspectj-autoproxy />

…

import java.lang.reflect.Method;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.CodeSignature;


@Aspect
public class ScopeAspect {
	final static Logger logger = LoggerFactory.getLogger(ScopeAspect.class);
	
	@Pointcut("within(dao..*)") 
	public void inDataAccessLayer() {}
	
	@Pointcut("execution(public * scoped*(..))")
	public void scoped(){}
	
	@Around(value = "inDataAccessLayer() && scoped() && target(target)")
	public Object addScopeIfNeeded(ProceedingJoinPoint joinPoint, GenericDao target) throws Throwable
	{
		Scope scope = ScopeUtils.getCurrentScope();
		if(scope != null) {
			logger.debug("rewriting dao method to use scope!");
			
			// gather the argument values and types
			Object[] originalArgs = joinPoint.getArgs();
			Object[] modifiedArgs = new Object[originalArgs.length + 1];
			modifiedArgs[0] = scope;
			
			// prepend the 'scope' arg
			System.arraycopy(originalArgs, 0, modifiedArgs, 1, originalArgs.length);
			
			// calculate the new argument type list
			Class[] modifiedArgTypes = new Class[modifiedArgs.length];
			modifiedArgTypes[0] = Project.class;
			Class[] originalArgTypes = ((CodeSignature)joinPoint.getSignature()).getParameterTypes();
			System.arraycopy(originalArgTypes, 0, modifiedArgTypes, 1, originalArgTypes.length);
			
			// find the method that has the same signature apart from the preceding scope type argument
			Method m = target.getClass().getMethod(joinPoint.getSignature().getName(), modifiedArgTypes);
			
			return m.invoke(target, modifiedArgs);
		}
		else
		{
			return joinPoint.proceed();
		}
		
		
	}
	
}

• It really is an aspect of my app and therefore it is nice that this lives in exactly one place.
• It means my I can achive the deired effect by convention rather than configuration: I just need to code multiple methods with the same name in my DAOs
• It also means that the dao methods still fully encapsulate the query specifics and if different entities have different requirements for the queries used to list them by scope that is fine
• Since only certain methods of my DAO's are affected I can still have lots of methods left that won't ever get touched by the aspect and so I can have getAll() or list() or whatever methods that I might need for an admin backend.

• Well if you forget about the aspect and forget to code the partner 'scoped' method in a dao you'll get a NoSuchMethodException
• I doubt this, but it could be, that it affects performance
• You still have to remember using your 'scoped' methods in your controller
• It might actually drive you nuts if you come back to this code in 2 years time, having forgotten about this aop business, and put a breakpoint in the method called inyour controller only to wait endlessly for that method to ever be called …

Posted at 01:53PM Nov 11, 2009 by joerg in Allgemein  |  Kommentare[0]

trouble with spring and maven assembly plugin

I've built a couple of small java tools recently that do some background processing and get run from cron. These tools are too complex for a maintainable groovy or python script (at least for my taste) and I therefore implement them in Java using spring for its wonderful JDBC template, database connection handling and some other niceties. Now to make for a simple deploy I pack them up using mavens assembly plugin and here is where the fun starts. If you do this you will get an error like this:

Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace

<assembly>
  <id>my-jar-with-dependencies</id>
  <formats>
    <format>jar</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <dependencySets>
    <dependencySet>
      <unpack>true</unpack>
      <scope>runtime</scope>
      <unpackOptions>
	      <excludes>
	      	<exclude>**/spring.handlers</exclude>
	      	<exclude>**/spring.schemas</exclude>
	      </excludes>
      </unpackOptions>
    </dependencySet>
  </dependencySets>
  <fileSets>
    <fileSet>
      <directory>${project.build.outputDirectory}</directory>
    </fileSet>
    <fileSet>
		<directory>target/classes</directory>
		<outputDirectory>META-INF</outputDirectory>
		<includes>
			<include>spring.handlers</include>
			<include>spring.schemas</include>
		</includes>
	</fileSet>
  </fileSets>
</assembly>

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <configuration>
    <descriptors>
      <descriptor>src/main/assembly/my-jar-with-dependencies.xml</descriptor>
     </descriptors>
     <archive>
       <manifest>
         <mainClass>my.main.Class</mainClass>
       </manifest>
     </archive>
  </configuration>
</plugin>

Posted at 03:31PM Nov 10, 2009 by joerg in Allgemein  |  Kommentare[0]

Atomkraft - nein danke!

Seit Donnerstag sind wir nun Stromerzeuger (pünktlich zum Ende des Sommers :-( ). Eine 5.6kWp Anlage aus 28 Modulen auf unserem Ost-West ausgerichteten Dach erzeugt bei dem momentan vorherrschenden freundlichen Spätsommerwetter zwischen 12 und 20 kWh Strom. Und wenn der Errichter richtig gerechnet hat sollten wir im Jahr ca. 1500 kWh mehr erzeugen als wir selber benötigen.

Posted at 11:08AM Sep 21, 2009 by joerg in Allgemein  |  Kommentare[0]

Image upload from the clipboard

A client recently asked me whether it was possible to modify a webapp that we've been building for them to allow for somehow uploading graphics from the clipboard. I hesitated and said that it wasn't possible with HTML / Javascript but then did a bit of research. It quickly became clear that one would need to use some Java applet / ActiveX control (do these still exist?) or Flash app to achieve the effect. So I set out to create something.

I am a decent backend Java programmer but I've never much liked the GUI side of Java and applets are GUI to some extent, so I searched the web and nicked ideas all over the place and this is what I came up with:

	package de.woerd.applet;

	import de.woerd.io.MultiPartFormOutputStream;
	import java.awt.Color;
	import java.awt.Image;
	import java.awt.Toolkit;
	import java.awt.datatransfer.Clipboard;
	import java.awt.datatransfer.DataFlavor;
	import java.awt.datatransfer.Transferable;
	import java.awt.image.BufferedImage;
	import java.awt.image.ImageObserver;
	import java.awt.image.RenderedImage;
	import java.io.BufferedReader;
	import java.io.ByteArrayOutputStream;
	import java.io.IOException;
	import java.io.InputStreamReader;
	import java.net.URL;
	import java.net.URLConnection;
	import javax.imageio.ImageIO;
	import javax.swing.ImageIcon;
	import javax.swing.JApplet;
	import javax.swing.JLabel;

	/**
	 *
	 * @author joerg
	 */
	public class ImagePaster extends JApplet {

	    Clipboard clipboard;
	    Toolkit toolkit;
	    JLabel status;


	    @Override
	    public void init() {
	        super.init();

	        toolkit = Toolkit.getDefaultToolkit();
	        clipboard = toolkit.getSystemClipboard();


	    }

	    /**
	     *
	     * @param targetUri - relative to location of page this applet is embedded in
	     * @param format format of resulting image bytes, currently only support 'jpeg' or 'png'
	     * @return
	     */
	    public AppletResult pasteImage(String targetUrl, String format) {

	        //get image data from clipboard
	        Image image = getImageFromClipboard();

	        if(image == null)
	            return new AppletResult(false, "Kein Bild!", null);

	        if(!("jpeg".equals(format) || "png".equals(format)))
	            return new AppletResult(false, "Format nicht unterstützt. Bitte entweder 'jpeg' oder 'png' wählen", null);

	        // create a byte stream of that image, encoded in the correct image format
	        ByteArrayOutputStream imageBytes = null;
	        try{
	            imageBytes = getImageBytes(image, format);
	        }
	        catch(IOException e)
	        {
	            return new AppletResult(false, "Bild konnte nicht gelesen werden", null);
	        }

	        // upload bytes to targetUrl as
	        String result = null;
	        try{
	            result = uploadImage(targetUrl, imageBytes, format);
	        }
	        catch(Exception e)
	        {
	            return new AppletResult(false, "Bildaten konnten nicht heraufgeladen werden", null);
	        }

	        return new AppletResult(true, null, result);
	    }


	    private Image getImageFromClipboard() {
	        Transferable transferable = clipboard.getContents(null);
	        if(!transferable.isDataFlavorSupported(DataFlavor.imageFlavor))
	            return null;
	        try {
	            Image img = (Image) clipboard.getContents(null).getTransferData(DataFlavor.imageFlavor);

	            BufferedImage newImg = null;
	            int w = img.getWidth(null);
	            int h = img.getHeight(null);
	            newImg = new BufferedImage(w,h,BufferedImage.TYPE_INT_RGB);


	            ImageIcon ii = new ImageIcon(img);
	            ImageObserver is = ii.getImageObserver();

	            newImg.getGraphics().setColor(new Color(255, 255, 255));
	            newImg.getGraphics().fillRect(0, 0, w, h);
	            newImg.getGraphics().drawImage(ii.getImage(), 0, 0, is);

	            return newImg;
	        } catch (Exception e) {
	            return null;
	        }
	    }

	    private ByteArrayOutputStream getImageBytes(Image image, String format) throws IOException {
	        ByteArrayOutputStream baos = new ByteArrayOutputStream();
	        if(image instanceof RenderedImage)
	        {
	            ImageIO.write((RenderedImage)image, format, baos);
	        }

	        if(baos.size() == 0)
	            throw new IOException("No image data found");

	        return baos;
	    }

	    private String uploadImage(String targetUrl, ByteArrayOutputStream imageBytes, String format) throws Exception  {

	        URL url = new URL(getDocumentBase(), targetUrl);
	        // create a boundary string
	        String boundary = MultiPartFormOutputStream.createBoundary();
	        URLConnection urlConn = MultiPartFormOutputStream.createConnection(url);

	        urlConn.setRequestProperty("Accept", "*/*");
	        urlConn.setRequestProperty("Content-Type", MultiPartFormOutputStream.getContentType(boundary));

	        // set some other request headers...
	        urlConn.setRequestProperty("Connection", "Keep-Alive");
	        urlConn.setRequestProperty("Cache-Control", "no-cache");

	        // no need to connect because getOutputStream() does it
	        MultiPartFormOutputStream out = new MultiPartFormOutputStream(urlConn.getOutputStream(), boundary);

	        // write bytes 
	        out.writeFile("cbUpload", "image/" + format, "clipboardImageUpload." + format, imageBytes.toByteArray());
	        out.close();

	        // read response from server
	        StringBuffer buf = new StringBuffer();
	        BufferedReader in = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));

	        String line = "";
	        while ((line = in.readLine()) != null) {
	            buf.append(line);
	        }
	        in.close();
	        return buf.toString();
	    }



	}
	
	package de.woerd.applet;

	public class AppletResult {
		private boolean success;
		private String message;
		private String result;

		public AppletResult(boolean success, String message, String result) {
			this.success = success;
			this.message = message;
			this.result = result;
		}

		public boolean isSuccess() {
			return this.success;
		}

		public void setSuccess(boolean success) {
			this.success = success;
		}

		public String getMessage() {
			return this.message;
		}

		public void setMessage(String message) {
			this.message = message;
		}

		public String getResult() {
			return this.result;
		}

		public void setResult(String result) {
			this.result = message;
		}
	}

Now for this to work as an applet you also need the accompanying class MultiPartFormOutputStream to do the actual multipart posting which I nicked as well and which you can download here and the applet needs to be signed.

I used Netbeans 6.7 following these instructions.

You can then embed the applet into an html page like this:

	<body>
		<script type="text/javascript">
			function upload() {
				obj = document.getElementById('paste-image');
				result = obj.pasteImage('/uploadscript', "jpeg");
				
				if(result.isSuccess()) {
					// do something with result.getResult()
					// result.getResult() will contain whatever the uploadscript returns. So it'd make some sense to make it return the URI of the new upload ;-)
				}
				else { // error
					// display result.getMessage() in some way
				}
			}
		</script>

		<object id="paste-image" classid="java:de/woerd/applet/ImagePaster.class" type="application/x-java-applet" archive="/path/to/signed/jarfile.jar" />" width="1" height="1"></object>
		
		<input type="button" value="Paste it!" onclick="upload();">
		
	</body>

When you then copy an image to the clipboard and press the 'Paste it!' button, the image content is obtained from the clipboard, transformed into a jpeg or png and uploaded to your server script.

Notes

• What I don't really understand is what happens in the 'getImageFromClipboard()' method, which - you guessed it - is also nicked. But if I take the image that I obtain with clipboard.getContents(null).getTransferData(DataFlavor.imageFlavor) directly and pass it to ImageIO I get a black empty graphic.

• What I haven't fully grasped yet is whether in certain circumstances I need to to a Base64 transfer encoding and how I would do that without additional libraries (I have implemented the upload with nicked code rather than with e.g. Commons HttpClient because in a signed applet context all libs need to be signed as well and I couldn't face the extra depoyment steps for that)

• This is code that I'm currently integrating into a client project. It is only being tested for the (controlled) environment at this client and might now work in all OS/Browser combinations

• My next step will be doing a drag'n drop file upload which is easy now that I got the basic understanding of how applets work

References

• http://tinymce.moxiecode.com/punbb/viewtopic.php?pid=18088#p18088
• http://lassebunk.dk/2009/08/04/clipboard-java-applet/
• http://forums.sun.com/thread.jspa?forumID=31&threadID=451245
• http://java.sun.com/javase/6/webnotes/6u10/plugin2/liveconnect/

My thanks once more goes to all the people on the net sharing their findings.

Posted at 11:20AM Sep 16, 2009 by joerg in Allgemein  |  Kommentare[0]

Bash Quoting Trouble

Gerade hab ich mir mal wieder die Haare gerauft ob der Bash Quoting und Escaping Rules. Ich möchte in einem Skript eine Datei per smbclient kopieren (warum ist smbclient eigentlich so sperrig?) und dabei die Verzeichnisse hüben wie drüben und den Dateinamen variabel festlegen. Das richtige quoting zu finden hat mich viel nerven gekostet aber hier ist eine Lösung die funktioniert

#!/bin/bash                                                                                                                      
#set -x  # this has been very helpful in debugging the quoting                                                                                                                           

umask 022

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

# copy target settings
REMOTE_BACKUP_DIR=backup
REMOTE_NAS_ADDRESS=1.2.3.4
REMOTE_NAS_SHARE=BACKUP

# source dir and file
BACKUP_DIR=/srv/backup/project
TODAY=`date +'%Y%m%d%H%M%S'`
BACKUP_FILE=db_bak_$TODAY.bz2

# create the file to copy
...

SMB_CMD="cd $REMOTE_BACKUP_DIR; lcd $BACKUP_DIR; put db_bak_$TODAY.bz2"

smbclient -A /root/smbauth.conf -c "$SMB_CMD"  //$REMOTE_NAS_ADDRESS/$REMOTE_NAS_SHARE

exit 0

Posted at 12:23PM Aug 13, 2009 by joerg in Allgemein  |  Kommentare[0]

A trip to charset/encoding hell

I spent far too much time the last couple of days trying to solve an encoding problem in a springMVC/jquery based web application and here is a potential solution for those who run into the same or similar issues. I have a webapp that is mostly pure HTML but has some ajax comfort features in the backend. Everything worked for a long time and then the client started testing and immediately turned up a problem with text put into ajax forms turned into gibberish when containing Umlauts and such like. I started to investigate and went to hell. I quickly discovered that the same form when submitted via the jquery form plugin would screw up the text, whereas when submitted calssically without any javascript interference it would work. I thought that somewhere I must have forgotten to be explicit about the content-type or charset-encoding and made sue that each page would have the content-type header set to "text/html, charset=UTF-8". Now the ajax submitted form worked but the classical non-ajax submit would screw up the content. It seemed like there was also a difference between POSTing the form and submitting it via GET, with one generally working and the other not. I tried all sorts of combinations of headers and meta tags - alas no luck: I could have either one or the other. I searched the web and didn't find anything helpful for quite a while until I stumbled across this post, which basically introduced me to the Springs org.springframework.web.filter.CharacterEncodingFilter which can be used to enforce a charset header for every request. You just specify a filter like so:

<filter>
		<filter-name>characterEncodingFilter</filter-name>
		<filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
		<init-param>
			<param-name>encoding</param-name>
			<param-value>UTF-8</param-value>
		</init-param>
		<init-param>
			<param-name>forceEncoding</param-name>
			<param-value>true</param-value>
		</init-param>
	</filter>
	<filter-mapping>
		<filter-name>characterEncodingFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

<property name="contentType" value="text/html;charset=UTF-8" />

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/ />

Posted at 04:42PM Jun 29, 2009 by joerg in Allgemein  |  Kommentare[0]

soundtrack

Bin gerade aus ungeklärten Gründen über die Rainbirds und ihre Sängerin Katharina Francke gestolpert - ist immer noch gute Musik. Dabei ist mir aufgefallen, dass das Debüt Album der Rainbirds wohl als Soundtrack zu meiner ersten Liebe gelten kann - zusammen mit U2s "Joshua Tree", Udo Lindenbergs "Phönix", Men at works "Cargo" und einigen weiteren Alben (u.A. The Communards).
Vielleicht liegt es ja am kürzlich stattgefundenen 20-jährigen Abiturtreffen, dass ich solche Rückerinnerungsmomente habe …

Posted at 02:04PM Jun 05, 2009 by joerg in Allgemein  |  Kommentare[0]

google wave

Es ist nicht gerade originell sich zu Googles Wave zu äussern - viele, zu viele kluge und weniger kluge Leute haben sich schon geäussert (und die meisten davon auschliesslich aufgrund eines YouTube videos) - aber ich tue es trotzdem, da mich diese Sorte Technologie einfach stark interessiert und ich einiges in dem Bereich über die Jahre ausprobiert habe, da ich die meiste Zeit in geographisch mehr oder minder weit verstreuten Teams arbeite. Am besten gefiel mir dabei bisher Groove (das 2005 von Microsoft gekauft wurde und leider Windows only war) - Wave könnte ein würdiger Nachfolger sein und durch das offene Protokoll und das open sourcing des Codes mehr Momentum gewinnen als das Groove jemals getan hat. Ausserdem ist die Eintrittsschwelle natürlich weit geringer, wenn man lediglich einen aktuellen Browser benötigt. Sollte das alles so funktionieren, wie in dem Demo vorgeführt, kann ich mir vorstellen, dass sich Wave zu einem sehr nützlichen Tool entwickeln könnte. Bleibt als Nachteil nur die Skepsis gegenüber der Datenkrake Google - aber man wird ja angeblich seinen eigenen Wave Server betreiben können. Ausserdem frage ich mich schon, wie die Technik skaliert, wenn ein Wave Server Hunderte oder Tausende gleichzeitige Edits an alle Nutzer streamen muss.

Posted at 04:50PM Jun 02, 2009 by joerg in Allgemein  |  Kommentare[0]