I would love to have an easier alternative, but I don’t really see anything on the horizon that replaces it without removing too many of the benefits. But that’s really something for a different post.

Some basics

So if you’re doing ORM, and especially Hibernate there are some basics to keep your options open when it comes to performance:

• Don’t do explicit flushing
• Don’t disable lazy loading in your mappings
• Don’t use Session.clear()

There are valid reasons for doing these things, but usually they are used because someone does not quite understand how Hibernate really works. And what’s even worse; they limit your choices later on when tuning and changing the solution. So if someone are experiencing problems and consider doing one of these things; make sure they talk to your main Hibernate guy first. I can guarantee you there is a lot of pain involved in removing these later on.

So if you’ve been a good boy or girl and avoided these pitfalls you should be set to do some performance optimisations in the areas reported as slow. Yeah, reported as slow. Don’t do any funky stuff before you know it is an area that needs improvement. Doing special tuning will limit your options later on, so only do it where it’s really necessary.

Diagnosing the problem and finding the culprit

YourKit Java Profiler is my absolute favourite for diagnosing problems related to Hibernate. It enables you to see all queries executed, and trace it back into the code to figure out why it is run.

Trond Isaksen from Zenior also held a talk at Capgemini last week where he talked about using stacktraces in core dumps for analyzing problems. It might actually be your only option, because introducing Yourkit in production will cause side effects.

The amount of information can become quite overwhelming, but learn these tools and you will have a trusty companion for diagnosing your database performance in Java for a long time to come.

Hibernate performance

The way Hibernate basically promises to deliver performance is through caching and changing the amount of data that is fetched each time. This works for most of the cases where you use Hibernate. If not, you might just need to do some good old SQL.

Understand 1st and 2nd level caching and figure out how you can tweak relations to change behaviour and you have a good tool set to tune Hibernate with.

Fixing issues

Once you find the reason for your performance problem, and if it’s database or Hibernate related, you basically have the following options. I try to follow this list top to bottom. The last solutions impact your code more and can also complicate your deployment and infrastructure.

1. Check that your database indices are tuned. To have effective fetches your indices must match your actual queries, so make sure they are correct.
2. Consider how you do key generation. If inserts are slow, it might be because it calls the Database for a key for each and every row it intends to insert. Change generators or assign yourself. Stuff like thr Sequence HiLo Generator can drastically reduce the number of queries Hibernate does to your database.
3. Fetch on primary key whenever possible. Hibernate has some default methods called get/load that lets you retrieve an object based on the primary key. These methods checks with the first level cache whether the object has been retrieved within the same Hibernate session, and if so avoids database communication. So if you use these you will only get one call to the database, even though your code actually calls Hibernate multiple times. Using Queries will bypass this mechanism, even though you query on primary key.
4. Enable second level cache for Read Only entities. This is a really good quick win for stuff like Currency or Country. Close to zero cost.
5. Consider wether you are always using a set of objects at the same time. You rarely retreive an Order without looking at the underlying Items in the Order. Setting fetch=”join|select|subselect” or batch size on the relation can increase the speed. Note that this will then happen every time you fetch the Order. It will also effectively bypass any caches you have enabled, so make sure you consider all the usage scenarios for this.
6. Write custom queries for the situation. Setting the fetch mode in an association as in the previous section will impact every fetch of the Order object. If there’s only a few cases where the performance gets really bad and that is separated from other parts of the system, you can write a custom query. This enable you to tune the fetching to the concrete case and let other parts of the system actually benefit from lazy loading. This is preferably custom Hibernate Criteria, but can also be HQL or even SQL.
7. Use plain old SQL. There is actually things that SQL is better at. Use it, and use something like the RowMapper feature in Spring with it.
8. Refactor your code to enable better performance. Changes in the model, or the design of services and request can affect performance and might be the way to go. Especially consider the flushing rules for Hibernate. Making sure you read the information at the start of your transaction can reduce the number of times Hibernate writes to the database.
9. Write cache your objects. This can become quite complex because of synchronization issues. If you’re running more nodes (most projects are), you’ll need to set up synchronization between your nodes and caches. This reduces scalability and complicates setup and deployment. Keep it simple.

Let me know if there’s something I’ve forgotten, I’m still learning.

Some resources:

• Hibernate Doc: Improving performance
• Hibernate performance tuning

Even though it is not very scientifically conclusive Joakim Karlsson has some very interesting results on the locality of code changes. As Joakim discusses this relates closely to how you evaluate whether the code you are currently looking at is worth a cleanup. It’s very hard to do a good evaluation of this, especially since we always seem to underestimate the time to, and frequency, of when we will be re-visiting that code.

On my own gut feeling I have started to do a lot more fixes when I stumble upon them, exactly because I have done this error so incredibly many times: I have tried to be cautious, and avoid doing unnecessary work by cleaning up stuff that I’ll “touch just this once”. But by touching it you introduce new bugs, change old behaviour and learn something new. All of this feeds back into the code, and you will most likely be debugging or making more changes to that code within the near future. It’s nice to see a more structured approach to investigating this.

There are some good suggestions and even references in the comments of the article. It’ll be interesting to see any refined results on this.

Tandberg did an excellent effort of providing us with the video equipment, and enabled us to make the videos available. Even though sometechnical difficulties resulted in loosing some talks (mine included), they did an excellent job, and a big thank you to them.

I held a talk on agile deployment (again), due to some last minute cancellations.

Because my talk was lost I decided to put some details here on how you could get the material if you are interested. So:

• Slides on slideshare: http://www.slideshare.net/anderssv/smidig-utrulling-at-smidig-2009
• A longer version I held with video at JavaZone 2009: http://tcs.java.no/tcs/?id=2E72A3F8-CD11-4C5C-A29A-63EB3E95149F

My own company, Capgemini had 7 talks, which was a really good effort. To see other videos (Norwegian only) go to http://tcs.java.no .

I’ll have to go through them for a little retrospective later. Let me know if you have some feedback.

Here they are:

• Smidig utrulling: http://tcs.java.no/tcs/?id=2E72A3F8-CD11-4C5C-A29A-63EB3E95149F
• Rules engines vs. domain logic: http://tcs.java.no/tcs/?id=8E55C6BE-7CB5-4BEA-AB67-30B5FF6D4585

My company, Capgemini was present as usual, and the bright girls and guys did an excellent job of tweeting and blogging from the conference. If you know norwegian check out our twitter stream and the technology blog.

We also had two full feature talks, and I was really satisfied with how they went. Always something to change, but over all very satisfied. The topics were (sorry only Norwegian slides):

• Smidig Utrulling (Agile deployment) – Slides
• Rules engine vs. Domain logic – Slides

For the Smidig Utrulling talk I spent a lot of time creating code and examples. They contain some simplified Maven setup and a Java program to do deployment from the Maven repo. Check it out at http://github.com/anderssv/agile-deploy/tree/master . Documentation is scarce as always, but let me know if I can make something better. And feel free to use.

The videos will be available later, but if you want to see the ones that are available (of others, some english speakers) you can check them out at http://tcs.java.no . Tandberg delivered the video equipment and it seems like they are doing a hell of a job for the community. Check out the videos, there’s a lot of good talks there.

Now for some vacation, see you.

System

• PandaHome – Home screen replacement. Gives me as many home screens as I want and a nice dock.
• Locale – Profiles for your android. Adjusts sound etc. based on location or other parameters.
• AutoLock – Delays the need to draw your unlock pattern.
• Useful Switchers – Easy access to switch on and off the things that consume battery.
• TasKiller – Kill off tasks.

SMS and contacts

• Handcent SMS – iPhone like SMS handling with bubbles and all.
• SMS Popup – Popup covering the screen on SMS instead of just the top bar notifications.
• SMS Templates – Templates for sending SMS.
• SyncMyPix – Get profile pictures from Facebook as contact photos.

Connectivity

• Toggle Data – If you don’t have a free data plan, you really don’t want your Android to go checking the net all the time. This lets you turn off and on data usage.
• Wisync – Automatically turn off data sync when not on WiFi.
• Wifi Analyzer – Nice to have when messing around with my APs to find the free channels.
• WeFi Connect – Scan, connect and test open connections in the area until one succeeds.
• NetSentry – Keep track of downloaded data.
• Bluetooth File Transfer – Send and receive files over Bluetooth (sadly not standard).

Productivity

• Share Your Board – Takes a snapshot of your whiteboard, corrects the angles, crops and increases contrast. I mail mine to Evernote for search and storage.
• Evernote – Capture anything, for ever. It even does OCR on any images so you can search for what you have written.
• GDocs Notepad – Notepad that Syncs to a folder on Google Docs.
• Remember The Milk – My essential GTD app. Nice widget for displaying list of current tasks. Only works if you are a paying member of RTM.
• Thinking Space – Easy mind maps for Android. Syncs to “the cloud”, but no alternative client for editing (yet?).

Media

• Spotify – Music like I like it.
• Google Listen – Podcasts
• Qik – Stream your video live, and store them when you’re done broadcasting.
• Shazam – Figure out which song is playing on the radio.
• SnapTell – Snap pictures of videos, books etc. and get online quotes and links to IMDB etc. Also supports bar codes.
• Backgrounds – Easy way of finding new ones when you get tired of the one you got.

Travel and location

• Flightstats Lite – Not really a good app (just sends you to the web all the time), but nice shortcuts for getting your flight status.
• TripIt – Keep track of trips and contacts trips. Nice sync option to have everything on your phone before you leave.
• Trafikanten – Bus, train, tram and subway info for Oslo.
• Here I Am – Send your current location via SMS or Mail.
• My Tracks – Get the tracks and stats from your trip and export GPX file for Geotagging.
• MyCityBikes – Status for bikes in Oslo and other cities. Lets me find nearest bike, and nearest free slot.
• Wikitude – Augumented reality and the ultimate show off app. Can also help you find restaurants etc. near by.

Misc

• Barcode Scanner – Scan barcodes and open in browser or do a search. Eases installation of applications on the links below.
• Seesmic – Twitter client with twitter lists.
• Transdroid – Add and manage torrents in a variety of torrent clients on your computer.
• Brightkite – Location based social microblogging. The best so far.
• Bubble – Level anything.
• Meebo IM – Connect to just about any IM service, and use the same account as meebo.com.

Anything else I should be using? Let me know.

My talk on “Agile deployment” got accepted for JavaZone this year! I’m extremely happy, but a bit scared too. I’ll be talking about rolling out changes in a controlled manner, and one of the things that are usually neglected in this scenario is the database side. I’ll cover stuff like packaging and deploy of the application too, but that’s probably the area where I know the least. The database side of things are really sort of my expertise.

In connection with the talk and things at work I did a quick search for Java migration tools. DBDeploy I have used earlier, but there are now a couple of other contenders. Here’s my list so far of tools that work on sql deltas that can be checked into SCM:

• DBDeploy – Tried, few features but works well. Ant based.
• DbMaintain – Probably has the most features. Ant based.
• c5-db-migration – Interesting alternative, similar to DBDeploy. Maven based.
• scala-migrations – Based on the Ruby on Rails migrations. Interesting take.

I’ll definitely be looking into DbMaintain and c5-db-migration soon. DbMaintain looks promising, or I migh just contribute to DBDeploy some features. I’ll let you know how it went.

(updated with scala-migrations after first post)

Calendar

Google calendar is the hub of my calendar. Even though you don’t really like or use gcal it’s a great hub because there’s support for it in nearly every program that’s available. So from/to Google Calendar you can use:

• Goosync to sync with my phone. I especially like the ability to sync several calendars with different keywords.
• Calgoo connect to sync with Outlook
• Google calendar extension to sync with Sunbird

Contacts

For contacts it has always been sort of separate for phone and mail. With more and more happening on my phone it’s quite relevant to join these two registers. For contacts I guess the phone serves as some sort of hub. I use:

• Goosync to sync contacts so they’re available in Gmail
• ZYB to sync and clean up contacts (the merge duplicates feature is pretty nice). ZYB also has the surprising feature of updating your contacts with the latest information from Facebook. This also includes pictures, so suddenly I have avatars for many of my contacts.

I have a Nokia Series 60 phone, but Goosync which is used for both contacts and calendar only requires SyncML so it should work on many phones.

My fundamental reason has it’s origins in my builder background. To me, architecture is a craft. You can’t separate the idea of a building from the execution of it, even though a lot of architects act like it. This has been catastrophic for architecture. – Renzo Piano

In the software world there seems to be a notion that you can draw the architecture, and then let someone else design the details. Because it is based on high level ideas and sometimes faulty assumptions it becomes a resistance for building a good solution, in stead of enabling a solid foundation. Stop separating the architecture from execution, and make sure it is something that is flexible and evolves as execution moves forward. In other words: Architects need to take part in the execution to see the constraints the architecture imposes, and see new opportunities for the architecture to evolve into.

I’m not going to go into that rant now, that’s content for another post, but that also means that the architecture must be evolvable. And less code is more flexible than a lot of code. Strangely enough most architectural constraints tend to lead to more code, not less.

From the comments and questions I got afterwards, I could see how I should have included more detail. That would have made it even more interesting for that kind of crowd. I probably also should have clearified that I had limited time to prepare and that this was just a slightly extended version of a lightning talk I held at XP Meetup last year. I hope to get the chance to correct this in a JavaZone talk with more details. If you did see the talk and have comments please do leave them at the bottom of the page.

Many of the questions I got revolved around the handling of the database, so I just thought I should give some pointers here to articles that better describes what I have been up to:

• Agile databases
• Migrations for Java
• The new guy and his database

Check them out if you’re curious.