Category: Tech

All about technology

Categories
Development

Agile databases

Ferris sent me a message a little while back and asked if I had any input on handling databases, so it gave me a reason to sit down and try to write something about my experiences. I’m not too happy about the flow of these articles, but I figured it was better to get something out there. Just let me know if something should be changed. Here we go…

Changes?

I see a lot of different approaches when it comes to handling databases. Sadly most of them are on an ad-hoc basis, and something that should be something of the past. With the introduction of agile, focus has been shifted from preventing change to embracing change. And we’re pretty pretty happy to change the code and do refactorings, but when a change affects our database we tend to think it’s quite scary. We need to find ways to do this better, and it is possible.

By implementing the practices mentioned below the projects I have been in are able to perform upgrades to our different environments (testing, pre-production and production) in a consistent way quickly. And I am fairly confident that the scripts have been thoroughly tested and that the chances for human error is drastically reduced. We can create new environments for testing with a new customer in minutes and fill the environment with test data. Because of this we can focus our time and effort on developing new features instead of wasting time on upgrades and errors that can be avoided.

Most of this is based on the Evolutionary Database Design from Martin Fowler and Datbase Refactorings by Scott Ambler.

I will try to describe more details with separate articles, but for now check out the migrations in Java which is the essence of what I think is necessary.

Your database is code!

If you don’t have schema specifications, triggers,  test data etc. checked into version control. You need to start doing it right now. When you tag, release or branch your “normal” source code you do the same with your database scripts. For Java developers using Maven I would recommend keeping it as a sub project in your maven project structure. That will make handling of tags/releases automatic.

Make it easy

Even though SQL, PL/SQL or whatever your poison is looks quite strange and awkward to many developers, you can script the things you need to do. For Oracle, sqlplus actually works for some basic scripting. If you don’t know how to; learn it.

These are the tools/scripts I think every developer should have as a minimum:

Making these things easy isn’t just because your developers do it all the time. It is also because having these tools will enable them to do it when it’s needed. There are a lot of good things in every project that simply is not done because it is too difficult or takes too much time. At least let us give the developers the tools we can, and remove that kind of mental blocks.

One of the most important parts of this is the migrations. Next: Migrations for Java.

Categories
Development

Learning to program

Tim O’Reilly has an interesting post on the O’Reilly Radar where he references some discussions they have had when it comes to learning, and in particular learning to program.

In essence it requires practice, and we need to work hard at it. There is however a bit of a paradox here. If you are going to practice that means doing the same thing over and over again with small variations to get good at it. I see in the discussions that it is often compared to learning maths, and I have no doubt that my utterly horrible math skills stems from the fact that I never took the time to sit down and practice. But the repetitions is difficult when it comes to software, as it is written to avoid doing repetitive tasks and we always learn that one of the basic principles of software development is DRY.

The way we usually work doesn’t really foster continuous repetition and learning either. When you are working in a project the solution is created and then you move on. The next time you get to try the same thing with a little twist from what you learned the last time around, is when the next project kicks off. With the worst projects around this means 2-3 years between each repetition. And according this definition of mastery you’re going to have a hard time reaching that:

According to Dizzy Gillespie, it takes ten years of practicing your butt off to achieve Mastery. His statement has been backed up by scientific research which shows that to reach a very high state of mastery, a task should be repeated about a million times, which takes about ten years. Better get busy! – (Sound the Trumpet, pp 67-68)

Related to practice is also another pet peeve of mine: We need to practice at it and use real OO. Usually we’re not good enough at using the power of our OO languages like encapsulation and separation of concerns. You can of course debate whether OO is the best way for developing software, but if you are using an OO language we need to get better at the basics. I believe that if we practice these basics more it will give us big returns in regards to quality and cost of systems. Way too many systems are built around POJOs acting as data carriers with all kinds of logic scattered all over the system.

So how and when to practice? Most companies should have structures for facilitating this during work hours, but ineveitably you will also need to spend some of your own time. One thing I like doing is participating in small projects. The smaller the project the sooner I get to move on and try a new angle on how to do systems development. Other ways may be to participate at a code dojo, or contribute to Open Source projects. I haven’t really tried a code dojo yet, but it sounds like a great idea. How would you practice?

The bottom line is that we can not read about something like programming and get good at it. Mastering software development requires practice and we need to focus on finding, using and creating spaces for practice.

Categories
Development

Norwegian government open sources travel expense project

The norwegian department of renewal and administration has released an open source solution for managing travel expenses (english translation). This is an interesting event and I hope and think that it is a practice that will be followed in the future.

No doubt, I think it is a terrifying experience for most developers to have to publicly publish their code in such a high profile project. And surely it didn’t take long before someone was picking it apart on Twitter, but that’s just part of learning. Quality review is now free, focus on that. 🙂

What is important now is that this is handled as a living product where feedback is included and others get to contribute. And you will need an active core of developers, so the people that did develop it needs to be kept on payroll for a good while to actively improve the product. Over time more developers will hopefully contribute, but in the beginning the core group is extremely important.

I can’t wait to see more projects like this. It will increase reuse and savings in the public sector, and it will put the code out there for everyone to see and contribute. In the case of consultants beeing hired to develop software for the public sector, they can no longer say that a project was successful because it delivered according to contract. They will also need to deliver quality. The code and system will be out there for everyone to scruitinize. Let’s hope more and more projects get open sourced like this.