Category Archives: Development

SSL to the people

I have lost count of how many hours, days and even weeks I’ve wasted because of un-trusted self-signed/generated certificates in my time as a developer. Some util always trips up, makes me add a flag and then still fails. Trying to install custom root certificates works for some software, not all. It’s a complete mess and I long for the day it will be over.

Now, there is a solution: Let’s encrypt. Just get trusted certificates everywhere. Free, quick and easy. This will not give you all the levels of verification and trust that you can get from an SSL certificate; but it gives you an encrypted connection to every server out there. It’s way better than just HTTP, and it’s quick and easy.

Still beta, but took me about 10 minutes to set it up for this blog. Run, answer a question, check something and answer another. Bam! Encrypted! 🙂

Some thoughts and further links from Schneier.

A kind of Microservices reading list

I’ve been working with Microservices for the last couple of years. It was kind of an uncharted territory back when we started. But the attention on the net and in the community has increased a lot over the past year and this spring and summer brought quite a few sobering experience reports. It is definitely not failing, but there are some real considerations to be aware of, and some of them are quite major non-technical issues. Culture and organisation needs to be changed to support working in a highly distributed manner with Microservices.

So if you’re considering Microservices, you’ll be in for an exciting ride. For many it makes sense, but do read these articles and watch the videos to make sure you don’t just make a gigantic leap of faith. Being prepared for the issues will help you move forward when the going gets tough. 🙂

So here we go:

And if you have only two hours, watch these videos. A very clear message by two very good speakers:

If you’re Norwegian speaking you might also want to check out our talk about our experiences at my current client: https://vimeo.com/album/1807533/video/105777592 .

Let me know what I’ve missed. 🙂

Speeding up development cycles with Docker

When I started doing Puppet development I was looking for a nice way to do testing. Vagrant was the best option at the time. But when Docker surfaced, and I figured it could speed things up considerably. And it does. 🙂

If you would like to move on to the code you can find my setup on Github. It should’nt take much work to transfer to your project or something else that is similar without Puppet. Go to Github for the code: https://github.com/anderssv/puppet-docker .

Let me know what you think. The README should help you getting started, but let me know if anything can be changed. 🙂

Docker is some kick ass technology. It can be used for a wide variety of many tasks. It gives complete isolation between containers when it comes to processes and software, while NOT reserving large amounts of disk, memory or CPU. And it is lightning fast (really, starting a new container takes less than a second)! Look into it if you haven’t.

Doing Puppet means that you need a VM, but you also you need to reset it every once in a while to simulate building a machine from the bottom up. Combining the mechanisms in Docker I was able to minimise what happens when you reset, except for the things Puppet does of course. So I have:

  • Created a Docker image (not uploaded to repo, created on the fly by the scripts) that can be used for launching Puppet scripts. Having everything like puppet rpms etc. installed, updated and available saves a lot of time when running.
  • Launch a SSH Daemon. This serves two purposes: Keeping the container running and allowing for a interface to run the puppet command.
  • Kill the container to reset, so you will get a “fresh” container to run on.

Doing this enables you to “boot” a fresh new image in no time, and get running Puppet scripts as fast as possible. It is my new favorite toy for automating and virtualising tests. 🙂 Let me know if there’s anything I can make clearer.

Creating system docs in source control

Wiki’s are a great technology. Well for some things. For documenting systems I am not convinced. Some problems:

  • All the old stuff that piles up. Nobody ever deletes anything.
  • Versions. If you have maintenance versions as well as active development etc. how do you update, merge and maintain several versions?
  • Historical snapshots. What was the state of the entire documentation at the time of release?

Is is possible to solve with a Wiki? Yes. Does Atlassian do it? Yes. Does it work for us? Nope. Maybe we lack discipline but it’s not working. Our tries at fixing this has only seemed like band aids or placebos to the real problem. I want my documentation released, versioned, branched and merged with my code!

I want to put it in my source control (ah, I dream for Git, but I have SVN). When I discovered Flatdoc I was quite excited. It is a JavaScript based rendering of Markdown. So you don’t have to have a server serving it up, or some extra compile steps before viewing. Just write your Markdown and commit. That’s it. Jekyll is nice and all that, but it involves a bit more tooling and different ways of doing stuff than I would like to introduce.

I was already writing some stuff in my README files as Markdown, so I decided to give it a spin. And it works quite nicely. 🙂

Hoping I will be able to replace system doc in the wiki with it, but that will take more testing. Just some short notes on making it work in Subversion.

Making it local

Flatdoc doesn’t really need installing, especially if you run it with a project on Github. But because we have network zones, and I wanted to host it directly off our SVN-server (over HTTP) I downloaded everything:

wget https://github.com/rstacruz/flatdoc/raw/gh-pages/templates/template.html
mv template README.html
mkdir flatdoc && cd flatdoc
wget http://rstacruz.github.io/flatdoc/v/0.8.0/legacy.js
wget http://rstacruz.github.io/flatdoc/v/0.8.0/flatdoc.js
wget http://rstacruz.github.io/flatdoc/v/0.8.0/theme-white/style.css
wget http://rstacruz.github.io/flatdoc/v/0.8.0/theme-white/script.js
cd ..

So now you have a README.html with a subdirectory called flatdoc/ with all the scripts and styles.

Enabling SubVersion hosting

To enable the SubVersion server to serve the files you need to set the mime types. But first, if you have not; add them to SubVersion:

svn add README.html flatdoc
svn commit -m "Installed Flatdoc"

Then set the mime types:

svn propset svn:mime-type text/html README.html
svn propset svn:mime-type text/javascript flatdoc/*.js
svn propset svn:mime-type text/css flatdoc/*.css

Hooking in your Markdown file

Edit the template file you downloaded into README.html.

Change the links to JavaScript and CSS to be:

<!-- Flatdoc -->
<script src="./flatdoc/jquery.min.js" type="text/javascript"></script>
<script src='./flatdoc/legacy.js' type="text/javascript"></script>
<script src='./flatdoc/flatdoc.js' type="text/javascript"></script>

<!-- Flatdoc theme -->
<link href='./flatdoc/style.css' rel='stylesheet' type="text/css"/>
<script src='./flatdoc/script.js' type="text/javascript"></script>

Change the Flatdoc javascript (inside README.html) to point to the Markdown file you want to display. It should look something likes this:

<script>
    Flatdoc.run({
        fetcher: Flatdoc.file('README.md')
    });
</script>

That should be it. 🙂 Commit to SubVersion and access through HTTP.

Note about testing and local rendering

Because of security restrictions in your Browser, local testing will not work. You will get an origin-error when trying to read the Markdown file. You can either disable that security check in your browser, or use some kind of Markdown preview locally. The Markdown preview for Sublime Text works pretty well. 🙂

Bash #4 – Quote function parameters

I try to extract stuff into functions both for re-use and encapsulation. Coming from Java Bash itself leaves a lot to be desired in this area but that´s a different post. What I´ve found is that if I don´t encapsulate input variables with quotes it can become quite confusing what is defined and not. If you don´t quote and a variable is unset, it will shift your input.

function myfunc() {
	local OPTION_ONE=$1
	local OPTION_TWO=$2

	echo "$OPTION_ONE and $OPTION_TWO"
}

VARTWO="hello"

myfunc $VARONE $VARTWO
myfunc "$VARONE" "$VARTWO"

In the first (unquoted) line above OPTION_ONE inside the functio will “become” VARTWO. In the second example with quotes it´s easier to understand that VARONE is the one missing, because it doesn´t shift it to the left.

Bash #2

Part of what I find really hard with Bash scripts is encapsulation and error handling. It all became a tiny bit better when I discovered how I can print the call stack when something occurs. Check out this blog post for how to print the call stack.

I’m thinking this fits nicely with trap, but I’ll have to try that another day. 🙂