From time to time I ask myself what the different memory columns in top mean. There are three of them in a regular GNU/Linux environment; VIRT, RES & SHR:

So I figured I should write it down:

VIRT

The following is counted when calculating VIRT:

• Application code
• Application data
• Shared libraries
• Swap usage

RES

The following is counted when calculating RES:

• The non-swapped physical memory a task has used

Note that RES is not CODE + DATA (for details see http://stackoverflow.com/questions/7594548/res-code-data-in-the-output-information-of-the-top-command-why).

SHR

This is the memory a task potentially has shared with other tasks.

I’ve been coding a service which communicates over SSH today. I ran into a issue when trying to run long commands and listening to their output. It was kind of tiresome to find out all tricks to do, so here’s my code:

import sys
import time
import select
import paramiko

host = 'test.example.com'
i = 1

#
# Try to connect to the host.
# Retry a few times if it fails.
#
while True:
    print "Trying to connect to %s (%i/30)" % (host, i)

    try:
        ssh = paramiko.SSHClient()
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh.connect(host)
        print "Connected to %s" % host
        break
    except paramiko.AuthenticationException:
        print "Authentication failed when connecting to %s" % host
        sys.exit(1)
    except:
        print "Could not SSH to %s, waiting for it to start" % host
        i += 1
        time.sleep(2)

    # If we could not connect within time limit
    if i == 30:
        print "Could not connect to %s. Giving up" % host
        sys.exit(1)

# Send the command (non-blocking)
stdin, stdout, stderr = ssh.exec_command("my_long_command --arg 1 --arg 2")

# Wait for the command to terminate
while not stdout.channel.exit_status_ready():
    # Only print data if there is data to read in the channel
    if stdout.channel.recv_ready():
        rl, wl, xl = select.select([stdout.channel], [], [], 0.0)
        if len(rl) > 0:
            # Print data from stdout
            print stdout.channel.recv(1024),

#
# Disconnect from the host
#
print "Command done, closing SSH connection"
ssh.close()

The code should be quite self explainatory, but here’s what it does in short:

• Connect to the SSH server
• Send the command (non-blocking)
• Create a loop, waiting for the channel to get an exist code
• The loop is looking for data to pring (stdout.channel.recv_ready()) and prints any data it receives

After I migrated my blog to Jekyll I started to look for a tagging engine to support tag clouds. Strangly enough Jekyll ‘supports’ tags in the markdown header, but they are not used (as far as I can see, that is). So I ended up using jekyll-tagging available at https://github.com/pattex/jekyll-tagging.

Here’s the steps I took to make it all work. The first steps are exactly as described in the jekyll-tagging installation instructions.

Install jekyll-tagging:

sudo gem install jekyll-tagging

Add the following to _plugins/ext.rb

require 'jekyll/tagging'

Add tagging options to _config.yml:

tag_page_layout: tag_page
tag_page_dir: tag

I then created a tag page layout. Save this under _layout/tag_page.html:

{% highlight html %}

layout: default —

{{ page.tag }}

{% for post in page.posts %}
• {{ post.title }} ({{ post.date | date_to_string }} | Tags: {{ post | tags }})
{% endfor %}

Tag cloud

{% endhighlight %}

You will now have to tag your posts, unless you haven’t already. So in each post, add the tags to the header. This will tag the post under jekyll and tagging.

---
title: My post
categories:
- jekyll
- tagging
---

When I was migrating my blog to Jekyll I ran into the following 403 permission denied error message, when I had restarted the local Jekyll server:

no access permission to /

There was no error log whatsoever. It seems like the error message was suppressed, because when I started Jekyll with --no-auto I got the parsing error causing this. So if you run into the same problem and have auto: true in _config.yml then try to start Jekyll with:

jekyll --no-auto --server

I ran into a problem when writing Python datetime objects to a JSON object. It can be solved like this:

date_handler = lambda obj: obj.isoformat() if isinstance(obj, datetime.datetime) else None
for doc in docs:
    temp_file.write(json.dumps(doc, default = date_handler))

The important part here is the lamba which is producing ISO formatted strings from a datetime object. Then the json.dumps takes the argument default pointing at the lambda.