Installing psycopg2 on OS X

I have recently been doing some python development that uses the psycopg2 driver for PostgreSQL. On Linux, this worked wonderfully, and everything was installed using the regular easy_install method with no troubles. However on OS X it wasn’t quite so easy.

First I tried using easy_install but it failed with a long list of gcc warnings and errors. Looking near the beginning of the errors I saw:
Warning: /bin/sh: pg_config: command not found followed by
./psycopg/connection.h:27:22: error: libpq-fe.h: No such file or directory Well this makes sense as I’m using PostgreSQL Plus for OS X, and it doesn’t put it’s header files or libraries in a standard place. There is an easy workaround though.

Thanks to Luke Tucker who provided this easy solution: PATH=$PATH:/Library/PostgresPlus/8.3/bin/ sudo easy_install psycopg2

Original Method

This is left here for historical purposes. I recommend using the above command instead.
We’ll need to download the psycopg2 package and install it manually. Don’t worry, this is very easy and doesn’t require much work at all.

  1. Open you favorite terminal application.
  2. Download the package. Extract the package, enter the directory.
    curl -o psycopg2.tar.gz http://www.initd.org/pub/software/psycopg/psycopg2-latest.tar.gz
    tar xzf psycopg2.tar.gz
    cd psycopg2-${version}
  3. Edit the setup configuration file. Use whatever editor you’re comfortable with.
    vim setup.cfg
  4. Uncomment the include_dirs line and change it to read:
    include_dirs=/Library/PostgresPlus/8.3/include
  5. Uncomment the library_dirs line and change it to read:
    library_dirs=/Library/PostgresPlus/8.3/lib
  6. Save and quit your editor.
  7. Finally, build and install the package.
    sudo python setup.py install

And now you should have a working PostgreSQL driver for Python. This was very useful for me since I like to use SQLAlchemy for my database work in Python and it uses psycopg2.

Alternative

As a alternative method, you can replace steps 4 and 5 above with:

  • Uncomment pg_config line and change it to read:
    pg_config=/Library/PostgresPlus/8.3/bin/pg_config

Thanks, pacopablo!

End of Another Semester

It’s nearing the end of another semester, and a lot has been going on. My classes are wrapping up well; most of the hard projects are over with. It’s been quite a bit of work getting used to a full course load, but I’m satisfied with my performance.

My classwork has been somewhat dull this semester. I finally got around to taking Personal and Professional Development which has been less than thrilling. The class has great information for underclassmen, but as a senior, it provided little new information. It also doesn’t help that it’s a one credit hour course with the workload of a 3 hour course. It is a bit demotivating to be spending so much time on a course you feel has less value than the others.

I am also taking Short Stories to finish out my humanities requirements. The course is alright, fairly well-taught, just quite a bit different than the math and computer science courses I’m used to. The reading took a long time to get used to, especially since my interest in the stories wasn’t extremely high. Although I really enjoyed several stories and found out that I really enjoy Poe’s work. The papers were my strong point in the class, which is surprising to me. I never thought I was that great of a writer, creative or analytical, but my professor was sufficiently impressed.

Computing and Society was another 1 hour course I needed to get out of the way. Pretty interesting topics, especially when we had guest speakers.

Calculus II is well, Calculus II. I did considerably better this semester than in my previous attempts, and it looks as though I will be passing. The course material turned out to be fairly easy for me to grasp once I applied myself.

The last class I took was probably the most interesting to me. Applied Matrix Theory has been surprisingly fun to take. The teacher has a great attitude about the course, and teaches it very well. The application focus of the course also makes things very interesting to my implementation based mind. Learning new ideas in class and then reading about how they’re used in my field days later proves to make the course much more tolerable.

That’s all there is to my academics this semester. I’m glad to be back in the swing of things.

Plants are Hard

I made a post about how Travis and I were starting an adventure with carnivorous plants. Well, we had done everything exactly as the directions said, but we were unsuccessful. The plants never sprouted, and we were very disappointed. In fact, just the other day I was informed that they had been thrown out because of the stench they were creating. I’m sure everyone will be as heartbroken as I am. If we stick together we can make it through this.

Amazing Flash Game

I’m usually pretty skeptical of flash games. Most of them seem to be mildly entertaining remakes of puzzle or arcade games, so usually I find them to be forgettable. But every once in a while I find a game that really grabs me. Tonight, I found one of those games. It’s called リネージュII, or Lineage II: The Chaotic Chronicle.

I’m not going to go into much detail as there isn’t much to explain, but I will say that there are no other controls other than the arrow keys and your mouse. This had me a bit confused for a bit since the instructions are in japanese. Basically, there’s forty items that you have to collect to clear the game. You will have to solve numerous puzzles, many of which are intertwined and fairly complex. After you collect all forty items there is one last task to get the “Complete” status. I was able to get the complete status in 1 hour and 39 minutes and had four out of five of the character’s heads at the end. I’m not sure what this means, but if you get a different amount, comment and let me know your time, as I think that may be what determines how many heads you receive.

Also, make sure you do this when you don’t have anything to do for a while. I was planning on doing something, but then this game distracted me and I got really caught up in it. Just thought I’d warn those of you that like to be productive. ;)

Running rTorrent on OS X

This guide is old and probably outdated. I used these steps to compile and install rTorrent 0.7.4/libTorrent 0.11.4

So, you’ve come to the realization that most of the OS X BitTorrent clients aren’t up to par with those offered on other platforms. Until today, I had been using the standard BitTorrent client but it left me wanting more, especially a “watch” folder. A watch folder is a folder where I can throw my torrents and know that my client will automatically pick them up and download them. Anyways, I was pointed to rTorrent today, and thought I’d write a little guide on getting it up and running on OS X, since I was unable to find something quickly with a Google search. Here is step by step what I did to get rTorrent working for me.

Install XCode

XCode is a set of development tools provided by Apple. They can be obtained by anyone at the Apple Developer Connection. You will have to sign up for an account, but you can do so for free. Download the latest version of XCode (2.4.1 at the time of writing this), mount it and run the installer. This should be a familiar process for most Mac users. As a warning the XCode dmg is almost 1GB.

Install/Update DarwinPorts

OS X doesn’t really have a package manager, and resolving and installing dependencies manually is no fun. This is what DarwinPorts is for. It is based on the BSD package manager and is pretty slick. If you don’t like DarwinPorts, you may want to consider Fink, although they didn’t seem to have rTorrent marked as stable when I looked.

Assuming you choose DarwinPorts, download the Universal Binary from their main page, mount it, and run the installer that it contains. Then you’re going to need to add one line to your ~/.bashrc file so that port will be in your execution search path. You can do this with one command in a terminal. Open up Terminal.app (In /Applications/Utilities or just use QuickSilver) and copy and paste this line into it and press enter. echo "export PATH=\$PATH:/opt/local/bin" >> ~/.bashrc If you’re using zShell then you can use the same thing in your ~/.zshrc, but if you’re using zShell you probably already know what to do.

If you already had DarwinPorts installed, you should probably run port selfupdate to upgrade it to the latest version (you’ll need to sudo to do this).

After you’ve installed or updated your copy of DarwinPorts, you need to update the package list so that you can get the latest stable version of rTorrent. You can achieve this with the port sync command.

Congratulations, you should now have a working copy of DarwinPorts.

Install rTorrent

rTorrent and all of it’s dependencies can be installed with one simple command. sudo port install rtorrent When sudo prompts you for a password, just enter your local account password. That’s all there is to installing rTorrent. It may take 15-20 minutes depending on your computer.

Configure rTorrent

Like many *nix applications, rTorrent uses a configuration file located in your home directory to decide how to behave. I’m not going to go in-depth and explain every option to you, as I have left comments in the file to do that. I have changed very little from the official documentation’s example config, so you may wish to just start from theirs instead. Anyways, here’s the link to my config file, just save it to your home directory and make any changes that you feel are necessary.

If you’re going to use my config, make sure you have these directories created. ~/torrents/files
~/torrents/session
~/torrents/watch

Start rTorrent

Open up Terminal.app and run the command rtorrent and you should be presented with a mostly blank screen that says rTorrent and the version at the top.

Add Your Torrents

Now, all you should have to do is drop your torrents in the watch directory you specified in the config file, and it should automatically pick them up and start downloading. If you chose not to use a watch directory, you can add a torrent by hitting the return key and typing in the name of the file. It has tab completion and I recommend taking advantage of it.

Learn the Commands

I don’t know them all yet, but I’m picking them up quickly. I’m not going to list them all here because the rTorrent devs already took care of it over here. A notable one though, is Ctrl-q for quit.

Encryption

More and more ISPs and campuses have started to throttle or completely disable bittorrent. Encryption can sometimes help you workaround their restrictions and let you utilize all of that precious bandwidth. Look in the config file you downloaded from earlier in this guide. On line 83, you’ll see an option “encryption” that has been commented out. If you keep this line commented out, you will never use an encrypted connection. But, there are other users out there that will only accept an encrypted connection. So, we want to change this. If you uncomment the line as it is, it will allow you to accept an encrypted connection but will prefer unencrypted if that’s possible. This will enable you to connect to any body, whether they use encryption or not. But, we probably want encrypted to be the default, so let’s change it a bit. encryption = allow_incoming,try_outgoing,enable_retry This line will initiate and accept encrypted connections but will fallback to plaintext if the other user is not using encryption. This is my preferred method, as it will allow me to connect to as many people as possible, but will use encryption where available. encryption = enable_retry,require
or
encryption = enable_retry,require_rc4
These two options (as I understand it) will require an encrypted connection, meaning no unencrypted connections will ever occur. The difference between require and require_rc4 is how much of the data it encrypts. Using require will only encrypt the header information, which will sometimes be enough to get around the your ISP’s throttling/blocking. The require_rc4 option will encrypt the entire stream, all the data that ever gets sent between the two clients. This is much more difficult, although not impossible, for a traffic shaper to detect. Note that require_rc4 will be much more processor intensive, so you may notice some performance differences.

Other Things to Consider

  • Run rTorrent in a screen session. You can learn more about GNU screen here.
  • Use Visor for you rTorrent session. Visor provides a quake-like console for OS X using Terminal.app.
  • I don’t know everything. If I have a typo, mistake or just plain say something that’s incorrect, please leave a comment so I can fix it.

Carnivorous Plants

Travis Bradshaw and I ordered some carnivorous plants for our office and they are going to be amazing. Too bad right now we’re just embarking on the start of the process known as stratification which (in our situation) is totally crappy. Basically, it means we wait while the plants pretend it’s winter. I’m impatient. I want to watch plants eat things now, not in 8 weeks.

Well, I’ve created a new category for posts related to the raising of our precious carnivorous plants, so expect more posts later.

Video Game Woes

So, I heard from Amber that GameStop was trying to get rid of a lot of their used games and had a giant $10 and under display. Well, since most of the games I buy are under $10 I had to go give it a shot. I ended up buying 7 games. This isn’t unusual for me. Considering I walked out of the store with 7 games and only paid as much as one new game, I think I did pretty well, and actually, this happens quite often. Last weekend, Duncan and I did the same thing at Hastings. Anyways, on with the rant.

Duncan and I aren’t just gamers, we’re collectors. A lot my friends are this way, we really care about the condition of our games. So, when we pick out a used copy of our game, we look for a manual, original case and of course the condition of the disk or cartridge. Well, today I managed to find all non-greatest hits copies of the games I wanted. All of them had manuals and the cases were in great condition. However, when I went to checkout, the clerk there went under the counter and retrieved what I thought to be the disks for each game. It turns out, he didn’t just put the disks in the cases, he replaced the cases with whatever was in the drawer. Of course, he was just putting them in the bag as he goes and I didn’t check the bag until after I left the store. Well, Rygar, was replaced without a case at all, atleast, not an original case. It was one of there generic cases with the title written on it with a sharpie. Also, Smuggler’s Run and Devil May Cry were replaced with Greatest Hits versions of the cases and disks. And with Dynasty Warriors 4, he gave me a Greatest Hits disk and a non-Greatest Hits case.

The Greatest Hits switches don’t really bother me too much, if they hand’t of had the normal versions of the games, I still would’ve bought the Greatest Hits version. Since they had both versions though, I prefer to pick up the original, but like I said, this didn’t bother me so much. However, Rygar pissed me off quite a bit. I went through all those games and picked out the case in the nicest condition and everything, and he doesn’t even give me a case. I couldn’t believe it. I understand that the majority of their customers probably don’t care, but I go in all the time, and I’m always obsessing about the condition of everything I pick up, and it’s the same guy helping me every time. I guess I’m not that memorable or something. But next time I’ll know to make sure that I’m getting exactly what I picked out and not whatever the clerk feels like giving me.

Command Line Tip

So, today I had to transfer a large amount of data from a NetBSD box to a new Gentoo server and I ran into some issues. First, I had to move eight gigabytes of data. Well, first thing I did was try a recursive scp but it seems that the scp utility follows symlinks. So the eight gigabytes of data was getting copied multiple times because this filestructure happened to be heavy on the symlinks. Then I thought I could tar it up and send it but it turns out there wasn’t enough disk space locally and the machine was a little slow to wait around to gzip or bz2 compress the data. Well here’s the solution:

tar cf - directory_to_copy | ssh user@destination.machine "tar xf -"

This will tar up the directory and put it to STDOUT which gets piped to ssh. Then, on the remote machine we extract STDIN. Worked like a champ. Pretty fast and much better than setting up a network filesystem just to get a transfer done. So, with two of the most commonly used unix/bsd/linux/whatever commands, you’re able to transfer files securely. Now, if you don’t need a secure connection you could do this without an encrypted connection and it would be even faster, or you -C the ssh command and compress the connection. This could also be done with gzip or bz2 with the tar commands but I have no idea which would yield the most speed. I imagine bz2 would not be the fastest. ;)

NoteHack, Ruby on Rails and TextMate

So, I don’t update this blog very often. Mainly because I find that I just don’t really have time to sit down and write a nice long update all the time. And, I don’t really enjoy writing long posts. They always turn into rants and I find that most of the longer posts I write are pretty useless to everyone but me. However, I do like my short little “OMG THIS IS NEAT” posts so you may see this blog shift more to that end of the spectrum. Anyways, on with the real shit you want to read. :D

I play NetHack quite a bit as many of you are aware. Well, I find my memory isn’t quite up to the challenge sometimes, especially when my games span over an entire week. I’ve decided to write a website that would be a note-taking system designed to aid NetHack players. I call it NoteHack and you can find it at http://notehack.jonypawks.net. It’s a pretty fun project, and it will be continually improving. I’ve been really excited about working on it every night and don’t see myself dropping this project like the others. I’ve been using Ruby on Rails for this project, which brings me to the next topic.

Ruby on Rails is a rapid development framework that has been gaining attention very quickly the last couple years. I’ve used TurboGears, which is a similar framework for python, but found it a bit harder to jump into. However, this could just be because it was the first framework of the type I used, and so my TurboGears experience has probably helped me learn RoR. Also, RoR is quite a bit more mature than TurboGears, and has much more complete documentation, which is a huge plus. Anyways, if you have an idea for a neat little (or big) webapp, I recommend checking out Ruby on Rails, you won’t regret it.

And for my last little plug, we have TextMate, a graphical text editor geared towards web programming. I’ve been using it the entire time I’ve worked on NoteHack and it’s a pretty robust little application. I like that it has a single purpose and it does that well. It’s not trying to be a godly text editor that can syntax highlight the configuration file for the proprietary software your grandmother uses to make to-do lists. But, I’m a minimalist when it comes to software, so this may be an opinion only shared by me.

Sorry if you aren’t a coder, this post probably wasn’t very interesting to you. :(

Snakes on a Plane?

So, I just got back from the premiere of Snakes on a Plane, and I must say that it is the finest work of American film that I have ever experienced. Once you get past the plot holes, bad acting, and complete lack of a resolution, there’s a spectular example of what every movie should be. There was a good 15 minutes of solid snake-on-genitalia action, and who can pass that up.

Well, I’m out for now. This weekend I will try and write a post about the week I spent at QuakeCon 2006 because I’m sure you’ve all been looking forward to it.