Django: Development to Deployment (Part 3)

I started the series with setting up your local environment using Vagrant and Fabric to quickly bootstrap. In the second part, we reviewed some conventions for Django development as well as useful tools and tricks. In the final part of the series we will cover a simple deployment to Amazon EC2.

Getting started with AWS

The first step in deploying to Amazon EC2 is to setup an account with Amazon Web Services. This is fairly straightforward. Go to http://aws.amazon.com/ and click on “Create an AWS Account”, and follow the steps. It may take a few hours for the account to be active, but then you will be able to login to the AWS Management Console. From there you have access to all the AWS services.

For our simple application, open the console and click the EC2 tab. The Amazon Elastic Compute Cloud (EC2) allows you to rent server infrastructure on a pay-as-you-go basis. This is great for startups or projects where a large investment in server infrastructure is not possible.This is perfect for our project, as we will deploy onto a single server. There are several important key concepts in EC2:

  • Instances : An instance in EC2 is a single host. There are a variety of types of instances, ranging from micro to high-memory large instances. Each type of instances has a variable amount of CPU power, RAM, I/O capacity, and disk space. The more powerful machines cost more on an hourly basis.
  • Elastic Block Store : EBS is a service which allows for persistent storage. A typical EC2 instances has “instance storage” provided which is very large and high speed, but there is no data backup provided. If the instance dies, then the instance store is lost. EBS instead provides disks which are backed up and will persist in case an instance dies. They are not necessary, but may be useful depending on the application.
  • AMI : AMI is short for Amazon Machine Image, and it is basically a “snapshot” of a running machine. When a new instance is started, it uses an AMI as its base image. This image may have any operating system or software pre-installed. Typically, you would start an instance with something like Ubuntu or CentOS with a default install, and then customize it from there. If you want, you can create your own AMI from an existing setup.

Read More

Django: Development to Deployment (Part 2)

Update: Part 3 is out now: Deploying to AWS.

In Part 1 of our series I covered setting up your local environment using Vagrant for virtualization and Fabric in combination with several other tools to do the bootstrapping. In Part 2, I’ll cover my Django setup and the development process.

Django Settings

At the end of part 1 we created a blank project and had a web server running that allowed us to reach the congratulations page. The next step of our setup is to setup Django, mostly by modifying our settings.py file. This file controls all the settings for Django, and if very important.

The first thing I do is add some helper methods to resolve the absolute path to the current directory. This has an edge case related to how we use symlinks inside Vagrant, that we need to check for, but is otherwise straightforward:

Read More

Django: Development to Deployment (Part 1)

Update: Part 3 is out now: Deploying to AWS.
Update: Part 2 is out now: Developing Django.

As programmers we adopt new tools to make our lives easier and increase the speed of development. Web development is an area of extremely rapid innovation with many incredible libraries and packages. I have worked on a number of Django powered websites, and I wanted to share my process to help those who are just getting started or who want to improve their setup.

I plan on covering an end-to-end project, from development to deployment on AWS, so there is a lot to cover. In an attempt to make this more manageable, I’ll break apart the posts into multiple parts. In this segment we will setting up the local environment.

Virtualization

It may happen that you have an idea for the next billion dollar, social, 2.0 cloud service and you want to just start coding immediately. So you download the tools you need and start hacking. Soon, you need to work with others and incompatibilities arise. Then you push to production and nothing seems to work. In our exuberance to build things, we sometimes forget the engineering part our work: it’s all about the process.

Read More

Hierarchical Clustering of Facebook Friends

Studying Artificial Intelligence has exposed me to the many sub-fields of research in the area. Machine Learning, which is the study of algorithms that allow computers to learn and evolve based on the data they process is particularly interesting. Unsatisfied with my shallow theoretical understanding, I recently ordered O’Reillys Programming Collective Intelligence book, which provides hands on experience with practical examples and code.

I have a few projects in mind that need to perform various forms of clustering, so I decided to try a quick project to get my hands dirty. First, I needed a source of data that I could cluster in a sane way. It struck me that it would be interesting to cluster Facebook friends based on the similarity of status updates. We just need to fetch some status updates for all (or some) of our friends, and build up from there.

The first step is to use the Facebook Graph API to fetch our friends. We need to get the base URL, and our OAuth access token. To get both, I just go to the API reference page, and copy the access token from the URLs of the examples. Lets, first get our friends:

Read More

Open Sourced: A tale of two Priority Queue’s

One of my favorite data structures is the binary heap. I first learned about it in my data structures class, and remember marveling at its simplicity and elegance. How could something so simple be so powerful and useful?

For those unfamiliar with the binary heap, it is a relatively simple data structure. It is a binary tree, which means each node has at most 2 children and n top of this, there are several constraints which are imposed. One is that the the children of each node must have a value that is equal to or greater. This means we have some sort of loose ordering, where nodes closer to the root have a lower value. Secondly, the tree must be complete. This means that all levels of the tree except for the last level must be full, and filled left-to-right. As described, this would be a Min-Heap, since the minimum value is always at the root. If we were to reverse our comparisons, so that the children must be less than or equal to the current node, we would have a Max-Heap with the maximum value at the root.

One of the primary uses of a Min/Max Heap is as a Priority Queue. If one inserts nodes based on their “priority”, then the nodes with the highest priority are at the root.

Below is a example of a Min-Heap, where each node represents a persons age and name. We can see that there is a rough ordering with the youngest person at the root.

Read More