Skip to Navigation | Skip to Content



Archive for 2007

Announcing FreeTheNet Hack Night at FreeGeek Vancouver | November 21st, 2007

From the email:

Come one, come all to the DIY Homebrew Event of the Year:

FreeTheNet and FreeGeek Vancouver present…

Wireless Router Hack Night
Time: 6:30 PM until 9:30 PM
Date: December 1, 2007
Location:
FREE GEEK VANCOUVER
117 East 2nd Avenue
Vancouver, BC

This is for people who want to learn more about how to hack Embedded Linux hardware. They’ve heard about all the WRT54G hacking for years, but either have bricked their routers, or have just not bothered to try to get them going. This is also being used by FreeTheNet to do some recruiting for Hardware Hackers for many of the plans that we have coming up.

So, if you have an item that is on the Table of Hardware with the status of Supported, Untested, WiP or Kamikaze, please take it down and we’ll take a look at it. We’re also looking for 3.3V TTL to RS232 or 3.3V TTL to USB converters so we can get Serial Ports on certain devices (such as Fon Routers), so if you have some you wish to lend, or know of a good source for them, please let me know ASAP.

Recommended Hardware are as Follows:

  • Meraki Mini (if you have any spares)
  • FON (Units that haven’t been on the Internet yet are preferred)
  • Netgear WGT634U
  • Linksys WRT54G (v1-v4), WRT54GL
  • WRAP/Sokeris Boards

So, back to what’s going on. Yes, I am going to host a hack night for various routers at FreeGeek. I have no idea if anyone is going to come to the hack night, but since Vancouver goes through cycles of activity when it comes to embedded stuff, I think it’s time for another big hack thing. The cool thing about this is that the embedded stuff can run WifiDog and then we can demo DogOnRails. I’m hoping that once I get the firmware done and released, with a modified version of the WifiDog Client, and with DogOnRails, that we will be able to replace the Meraki Stock firmware in the new year and have an almost Free Software based system from Management Server to Access Point.

Next, I’m going to see about getting a screencast of Vlad when we deploy DogOnRails on the live server. I’m going to use that instead of the “Anonymous Rails App” that I’ve been working on as of late.

Posted in Uncategorized | No Comments » | Add to Delicious | Digg It

Gutsy Gibbon, Toronto, Wireless and Ajax! | November 9th, 2007

I realized that I haven’t actually blogged on my Nitobi blog in quite a while, and it’s not because I don’t have a lot to blog about. Here’s everything that’s gone down in the last few weeks:

  • DogOnRails got voted the best hack at Nitobi Hack Day! It takes two of my favourite things, embedded gadgets (especially those that run Linux) and RIA and smashes them together.
  • I’m now down to one laptop at work! I’m going to blog more about Ubuntu Gutsy Gibbon on the Dell Latitude D630, but I have top give props to URandR, it saved me a lot of time getting dual monitor mode setup, as well as using VMWare Server for IE Browser testing
  • I’m going to Toronto to attend the CWIRP conference on behalf of FreeTheNet! It’ll be interesting to see what other people are doing.
  • I’m now a fan of Vlad the Deployer! Vlad really is the way to get deployments done. Once I get it to ignore apache altogether for nginx deployments, I’ll post my experiences on here!

Perhaps when things slow down more, I can get into more detail, but for now that’s a sample of what’s going on! It’s craziness!

Posted in Uncategorized | No Comments » | Add to Delicious | Digg It

Creating a Production Ruby on Rails: nginx | October 14th, 2007

I was talking to someone about a new rails project, and we asked about which server would be the best one to work on. Of course, me being a person who loves apt-get and hates RPM, I said one word.

Debian

But then I had to point him to a place to go to make things easier. The problem is that deploying a Rails app is rarely easy, especially one that actually is meant to not die. Not only that, but I don’t always agree with all the tutorials. So I’m going to write a series of blog posts about how I’m going to do it. I’d recommend reading this post when it comes to installing ruby, rails and gems on Debian/Ubuntu but forget any of that Capistrano/Apache stuff that they talk about. I recommend vlad an nginx for setting up and deploying rails apps. We’re going to talk about nginx in thsi post.

So, the first thing that is done when setting up a production server is to choose the webserver. Now, conventional wisdom says that when you’re running Linux, you will most likely use Apache. Conventional Wisdom is very wrong, since Apache is a giant 800lb Gorilla of a webserver that has more features than you’ll ever possibly need. It’s great if you want to load things like mod_php or mod_python (which you would do with django, but that’s the topic for another post), but it sucks if you want to use it for Ruby, since we’re going to be forwarding everything to the mongrels anyway.

So, what do we use? We’re going to use the Big Red Webserver from Russia, nginx. nginx is a nice http/reverse proxy server, with small, human readable files. The first thing that we’re going to do is install it on Debian. Sudo as root and do this:


apt-get install nginx

See, isn’t apt-get the coolest thing ever! Beats the crap out of yum! Anyway, what this just did was installed nginx, so in /etc/nginx, you are now going to have to delete your stock nginx file and create a new one. The first thing that you do is specify the user. It’s best to create a user for this such as www-data.


user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log debug;

Note, we also set the log files. Now, we have to set some basic settings, such as the mime-type includes, the connections that we will accept, and gzipping your data. Simple, commonsense stuff. This begins the http configuration block:


http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_min_length 1100;
gzip_buffers 4 8k;
gzip_types text/plain;

OK, so far so good. Now, let’s specify some mongrel clusters. Depending on your app, you may want more or less clusters to balance the load. I’d ideally say at least 2 per processor, but sometimes you may want to run less of these for some weird reason. So, here’s what I have setup for a dual-processor machine.


upstream mongrel {
server 127.0.0.1:3000;
server 127.0.0.1:3001;
server 127.0.0.1:3002;
server 127.0.0.1:3003;
}

We’re going to show how to setup this in mongrel later. This is what we have currently. Now, we have to specify the server.


server {
listen 80;
server_name www.dogsridingrails.com;
root /var/www/dogonrails/current/public;
index index.html index.htm;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

Not much to see here. We’re using nginx as a proxy to the mongrel servers. We point to public just like how we would in any rails application that’s going to production, and we specify what we do in the case of a filename request. In the case that we request index, and index exists, we show the index.html page. Otherwise, we pass it all to mongrel. Then we use a closing brace to finish the scope.


}

Now, that was MUCH simpler than the beasts of Apache logs that you’d have to wade through to do the same thing. It’s interesting to note that nginx is a lightweight proxying server, and is actually designed to do this, as opposed to Apache which is more general purpose, and is meant to load web apps using shared libraries which is always much faster than doing something like using mongrel.

I’m not saying that nginx is the right tool for every job, in fact, I would use think seriously about using Apache for a Python/Django project, but that’s the topic of another post entirely. Stay tuned for my next post about Vlad the Deployer!

Posted in General, Linux, Rails, Ruby | 2 Comments » | Add to Delicious | Digg It

S3 | October 5th, 2007

In the life of a web application, there comes a point where that shared hosting account just isn’t good enough (and you found out because your provider kicked you off), or your server just isn’t able to pull the queries from the database fast enough. Then one day, you finally get the filesystem error EMLINK, which you have a VERY hard time googling.

This is simple, you just created the maximum number of subdirectories that you can have in a directory. This is suprisingly not a common issue with file_column, acts_as_attachhment or attachment_fu, although I’m shocked as why it’s not. So, what do you do when you’re faced with scalability issues, and you’re image handling plugin is broken!

THROW IT ALL AWAY!

That’s what I had to do. Recently we worked on a site and we decided that because it was getting too hammered, that we would put the images on S3. Then we found the ultimate weakness of S3, which is that it’s not able to easily handle batch processing. We used the AWS:S3 library for most of the movement of the files, but we found that if we made a mistake, it would cost us hours to get these back.

Eventually, the day was saved with jetS3t, and Cockpit. Using jetS3t, we were finally able to actually get through all the S3 issues, and it saved the day at the end. (Actually, Dave saved the day at the end, my computer kept running out of memory). But we managed to get S3 support into it, and all we had to do was sacrifice File Column and replace it with this:


def user_image=( blob )
# establish S3 connection
AWS::S3::Base.establish_connection!(:access_key_id => AWS_ACCESS_KEY_ID, :secret_access_key => AWS_SECRET_ACCESS_KEY)
datestamp = Time.now.strftime('%d%m%Y')
identifier = UUID.random_create.to_s
object_path = "images/" + datestamp + '/' + identifier + '/'
object_key = object_path + blob.original_filename
self.image = blob.original_filename
self.image_dir = 'http://s3.amazonaws.com/bucket/images/' + datestamp + '/' + identifier + '/'
image_data = blob.read

#Send the file to S3
AWS::S3::S3Object.store(object_key, image_data , 'bucket', :access => :public_read)

# resize to thumnail here
img = Magick::Image.from_blob( image_data ).first
thumbnail = img.resize_to_fit! 96, 96

# Set the thumbnail directory path
thumb_key = object_path + 'thumb/' + self.image

AWS::S3::S3Object.store(thumb_key, thumbnail.to_blob , 'bucket', :access => :public_read)
end

However, if you have to do S3, I would highly recommend using a long key so that you can sort your re.sults better based on this key! However, the biggest gotcha I found when adding S3 integration to my rails app was including AWS/S3. If you include and require it, it will break your routing, this is something that can cause hours of headaches, especially if you are doing something else. At the end, we learned that S3 is a misnomer. For a large number of files, it’s far from simple.

Posted in Linux, Rails, Ruby, Uncategorized | 1 Comment » | Add to Delicious | Digg It

How the mesh works | September 18th, 2007

Many people have picked up on the Meraki Mesh idea, but people seem to be confused as to what a Mesh Network actually is. Here is the Wikipedia definition of a Mesh Network:


Mesh networking is a way to route data, voice and instructions between nodes. It allows for continuous connections and reconfiguration around broken or blocked paths by “hopping” from node to node until the destination is reached. A mesh network whose nodes are all connected to each other is a fully connected network.

The truth is that the Mesh Network is not really a Mesh but a Mobile Ad-Hoc Network. This means it has the properties of a mesh, but it’s mobile! For those who are interested, here’s the link to how this works from wikipedia:

ExOR Wireless Network Protocol

This should explain the basics of how this works, which is what Wikipedia is really good for. The Routing protocol that is being used is SrcRR, which what was used in Roofnet. This is an open-source protocol, and can be used in anything that has an Atheros Radio.

The nice thing about the Meraki Hardware is that it makes it accessible to people who want a finished product. It could be possible to mesh with the Linksys WRT54G using Optimized Link State Routing, but then there’s the problem of forcing the radio into Adhoc mode because Broadcom has a more closed design than Atheros. It also could have been done with Netgear WGT634Us and the new Linksys WRT150N, but these devices are twice as expensive.

Also, I find myself warming a bit to the Dashboard. It doesn’t allow people to configure custom spash pages per node, which would be nice functionality, nor does it allow for very much customization of the spash page. I think that this will be added down the road by Meraki since a lot of
people seem to be asking for this. Also, I find that it’s an interesting project working with a group of people who have admin on the nodes.

We’re definitely learning as we go along, and that’s what makes this interesting.

Posted in FtN, Linux, mesh | 2 Comments » | Add to Delicious | Digg It

Pics of the Meraki Gear | September 11th, 2007

I managed to get my girlfriend‘s Digital Camera to work, and now I finally have some photos in on my Flickr account (many years after I registered the account)

Check them out here

Posted in FtN, General, Linux, mesh | 1 Comment » | Add to Delicious | Digg It

Mesh Wireless goes to the Mainstream, (maybe) | September 10th, 2007

I have a hobby of hacking the firmware on the Linksys WRT54G. I originally started doing it because I wanted to learn about how Embedded Linux worked, and I thought it was cool that it could run Linux. That’s how I got introduced to the Community Wireless movement.

Basically, the problem with the DIY Community Wireless hacking is that you’d have to either take off the shelf routers, flash them (and void your warantee) and then hope that you got something working. Then you can write applications for it like WifiDog, or various Mesh Networking Applications such as Optimised Link State Routing. This was great, but it ran into two big problems:

  1. It’s hard to convince someone to run a hacked Linksys router in their home, because it looks sketchy
  2. You’re at the whim of the manufacturer, who may not like that you can extend your hardware, or may change the hardware randomly or End of Life(EOL) it because they can make something that is cheaper.

In fact, the original Linksys WRT54G was changed after Version 4.0 to run vxworks because it took less processing power, voltage and memory to do what they wanted than they needed from their prior cookie-cutter design. Also, Netgear also discontinued the WGT634U because of similar logic. The reason I mention the WGT634U router is because that is what MIT Roofnet originally used to build the prototypes for what is now Meraki Mesh.

After BarCamp and talking to Boris at Bryght, I decided to buy some Meraki hardware. Now, I was expecting some unmarked boxes, and the devices to be large, but I was very suprised to find a branded box, like what you would find in FutureShop, and a very small device. Not only that, but it is extremely user friendly. I was also impressed with the range of the device. I put one in my Window at my apartment, and it seems to have a 100m (about 300 ft) range. Now, this is important, since the way mesh works is that you put a bunch of mesh nodes out into the world, and they route between each other to the nearest gateway node, the node with the least latency.

When I compare the Meraki out-of-the-box solution to the alternative, which is the Freifunk OLSR, there’s really no comparison for how easy it is to use. I think that Meraki has a very interesting project and it’s worth testing out. The main advantage of us testing out mesh is obvious, since we can facilitate a test bed for Ajax components in mobile devices right outside our window. With the release of the iPhone and the iPod touch (more importantly the iPod Touch, since we’re in Canada), content that is dynamic, and takes advantage of both geography, as well as the various user agents, is critical to providing a user experience like nothing else.

With more and more mobile devices equipped with Wifi for mass adoption, it just makes sense to at least play with the stuff. I’ll have pictures up here soon of us playing with the hardware!

Posted in FtN, General, Linux, mesh, UI | 1 Comment » | Add to Delicious | Digg It

Nitobi Grid on Rails – Part 1 | July 5th, 2007

Even though we’re doing a lot of work in Ruby on Rails, and we blog about it a lot, we don’t sell a back-end for it. So the question that people may ask is how do we use our Components with Rails?

Here’s some information on how I’m using Rails with the current version of Grid. In Rails, you can use Builder to create XML, so the first thing that I am going to create is a grid controller. In that grid controller, I define a getHandler method, which looks like this:


def getHandler
if params[:SortColumn].nil?
@tblcustomer_pages, @tblcustomers = paginate :tblcustomers, :per_page => params[:PageSize].to_i
else
@tblcustomer_pages, @tblcustomers = paginate :tblcustomers, :per_page => params[:PageSize].to_i , :order => params[:SortColumn] + " " + params[:SortDirection]
end
render :layout => false
end

What this does is use the paginate function to do some pagination of the table. We set the PageSize using the Nitobi Grid’s PageSize paramater, and we use the SortColumn and the SortDirection coming into the function from the grid to determine the column sorting.

I then have a view (rxml) that looks like this:


xml.instruct! :xml, :version => "1.0"
xml.root(:fields => "CustomerName|ContactName", :keys => "CustomerName|ContactName") {
for tblcustomer in @tblcustomers
xml.e(:a => tblcustomer.CustomerName,:b =>tblcustomer.ContactName, :xk => tblcustomer.CustomerID)
end
}

This is very basic. I’m just starting out with builder, so I don’t have the Error Handler here. I suppose that I should have that there, but I don’t know what will happen if its nil. (I suspect it will break). That’s definitely something I’m going to have to ask around about.

And, I make sure that I have the getHandler set in the Grid Definition like this:


gethandler="getHandler/"

Of course, I make sure to include grid and toolkit. Once this is all done, I have a grid that can load. I still have to get the savehandler working on Rails, but this should be enough to get started with Rails. If I find any interesting interactions with Grid, Toolkit and Prototype, I’ll post them here.

Posted in Grid, Rails, UI | 1 Comment » | Add to Delicious | Digg It

Rails on the Weekend? Why????? | July 4th, 2007

Well, this weekend I had to write some Rails stuff. Once I did the work stuff that I had to do on Saturday morning while I’m sure some kids watched some cartoons. I then decided to revisit something that I used to contribute to.

The result of a couple hours of hacking is this:

http://code.google.com/p/dogonrails/

That’s right. DogOnRails! I used to advocate running WifiDog hotspots back in the day. The problem that I had with Wifidog is that it was hard to run unless you had a dedicated server running Postgres. There was no way to run it on Shared hosting for just your one hotspot. So, I decided to hack this thing together.

It’s pretty much just written for my old WRT54G running OpenWRT, but I might do something interesting with it. It’s under GPLv2, so check it out!

Posted in Linux, Rails | No Comments » | Add to Delicious | Digg It

Ruby on Rails versions | June 25th, 2007

I really like doing Rails work, but if there is one pet peeve that I have, it’s moving between Ruby on Rails versions. There are a fair number of Ruby apps out in the wild, and when handed a Rails app to start maintaining, you need to move from version to version. Of course, what’s interersting is how to check for the different rails versions. For example, some people think doing this will work:


rails -v

That does work, IF you are not using the vendor/rails version of rails. However, this means that you have to know what rails version you’re developing for. This becomes a major pain for certain tags that are simply ignored in Rails 1.1 and are rendered in Rails 1.2. Days can be done developing in Rails 1.2 before deploying on another site, and if you don’t know that you’re running Rails off of vendor/rails, this could be a major problem.

So, how do you determine which rails you’re running?

One way to do it is to go into the console, and run this:

Rails::VERSION::STRING

This will load what the application environment sees as its version of rails, which is much more useful than the previous command. However, if you run this command:


script/about

You’ll see the Ruby version of phpinfo(), which will give you all the information that you need to rock and roll. There are cases where you may want to downgrade to a prior version, namely if you’re using a plugin that requires a certain syntax in a previous version. However, there’s arguments for and against this behaviour. I would always use the latest version of rails, mainly because it’s easier to give to someone else who’s not as experienced with Rails, and it’s better to have them learn how to use the latest version as opposed to trying to find documentation for previous versions of rails.

So far, I have to say that going between versions of rails is one of the most frustrating things that I’ve found so far with Ruby on Rails.

Posted in Rails | No Comments » | Add to Delicious | Digg It