Guyub – Konsultan F/OSS

I was recently asked the question: Rails or Grails? I needed to summarize the key differences and industry sentiment. This was my response.

Before I make any subjective comments, let me start with some objective metrics I found:
http://www.expectationgap.com/posts/13

Now for my sentiments, I believe there are three key-dimensions: momentum, cloud-support, and people.

Momentum

Rails has the momentum as evidenced by the number of committers and plugins available. Many of the rockstar developers I know are all now developing in Rails, supporting that community, etc. The underlying Ruby language is incredibly expressive and allows dramatic productivity improvements. (write less code that can do more)

Only a handful of rockstars I know are developing on Grails. Typically, those developing on Grails are in enterprises or markets that are comfortable with Java and they want to continue leveraging that investment. (infrastructure, app servers, production support, etc.) Grails/Groovy does have the advantage that It can continue to leverage the output of the Java community (e.g. Spring). The better projects within the Java community are quickly integrated into Grails. For example, Grails uses Hibernate and Spring Security under the hood. As those projects improve, so will Grails.

Cloud Support

Rails and the cloud go hand in hand. Again, of the rockstar Rails developers I know, nearly all of them are running on virtual hardware owned and managed by others. Some of those let others manage the entire application server / platform (e.g. EngineYard). In contrast, all of the Grails development I know of is running on hardware (virtualized) owned and managed by the company itself in their own data centers (or rented space). As mentioned before, this may actually have been the motivating factor to select Grails. That is not to say that there isn’t cloud support for Grails. And now that VMWare owns SpringSource, this may change rapidly now that enterprises will have a trusted name (in VMWare) to host their Grails applications.

People

Rails has the momentum, more committers and potentially a more active community, but lets not fool ourselves — there is an army of labor out there to sling Java code. Offshore resources are readily available. The market for Rails is developing, but demand isn’t yet high enough to drive consulting companies to substantially increase their supply. As one of my good friends put it, “I’d love to develop in Ruby, but Java pays the bills.”. It is that sentiment, that has may limit the availability of resources.

However, even with readily available Java resources, that doesn’t guarantee the availability of Groovy resources. Both Grails and Rails require a slightly different mindset. If you take a Java developer and tell them to code in Groovy or Ruby, you’ll end up with Java code using Ruby/Groovy syntax. (not good) But at least the tooling, debugging practices, and libraries will all be familiar.

And just to stoke the fire…

Here are my entirely subjective scores:
Momentum: Rails (9) vs. Groovy (4)
Cloud Support: Rails (8) vs. Groovy( 5)
People: Rails (6) vs. Groovy (8)
—————————————————
Rails (23) vs. Groovy (17)

I was moving a Rails 2 app up to Rails 3 and, because I have PostgreSQL installed in a non-standard location on my server, I ran into problems when bundler was trying to install the pg gem. After fiddling about for a bit I ended up with this in my config/deploy.rb:

require 'bundler/capistrano'

task :set_config_for_pg_gem, :roles => [:app, :db] do
  run "cd #{current_path} && bundle config build.pg --with-pg-config=/usr/local/pgsql/bin/pg_config --no-rdoc --no-ri"
end

before "bundle:install", :set_config_for_pg_gem

$ cat ~/.bundle/config
---
BUNDLE_BUILD__PG: --with-pg-config=/usr/local/pgsql/bin/pg_config --no-rdoc --no-ri

A while back I posted a link to a talk by Gleb Arshinov that he gave at the SF PUG. This talk was on “PostgreSQL for high performance Rails apps”, and was full of fine suggestions from their experiences with their Rails apps.

Gleb is back again, this time on May 21 2010 at PGCon where he and Alexander Dymo talked about PostgreSQL as a secret weapon for Rails apps. Some of the same ground is covered (use SQL DDL vs ActiveRecord create_table, etc), but there’s lots of new information too. Here are some notes:

• 1:10 They’re using PostgreSQL 8.4, nginx, and mongrel
• 4:00-6:00 Talks about dropping down into SQL via ActiveRecord
• 6:30 Use include to eliminate N+1 queries.
• 7:30 Watch for things like acts_as_tree that reintroduce lots of queries in exchange for the improvement in abstraction.
• 9:00 One query, 12 joins – complicated, but query time goes from 8 seconds to 60 ms.
• 14:00-17:00 A technique for recording SQL queries; this helps ensure you’re not running unexpected queries
• 19:00 Suggests use straight SQL for DDL rather than the ActiveRecord DSL
• 20:00 Use constraints, FKs, etc to preserve data integrity – “anything you don’t have a constraint on will get corrupted”
• 23:00 Don’t use CASCADE since app won’t know about the deletions
• 28:00 Keep a log of times for the most frequent user requests. Alex suggests using integration tests for this; code is at 29:10 and 29:30.
• 32:30 A technique for loading data with ActiveRecord’s select option with PostgreSQL arrays to save on object creation. Questions from the audience about normalization vs efficiency.
• 38:50 Role/user/privilege checking can be slow; shows a technique for using PostgreSQL’s bool_or and GROUP BY to get the data in one fell swoop. Query time went from 2+ seconds to 64 ms.
• 42:00 Do analytics in the database. Saw speed improve from 90s to 5s and saved tons of RAM.
• 44:40 Some excellent new PostgreSQL features that are either here now or are on the way (replication, windowing functions)
• 46:30 Demonstrates a problem with PostgreSQL’s LIMIT and OFFSET when used with subselects. Some discussion of pagination with the audience. Here’s an excellent discussion of pagination alternatives written by Justin French.
• 50:30 How to force PostgreSQL to use a subselect vs a join; the example goes from 605ms to 325 ms.
• 52:20 Be careful with generate_series. Apparently these functions cannot generate hints for the planner.
• 55:30 General props to PostgreSQL community.
• 59:40 Need to test queries both in cold state and hot state; they saw 14x speed difference.
• 1:01:40 Tune PostgreSQL – shared_buffers, work_mem, autovacuum, etc. Rely on community knowledge for initial configuration.

Lots of good stuff there, enjoy!

I’m slowly catching up with my podcast backlog and came across a Pivotal Labs talk from May 2009. In this talk Josh Susser and Damon McCormick are presenting on Scaling a Rails App with Postgres . It’s a little dated now – this talk was given was when PostgreSQL 8.4 was in beta – but, still, lots of good stuff. Here are some notes:

• They started with an existing Rails app with lots of data, so they had some constraints – not greenfield development.
• Around the 5-6 minute mark there’s a good discussion of PostgreSQL’s query optimizer and how it analyzes a table’s data distribution. One takeaway (mentioned around 16:20) is to run vacuum more often on a particular table if there are a lot of writes.
• 10:00 How to set STATISTICS for a particular table.
• 11:00 Using partial indexes.
• 14:00 Indexing on expressions.
• 18:10-23:00 A nice discussion of the EXPLAIN output.
• 23:45 Here they talk about wide columns. I’ve seen this in MySQL as well, where splitting text data out into a separate table yielded some good speedups.
• 26:10 Some discussion of pg_bench.
• 35:30 How long does it take to add an index to large tables? They saw times of up to an hour for tables with millions of rows.
• 36:30 clustering your data in order to get PostgreSQL to write it more efficiently.
• 37:30-48:00 A thorough discussion of partitioning tables via table inheritance. They used an ActiveRecord model (39:23) with a bunch of utility methods. They also had a cron to periodically create new partitions. At 45:15 they make a nice distinction between using partial indexes and partitions – one advantage is that a partition’s indexes can be different than its parents indexes. At 49:00 they mention maybe doing a plugin, not sure if that happened.
• 52:00 Some discussion of full text search via tsearch.
• 53:00 PostgreSQL’s lack of built in replication outside of WAL shipping, Slony, etc. Thank goodness 9.0 will address this!
• 54:00 Some props to Engine Yard on their PostgreSQL support.

Good stuff all around, and thanks to Pivotal for posting these great talks!