As of 19 December 2010, web2project v2.2 is officially live! You can download it from SourceForge now.

While in many releases we might focus on cleanup or functionality or developer aspects or similar, this one is a mishmash of a bunch of useful updates on numerous fronts. This isn’t all of the updates but a bunch of the important ones:

For the Project Managers:

• We reworked much of the Gantt Chart logic. We’ve added a few icons to better represent the status of tasks and milestones. To make the overall charts easier to understand, we’ve added a legend and shaded alternate lines.
• Gantt Charts are now exportable as single-page PDFs. You can print, email, share, or whatever with just a click.
• We updated Token Tasks – which represent Subprojects – have been updated to prevent direct editing. Further, more of the Subproject data synchronizes as expected.
• Depedency Tracking is now ON by default. This will provide better cascading updates when Tasks move.

For System Admins:

• If System Timezone or System Admin Email are not set, the system will provide warning messages with links to the specific fields in the System Admin screens.
• If filesystem permissions are properly configured, the Module screen supports direct uploading.
• The Reports module has a permissions check applied. Previously, it did not.. though the underlying data was filtered appropriately.
• The core system supports configurable pagination for the Project List screen and a few others.

For Developers:

• We replaced our old Javascript library (Mootools) with jQuery. It’s faster, easier to use, and widely supported.
• Added approximately 40 Unit Tests covering areas such as CDate and other classes.
• Added a call in the Calendar Module to display arbitrary date-related information. A reference implementation is coming in the next release.

For General Users:

• We’ve added Czech and Russian translations. If the dice rolls work out, we’ll move to Kamchatka next.
• We’ve set the field focus to the first text box on each screen.
• Private Tasks are now respected on all screens including Gantt Charts.
• We fixed a number of visual issues.

Finally, special thanks goes to community member Opto who consistently submits great bug reports, shares useful patches, and provides random insights that help and speed things along. Thanks!

And that wraps our releases for 2010.

This year we managed 1 Major Release, 3 Minor Releases, and 1 Patch Release. Most of the releases were a week or two later than we wanted but we managed to get all of them out on schedule. More importantly, each and every release has managed to provide new functionality and close bugs while collecting and responding to community feedback.

And in case you missed our recent coverage on SourceForge, check it out!

Corporate developers seem to struggle a bit more when a company open sources their internally developed code. It’s not so much the development or coding aspect of it, but more the interaction and community building aspects they struggle with.

Some common items they struggle with:

1. No knowledge silos. An open source project can’t afford to have knowledge silos. A successful project needs a team that can work on any piece of the project. This goes against most corporate training where people have very specific roles and responsibilities.
2. No Testing Team. Developers are required to write their own unit tests, and integration tests. Many corporate developers seem to struggle with this. They have never had to do it, it’s always been the responsibility of the QA team to write the tests.
3. Maintaining the Build. Again, many are used to just writing and submitting the code. Another team takes care of building the software. Note: This isn’t just a corporate problem, many open source projects struggle with this as well.
4. Answering and Responding to forum/mailing lists. Typically a developer may never actually communicate with the person that filed a bug report. Part of growing a community is timely responses to questions and bugs.
5. Marketing and Promotion. That’s the job of everybody on the team, not just the team leader. Again many aren’t used to having to do this, as the Marketing Department will handle promotion, press releases, etc.

None of these are simple things to address, but when choosing the initial team, look for those team members that may already be involved in open source projects. Seed the team with some of these people, and the initial growing pains for the project will be less severe. The biggest thing though, the team has to be willing to adapt and change, what they did in the corporate environment more than likely will not work with their open source project.

Wow, perfect timing for our Lotusphere presentation. You can download the tool right from Google or you can read about the functionality it brings.

Tools being donated include the WindowBuilder Java UI design tool as well as CodePro Profiler, a runtime Java analysis gauging factors like memory leaks. Both tools became Google property when the company bought Instantiations in August; they will now become open source projects at Eclipse. WindowBuilder has been used for development related to Standard Widget Toolkit, GWT (Google Web Toolkit), and Swing.

Check out the full article on InfoWorld.

This post was lifted wholesale from Cal Evans’s post from DevZone. I think he summed it up well and I had nothing to add.

Admittedly, I only knew Richard in passing. We’d met at a couple conferences and he was at CodeWorks last month. I wish I’d gotten to know him a little bit better.

I was recently asked the question: Rails or Grails? I needed to summarize the key differences and industry sentiment. This was my response.

Before I make any subjective comments, let me start with some objective metrics I found:
http://www.expectationgap.com/posts/13

Now for my sentiments, I believe there are three key-dimensions: momentum, cloud-support, and people.

Momentum

Rails has the momentum as evidenced by the number of committers and plugins available. Many of the rockstar developers I know are all now developing in Rails, supporting that community, etc. The underlying Ruby language is incredibly expressive and allows dramatic productivity improvements. (write less code that can do more)

Only a handful of rockstars I know are developing on Grails. Typically, those developing on Grails are in enterprises or markets that are comfortable with Java and they want to continue leveraging that investment. (infrastructure, app servers, production support, etc.) Grails/Groovy does have the advantage that It can continue to leverage the output of the Java community (e.g. Spring). The better projects within the Java community are quickly integrated into Grails. For example, Grails uses Hibernate and Spring Security under the hood. As those projects improve, so will Grails.

Cloud Support

Rails and the cloud go hand in hand. Again, of the rockstar Rails developers I know, nearly all of them are running on virtual hardware owned and managed by others. Some of those let others manage the entire application server / platform (e.g. EngineYard). In contrast, all of the Grails development I know of is running on hardware (virtualized) owned and managed by the company itself in their own data centers (or rented space). As mentioned before, this may actually have been the motivating factor to select Grails. That is not to say that there isn’t cloud support for Grails. And now that VMWare owns SpringSource, this may change rapidly now that enterprises will have a trusted name (in VMWare) to host their Grails applications.

People

Rails has the momentum, more committers and potentially a more active community, but lets not fool ourselves — there is an army of labor out there to sling Java code. Offshore resources are readily available. The market for Rails is developing, but demand isn’t yet high enough to drive consulting companies to substantially increase their supply. As one of my good friends put it, “I’d love to develop in Ruby, but Java pays the bills.”. It is that sentiment, that has may limit the availability of resources.

However, even with readily available Java resources, that doesn’t guarantee the availability of Groovy resources. Both Grails and Rails require a slightly different mindset. If you take a Java developer and tell them to code in Groovy or Ruby, you’ll end up with Java code using Ruby/Groovy syntax. (not good) But at least the tooling, debugging practices, and libraries will all be familiar.

And just to stoke the fire…

Here are my entirely subjective scores:
Momentum: Rails (9) vs. Groovy (4)
Cloud Support: Rails (8) vs. Groovy( 5)
People: Rails (6) vs. Groovy (8)
—————————————————
Rails (23) vs. Groovy (17)

After my earlier post Referrer and Comment spammers are a PITA I came up with two mod_perl plugins to Apache and an “apache level” firewall.

The reason for the apache-level firewall is two-fold.? There is no direct way for the Apache user to manipulate an iptables chain (as it doesn’t run as root), and second; I was not happy with suid root access or other forms of message passing to a daemon which would manipulate the firewall for me.

Architecture is thus, in httpd.conf place the following two lines:

PerlPreConnectionHandler PGREGG::httpBLBlock
PerlLogHandler PGREGG::httpBLLog

The first tells apache to run the handler in my httpBLBlock.pm module when a connection is received (before the request has been sent by the client).? In this handler, I am simply looking for a filename matching that IP in a directory that is writable by the apache user.? The contents of the file are a SCORE:httpBL_answer:[LIST].? Based on this, the module checks the mtime of the filename is in the last SCORE days, then the firewall is in effect. If so, we simply tell apache to drop the connection.? If the file has expired, we delete the file.

The second line is more interesting, and what creates the firewall filenames. In order to not impede the general speed of request handling, processing is performed in the Logging section of the Apache process. Our module is called by apache after the response has been sent, but before the access_log entry has been written.? In our module we perform the http:BL API call and compute the above SCORE based upon the Threat* level and Age* of the API response. (* both Threat and Age are octets in the DNS lookup).? We merely discount the Threat down to zero based on the Age (0-255) where an entry 255 days old reduces the SCORE to zero.
If the SCORE is larger than our trigger level (3) then we create the firewall filename, log the entry in our own httpbl.log and return Apache2::Const::FORBIDDEN.? This causes Apache to not log the entry in the normal access_log.? Otherwise, if all is ok, we return Apache2::Const::OK and Apache logs the hit as normal.

I have a bit of code tidy up, restructure the config/firewall directory and pull some common code out to a shared module before I can release to the world.

An interesting side effect to publishing the last story out through Planet PHP and other news sources along with the Project Honey Pot image is that when browsers viewed those sources, they all asked for the image off my server. In several cases, these were known spammer, Comment spammer, and other abusers. My server then created the firewall entry blocking them before they were able to follow the links back to my server.
?
I have been reading up more on Apache Bucket Brigades in an attempt to allow the firewall filter to be placed immediately after the request has been received and allow a custom response to the browser. This may help an otherwise unsuspecting user if their machine had been trojaned. I don’t mind admitting I’m thoroughly confused right now

APIs are commonly an afterthought, like a hot tub awkwardly attached to a house???a shoehorned
approach that produces a suboptimal app with scarce support that lacks
documentation. In effect, APIs are the ugly stepchild of the Web.

This is a sad reality that we are faced with, because
many companies make their living consuming third-party APIs and
mixing in their own data to create amazing
and interesting mashups.

In the initial phases of development, there is rarely
enough money to develop the app and its API. By the time there?s both demand and money, it can be hard to fit an API on top of the
architecture in such a way that the whole thing won?t fall over.

APIs should be first class citizens of the Web. Inconceivable?
Possimpible?
Not at all!

What can be done to solve this problem?

Is there a silver bullet? I?m afraid not.

My company has done a lot of API
work where we have encountered and solved a myriad of different API
problems for customers and our own apps. We even released an
API framework along the way. That?s not
to say that we haven?t made our share of mistakes on that journey???in fact, we have
made every single mistake in the book, but that?s how you grow and
learn. I firmly believe that you cannot solve a problem unless you?ve
failed at least once or twice before.

In the past, we have often been involved in bolting an API on top of
legacy apps, and on top of startup code we were directly
involved with from the beginning. Most of the time, neither situation is
ideal for attempting to use the current code base
(or to bring in behemoths like Zend Framework or Symfony)
to write an API service layer. We put together our
own lightweight API framework which can
be dropped into a code base without much fuss, and it can reuse the data
models. As a side bonus, it will automagically generate API
documentation.

Now we are getting somewhere! We had solved some of the pain
points of building an API, even if it?s bolted on afterward. That was not
enough for us, though.

The solution we came up with is fairly simple, but not glamorous at all,
I?m afraid.

A traditional web site has data models that interact with the
database and any other data source. Add an API into the mix, and it will
either be interacting with the data sources directly or potentially using
the same data models as the web site.

Our approach shakes things up a little bit.
The API is built first, and it interacts with the database and any other data
source (e.g., Oracle, MySQL, or Solr). The web site then consumes the API just as a client would.
The web site is still built with a MVC framework,
but it?s completely oblivious to where the API is pulling the
information from, so the API becomes a middle layer, a data layer.

The benefit is that the API becomes core to the system, is
built from the start, and will be maintained properly. This way you can make
an iPad app, and the web site use exactly the same API.

I also find APIs to be easier to scale, but that?s a topic for another
article.

The downside is that this development model does cost you more money in
the short term, and it means a tad longer time to market, but if you compare
it to adding the API at a later stage and maintaining to separate systems,
then you will clearly see big financial and operational gains.

Story time

An example may explain this better. This is the tale of the elusive Twitter
API.

Back in 2007, Twitter took off at SXSW with a roughly 300% increase in
tweets per day. Everyone was happy, but many people felt like the web
interface was not all that great. People asked for an API, so they could create
their own apps and mashups. Lo and behold, an API was
bolted on top of Twitter. It became evident that the initial API was not
enough for people, and thus began the organic growth of the monster that is
the Twitter API. It was a fairly under-maintained and under-staffed API in
my opinion, given it was the lifeline of Twitter. It is what kept millions
of people using the service.

This is a classic example of a bolt-on API that was not accounted for properly in
the infrastructure, and it quite possibly caused a majority of the fail whales.
The web site and the API (to my knowledge) were separate.

A shift happened in 2010, and #NewTwitter was released
in all its glory. A new web interface with many bells, whistles, and, quite
possibly, a lawnmower. However, what was most exciting about the release was
that the web site started consuming; it became a

Truncated by Planet PHP, read more at the original (another 1751 bytes)

Lighttpd is a secure, fast, standards-compliant web server designed
for speed-critical environments. This tutorial shows how you can install
Lighttpd on an Ubuntu 10.10 server with PHP5 support (through FastCGI)
and MySQL support.

I talk with a lot of people about NoSQL. I?ve been following it pretty closely for about 3 years now. One of the things I often tell people who are trying to wrap their heads around the concepts of NoSQL and what all of the data stores mean is to go search for Justin Sheehy and his NoSQL East Talk.

Then it dawned on me, why don?t I just embed the video on my site? Makes it super easy for people to find (travisswicegood.com/tags/riak) and I don?t have to go searching for it.

So, here?s what I consider one of the best talks to date on NoSQL.

There are times in career when you get excited about having an experience for the first time. I well remember how I got excited about seeing my first self-coded shell node popping up in the Windows Explorer (a.k.a custom shell namespace). A bit of excitement I noticed seeing my first reader’s comment printed in iX. And I was really excited about receiving my first printed articles in JavaMagazin and iX. Or when heise.de published my first online article on their well-known site. I really got excited when I was asked by the JAX management to speak there, which was an honour back then in the first days of that conference. I got excited when I got told that I was nominated as the CEO for an affiliate. And certainly I was excited when I was awarded the "GAP" (GlassFish Community Award).

So today once again was one of these days when I got excited about a new experience. This time, it was about someone citing and discussing a blog article of mine (Even when it was just in a blog rollup. Hey, they picked up MY blog entry for that!). And it was not just someone. It was JavaWorld. So I not only was excited, but also proud. A bit, at least.

While obviously all of these is nothing compared to being nominated for an Oscar(R) or a Nobel Prize, it actually is really fun to see myself getting excited still. Ain’t this part of that abstract idea of "worth living for"? I do think so. Excitement drives live (at least mine) and development of people, companies, products and markets. So I do hope that I will get excited again soon. Let’s see what comes next.

Regards, Markus.

An overview of all my different publications and products can be found on my personal web site Head Crashing Informatics (http://www.headcrashing.eu).