Collective Intelligence in Action

Don’t you hate it when you spend months (or years) working on a pet project / book / mad take over the world idea, then somebody comes out with something even better?

Yep, it’s just happened to me. Years working on the idea of the ‘Wisdom of Crowds’ (even prior to web 2.0 in the shape of Red-Piranha). Month’s working on a Masters Dissertation on applying Web 2 techniques to the Finance industry (pdf link). And somebody comes out and does it even better.

Not just better. But much much better.  The sort of better as in ‘If I had this earlier, I’d have just copied it and changed the words around a bit’. The book is available from Manning as ‘Collective Intelligence in Action‘. A free, first chapter (Understanding Collective Intelligence) is available here (pdf).

Collective Intelligence in Action

So what’s it about? We’ve all heard about the Wisdom of Crowds idea. But what if you need to actually implement it on your website? This book shows you how to (using both concepts and practical code, as well as the theory behind all of it that I was missing). It includes

  • Intelligent, learning search, using Lucene.
  • Extracting data from blogs using web-crawling.
  • Executing Real time feedback on facebook-like sites.
  • Scalable data-mining techniques to manage the torrent of information
  • Making personalised recommendations based on all of the information.

Disclaimer:Manning provided me with a free review copy of the book – but no strings attached. And , maybe if I’m nice enough to the Author (Satnam), I can persuade him to talk about making millions using JBoss Drools and Complex Event processing in the book.

Spring Belfast and Spring Dublin – two of the biggest Java events of the year

No apologies for over simplifying this :

  1. If you’re in business you need computer systems to support your team. Systems to find previous dealings with a customer, systems to allow your team to work together, systems to stop people finding out things that they shouldn’t.
  2. Once your business passes the 100 employee mark and is heading for ‘Enterprise’ scale, chances are you need custom software written just for you, in addition to he ‘shrink wrap’ stuff you’ve been able to get away with until now.
  3. Most custom Enterprise software is written in (or uses a large element of) Enterprise Java.
  4. Enterprise Java is hard to get started with – it’s a big and complex framework because it solves big and complex problems. The Spring framework makes it easy.

spring logo

OK, so we’re Spring nuts. But nuts only because it’s solved problems for FirstPartners over the last 4 years. What we like:

  • Spring allows you to use just enough Enterprise Java to solve your problem
  • Spring complements Enterprise Java, not replaces it.
  • Spring gives you a gentle slope to using Enterprise technologies.
  • Spring works well with Java, Oracle, .Net , (J)Ruby and pretty much any mainstream technology – including most of the widespread Java Frameworks like Struts and Hibernate.

So you wait ages for one Spring Event in Ireland, then two come along at once. We’ve written about Rod Johnson speaking in Dublin on Tue March 11th. Now there is a full day Spring Event in Belfast the day before (March 10th). And it’s sponsored by Momentum NI, so it’s free. And the Hilton Hotel is right beside Belfast Central train station, so it’s easy to get to from Dublin.

The full agenda is here (more details below), but given the importance of Spring to the Enterprise world, and the fact that the top four Spring guys are speaking, we reckon that it the Enterprise event of the year. The booking form is here.

Spring Ireland 2008

10th March 08:30 10th March 17:30
Hilton Belfast, 4 Lanyon Place, Belfast (Beside Central Train station)
SpringSource is proud to announce Spring Ireland 2008. Join us for a free one-day conference with presentations from the SpringSource team including a keynote from Rod Johnson.

Keynote: Spring into the Future – Rod Johnson

The Spring Framework began in 2002 with Rod’s best-selling Expert One-on-One J2EE Design and Development; one of the most influential books ever published on J2EE, Rod is recognised as one of the world’s leading authorities on Java and J2EE development.With the rapid expansion of SpringSource and move to new corporate HQ in Silicon Valley, this is a rare UK opportunity to hear Rod provide his views and explore the future of J2EE application development.

What’s New in Spring 2.5 – Sam Brannen

Sam is a lead architect at SpringSource and one of the most experienced developers within the organisation.This session provides delegates with an overview of the new features available in Spring 2.5. Specifically, highlighting the simplified and extensible configuration support provided via configuration annotations and new XML configuration namespaces, new Java 6 support, updates to Spring AOP, improved JDBC and JPA support, annotation-driven web MVC controllers, the new annotation-based TestContext framework for unit and integration tests and more!

SpringSource Professional Services – Greg Southey

Greg has built SpringSource’s UK Professional Services organisation into the leading Spring consultancy business in the world.

This is a brief resume of some of the 97 major projects completed by the UK Professional Services team in 2007. Delegates will hear about the business drivers behind some major development projects, the business issues faced and how they were resolved in partnership with the client.

Spring as a Full Stack Web Framework – David Syer

As Principal Consultant at SpringSource, Dave leads the way in the understanding of satisfying business requirements using the Spring Portfolio. His easy manner cloaks a prodigious knowledge of application development.

This session explores the “full stack” web framework trend and answers the question: how does Spring stack up? This session defines what a full-stack web framework is, then provides a fair technical comparison between a Spring-centric web development stack and the alternatives. Delegates will head about the feature-set of modern “full stack” web frameworks, and what Spring has that differentiates itself from the pack.

Spring in Ireland – Ian Graham

Ian Graham, Momentum, will explore the use of Spring in Ireland and introduce case studies from companies who are using the Spring family of products.

Round Table Discussion – Rod Johnson, Rob Harrop, Dave Syer, Sam Brannen & Greg Southey

Your chance to ask Rod and his team anything that’s on your mind regarding Spring!

What you do if you weren't doing your current job?

What you do if you weren’t doing your current job? While we all harbour dreams of running a magically profitable coffeeshop, working only 3 hours a day, what would you really do if you wanted a change of career?

If I wasn’t in IT , I’d be in Finance, on the basis of …

  • My original degree is in Business (with French). Somehow I got seduced into IT (you don’t hear that very often).
  • Both Finance and IT require their own set of knowledge and expertise. Once you’ve acquired that expertise, the work can be quite profitable, as not everybody can do it.
  • Both are quite strong employment areas within Ireland, with the IFSC being one of the easiest parts of Dublin to get to from Drogheda (think Trains).

Sadly (but very sanely), neither Finance nor IT is considered ‘sexy’. There again, you can’t have everything. They’re both quite hard to explain to your Mum – as far as she’s concerned , I work ‘in computers’. This is akin to lumping Salesmen , Mechanics, road sweepers and Michael Schumacher in a category ‘something to do with cars’.

However , this lead-in does explain the contents of the ‘what’s Paul Reading?’ list. All links are to Amazon. I’d recommend all the books with the exception of the last one – it was written by a newspaper journalist and the slightly jingoistic style reflects this.

  1. Economist : The City – a guide to London’s Global Financial Centre
  2. Freakonomics
  3. Java security
  4. How the City Really works

Google Spreadsheets Mean the end of Java

Or to be more accurate ‘Google Spreadsheets mean the end of Java as we know it’.
Google Spreadsheets Logo
Think about this. Who pays your wages Mr Java-Developer-who-has-just-had-a-couple-of-years-at-the-top-of-the-pile? Clients, or if you’re in a larger organisation , the business folks (i.e.’internal’ clients). Do you think any of them care about Java? Do any of them know what Java is? All they want is to get things done, quickly , and with as few mistakes as possible.

These business people would be happy to run their organisations on Spreadsheets. Do you remember the cartoon where Dilbert convinced the pointy haired boss that he could fly the plane using Excel? There’s more than a element of truth to this. I know of at least one US Fortune 100 company that (until recently) conducted most of it’s operations on little more than Microsoft Office and duct-tape. It worked, not very well, but it worked.

Until now , the next line would be ‘Excel (or any other type of Spreadsheet) is not secure / scalable / sharable / not web friendly’. That was until Google launched their Docs and Speadsheets. It’s an online version of Office with some spreadsheet functionality. Play with it a bit and you’ll see that there’s plenty missing. But this being Google , I’m willing to put good money on

  • (a) new features rolled out (think steamroller) and
  • (b) These Spreadsheets being massivly scalable / secure / sharable.

This being Google, there is also an API (developer page here). It’s got massive holes in it (e.g. you can’t yet use it to create a new spreadsheet). But when Microsoft bring out their version of online spreadsheets (and they will) not only will they clone the Google API (to get market share), they’ll need to go one further and introduce new features / remove the usage restrictions in order to compete.

So, secure, scalable, sharble online spreadsheets are here to stay. So lets take a look at Mr. (or Ms.) Pointy haired boss thinking about their new project:

  1. Hmm, I think we need to be able to gather which health plans our employees are enrolled in.
  2. OK, I’ll throw together a spreadsheet to show people what I want
  3. Before I’ll give to our friendly Java developer and let him ‘do’ a website from it.
    Soon I’ll just share this on Google.
  4. Great , Loads of people are now using it, I’ll just the (Ruby / PHP / Insert other language here) guy to add one or two extra features.
  5. Most Excellent. Why don’t we spin this off as a Web 2 company and sell it to EBay??

There you have it, Massively scalable , Highly secure websites (see Google Authentication API), without needing to know anything about EJB, JMX , JBoss, JDBC or any of the hard won knowledge that us Enterprise Java Developers have built up over the last 7-8 years. I’m exaggerating, but not much.

What do you think? Is Enterprise Java dead, or is Web 2 just another boost and a slightly different way of doing things for us Java people?

Other Java Posts from Technology in Plain English

Some other notes:

This article was originally published on the O’Reilly books OnJava Website.

Free Money

Imagine writing an ‘I owe you’ note that people would accept them for whatever you wanted – in pubs , as payment for your car , or even as a downpayment on a property in Dublin 4. In your dreams , imagine that people trusted you enough that they would then use your IOU’s in shops as payment. This is even better, as people now want more of your IOU’s, so you can write even more of them and get even more free stuff.

This is fantastic – you’ve got a massive free loan. Best of all many of these IOUs will get lost behind the sofa or eaten by the cat so you’ll never have to pay them back. Free money. The situation is not so absurd as it seems : The 50 Euro note in your pocket is an IOU from the Irish (and other European) governments. Technically we can reclaim payment, but what would be paid in?

FT Logo

What has sparked this ‘free money’ post is the news from the Financial times is that the Euro has edged out the US Dollar as the international IOU of choice. Given that the Irish Government is allowed to issue a fixed percentage of the Euro in circulation, the Irish Government now has even more money in it’s coffers ahead of the next election.

Applets coming back from the Dead?

If you remember Applets, then you are so 1990’s man. Right back before the dot-com boom , everybody was putting these Java programs in their web pages to do things simple things like display a financial chart (guilty as charged , my ‘lud). Oracle still uses them in some versions of it’s applications as a half-way house between it’s older desktop applications and a completely web only solution.

Trampoline Logo
Fast forward to 2006 and these ‘heavy’ applets have been replaced by light web pages using Ajax and Flash, which don’t require a user download to run. Pretty much anything a Java Applet can do can be done in Ajax (if you have enough time and patience). Ajax developers have been pretty inventive in using Flash to solve problems (e.g. allowing web pages to store information on your local PC), so it may only be a matter of time before this gets picked up.
It’s ironic then, that the very technology that is meant to replace it , ends up giving Applets a new lease of life. It won’t suit everybody or every problem. An ‘ideal’ problem for this to solve is where the user interaction, business logic or security requirements are at the ‘higher’ end of the scale. While you could solve it using Ajax (basically javascript on steroids), using Java Applets and having a nice IDE to develop and solve your problems in may pay off in the long run.

Jan of Trampoline Systems explains in more detail.

Using a database to store business rules

On a recent client project, we ended writing more than 300 business rules to reflect some of the financial decision making that they applied to an application (excuse me if I’m being suitably vauge with this). Many of these rules would be shared across applications. The rules engine used was the Java based JBoss rules engine (formerly known as Drools).

Obviously , these rules need to be stored somewhere. Most large organisations are comfortable with the idea of using a Database such as Oracle or Sql-Server to carry out this task. This article , for version 2 of Drools but also applicable to the latest release, shows you how to use a database to store your business rules.

What you missed at the Dublin Java Meetup

15 people turned up to talk about Java yesterday evening in the Forum Bar Dublin. Actually we talked about the World Cup (Ireland didn’t make it , but we had the choice of 6 other teams, including Jakub who was very selfish and had two!), Ruby (which is going to eat Java’s lunch in some areas), PHP, Naked Objects (write in Java, deploy in .Net), JavaKicks (a Digg like service, targetted at Java), .Net, Enterprise and Financial software, Spring , EJB 3.0, Apache, Blogging , Web 2.0 , Ajax , Service Orientated Architecture, Government Tenders, the Dublin Contract market and a little bit about Java.

Can’t make it along in person? We’re going to do a 2nd Java meetup, but in a ‘virtual pub’ for people who find it hard to get to Dublin. We’ll still carry on with the existing (face to face) meetups. As an extra, at 7.30pm the last Tuesday of the month we’ll have an open chat session (We’ll setup chat software on this or another website).

So, for 7.30pm on Tuesday 25th July I’ll be online. We’ll start with ‘What technologies should I be learning in the next 12 months?’ and let things develop from there.

Architecture? One size fits all

No matter what your system does , be it insurance , banking , online travel booking or telecoms, the chances are it does the following things:

  • Gets information from users over the web
  • Does some business processing on that information
  • Saves the information in a database.

At a conservative estimate , about 99% of Enterprise systems would fall into this category.

If so, why do you need an architect , when you can use our ‘one size fits all’ architecture diagram (below)?! Most non-trivial systems, regardless of the language they are written in (be it Java, .Net , or your language of choice) follow the pattern seen in this diagram.

3 Tier Enterprise Diagram

There are 3 Pieces to the Solution:

  • Web Browser (for the user / client).
  • Web and Application Server – carry out business logic.
  • Database Back End – to store data and ensure data integrity.

Within the Application Server (the middle bit above, which as Java Architects is the bit we are interested in), there are a further 3 tiers

  • A Presentation tier (or layer), which is mainly about talking to the user (it gets and sends requests to the web browser).
  • A Service layer , which is mainly about talking to back end such as databases, legacy systems (such as mainframes) and XML-Web services that we may use.
  • A Business layer, the ‘meat’ of the sandwich, where the ‘Value add’ is in terms of business processing and validation.

For each of these layers , your priority in building them are slightly different.

  • The Presentation layer is the bit the user sees. You want it to be fast and give a good impression to the client. Underneath, use a standard framework (link: pick your framework here) and then customize the look and feel.
  • The Service layer you want to work fast and well (e.g. no data faults), but then then forget about. Unless things go wrong, no user is going to complement you on the quality of database persistence! Use standard libraries for the entire layer.
  • Unless your company is a clone or franchise, the business layer in the system is going to be completely different. Aside from the user-interface , concentrate most of your project effort here as this is the core of what system does. We’ve written quite a bit about how to increase the value-add of the business layer (link to O’Reilly Technical Articles)

By the way , we’re only half-joking about the ‘why do you need an architect’ bit. We can be contacted here.