Idiots guide to Service Orientated Architecture (SOA)

Lost in the hype around Service Orientated Architecture (SOA) is the fact that the idea is really really simple. It’s all based on the idea that most applications (and that includes websites) are built either to be used by people , or used by computers. The pictures below (a free preview from the Enterprise Java Briefing) show what I mean.

In a ‘normal’ application, such as a online banking website, we need to remember what the user did last (are they logged in, what account are they looking at, are they in the middle of making a payment). If we didn’t , the user would get annoyed about having to repeat themselves every step of the way. It would also make for pretty complicated screens, to allow the user to enter all the information in one go. Instead , we allow the user to enter information in several steps, and remember where there are each time.

Soa Client

In an application designed to be used by computers, we don’t have to worry about this. We can force the computer to give us all the information required all in one go – username , password, bank account to take money from , bank account to give money to, date to execute transaction. For a computer , this is actually easier ; we make one call to our banking service and we are told it has succeeded or failed. It’s also easier for us to build our service:

  • Each service (transfer money, book flight , execute share trade) only does one thing.
  • Because each service ‘forgets’ after each call, we don’t need to worry about trying to remember what we were doing before.
  • Because we have no memory, services are very scalable; we can make several copies of the same service and put them in a pool. Any client can talk to any service – no waiting for a particular server to become available.

Soa Service

So that’s Service Orientated Architecture (SOA) : programs that do one thing (a bit like a function call to a method) exposed that other computers can call. So what’s the big deal? Like all good ideas , a simple concept goes a long way.

Take a look at the picture below. It’s like a Visio diagram, but in fact it’s drawn by the Eclipse Based JBoss IDE. It shows a workflow for an online commerce store – pretty easy to understand. This example uses JBoss Java Business Process Managment (jBPM), but companies such  Tibco, Cape clear and Oracle BPEL have similar products.

soa workflow

Here’s the clever bit; each of these steps is executed by one of the services that we talked about earlier. This means that if the business process changes (and it will), then all you have to do is re-arrange the diagram ; little or no coding changes should be required.

This abilility to mix , match, combine and remix services leads us to a lot of other good things (and we’re only scratching the surface here).

  • Because our services don’t have to run on the same machine, we can use SOA to create a distributed application. This is the concept behind the BPEL (Business PRocess Engineering Language)
  • Services tie well to Ajax and Web 2: Our Ajax web page or portlet can call as many services as it requires to get the job done (it’s one of the reasons Tibcom is sponsoring the open source DWR project)
  • We can call many services at once. If these this service calls are xml based ,or we send these calls as a message then we can filter, duplicate, pass and other distribute these calls as we set. These are the ideas behind Apache Synapse, Apache Servicemix and the  Enterprise Service bus (ESB) in general.

What do you think? Is SOA useful , or over hyped?

related posts

Advertisements

What would you like to see on an Advanced Java Course?

Most people building systems run into the following problems again and again:

  • How to capture Business rules , in a way that both the Business users and the computer can understand.
  • How to capture the flow of actions in a system, in a way that both Business users and the computer can understand. This flow is across multiple users, and may extend over days or weeks.
  • How to deliver a system to the user (e.g. Via the Web), but to give the user a rich interface , similar to what they are used to on the desktop.
  • How to maintain and enhance older systems , now that Java has been mainstream for more than years.
  • How to take advantage of the new Features afforded by Java 5 and EJB 3, and what business problems to the solve.
  • How to build components for reuse in all environments (Web , Enterprise, Command Line and Desktop).
  • How to map information in a Java System to and from a Database (Persistent Storage or Legacy System).
  • How to deliver value to the business at every point in the project.
  • How to use the many resources and solutions already available in the Java community.

It’s to address problems like these , that I’ve been asked to put together a Advanced Java training. It’s early days yet, but I’d like to get your input as to what you’d like to see on such a course.

Java Logo
Full details of the Advanced Java course are available on the knowledgebase / Wiki. Leave your comments here.

Update: I posted a similar question on the (technical) O’Reilly Books Java blog. If you’re interested in seeing the responses , click here.

Agile Architecture – a new blueprint every day

The best thing about doing presentations is the questions you get asked at the end. Apart from the stomach churning moment of ‘will I be able to answer this one?’ they give you a new angle on things that you may have always assumed, but force you to think of in a different way.

Take one of the questions after yesterday’s Enterprise java presentation at DCU. One of the topics mentioned in the final ‘putting it all together’ was the Agile and RUP (or other upfront design) methodologies. The question , coming from an attendee that was keen on using Agile , was ‘How do you do Architecture in an Agile project?’
DCU Logo

On the face of it this seems a contradiction. Agile in it’s most extreme form is ‘make up just enough design as you go along’, with automated tests to make sure changing things later is a relatively low cost and pain free process. In real life most projects are a balance between Agile and need some element of a more formal process (often trying to answer the question ‘how mucn is this going to cost?)’

So , how do you merge Agile with an upfront design process? It’s easier than you think.

Most systems built the ‘traditional way’ do not get all their design done in one go. They might start out with the best of intentions for phase 1 with a clean sheet but over the months / years people come and go, business requirements change and different phases try to deliver different things. Over time the original clean design twists and turns and you work hard to try and make it work out ok. Some of features you thought were key may end up getting thrown out as too complicated.
Now Agile architecture is a similar situation. Each weekly / monthly iteration is like phases on the larger project , with twists and turns that may be unexpected. The difference now is that you have a safety net comprised of your (J)Unit tests, to allow you to make radical changes if your blueprint ends up in a cul-de sac.

Yes, it is ok to have an idea the bigger picture and where you like to go with the design. Yes, a good architect will find reasons in the current release to build towards that design. And yes, a good architect may admit that some of the design features he / she thought were required weren’t actually needed. The difference between Agile and Upfront architecture is in when you find your ‘Don’t really need it’ point. With Agile , you find it just before you build it. With upfront design / architecture you find it when it’s already too late.

Business Process Management is Service Orientated Architectures Killer Application

Ismael Ghalimi has put it in a nutshell:

BPM is Soaps Killer Application

  • BPM or Business Process Management , is the art / science of capturing what your staff actually do in an IT system (and hopefully help them do their job better in the process).
  • SOA or Service Orientated Architecture is designing your system as a set of endpoints (e.g. Login, get bank balance, transfer money, logout). Most systems already have this functionality, although maybe not clearly laid out.

Ismael goes into more detail , but the idea is that BPM (think Visio Diagram) allows you to draw your workflow. Each step on the workflow is carried out put an action / endpoint provided by some system (using the SOA type design).

What comes after Java and .Net? Agents.

Most systems until now have been centralised : A bit like the old Soviet Union, everything is centrally planned. The trouble is real-life isn’t like that – it’s a market economy with no central control. There’s a story about a Russian Diplomat posted to New York in the 60’s. On a visit to a bakery he asked – who decides how many loaves are baked in the city? The answer is no-one – each baker individually decides how many to bake based on how many he sold the day before. Somehow (almost) everybody gets fed.Current OO systems are like the Russian’s view: everything is centrally controlled. Agents are more like New-York (or Dublin) city today – a place full of people (agents) acting in their own self interest. Somehow everything works ok. Economists have a theory that backs this up ; in general a set of people acting in their own self interest gives the best solution at a global level. Or, if you prefer it’s a bit like Ants. Individual Ants are stupid, but together they are clever enough to mark a trail to food and carry it back to the Anthill. It’s called Emergent Behaviour – simple programs combining to give the answers to complex problems.

How does Web 2.0 give a push to Agents? Before, Systems were standalone , and everything planned in advance. With Web 2.0 everything is connected and too complex to manage by one person. We need to look at what works successfully in real life. Just as Market economies overcame the ‘Command and control’ of communism, so Agents will overcome the Command and control of Objects. It may not be perfect, but it will be (slightly) better.

Will agents replace Java and .Net ? A sign that ‘the future is already here’ is that when you read the list ‘what makes an agent’ , you may go ‘but we’re doing that now’. Java and .Net have been around for so long now that it’s easy to forget the Object Orientated Programming (OOP) was once a radical new departure. It’s also easy to forget that languages such as C++, Visual Basic 6 and Powerbuilder were once ‘king of the hill’ and commanded respect from your colleagues when you mentioned your latest project was using them.

So what are agents? Compared to Objects :

  • Agents act in their own self interest , they may decline a request if they think it makes them better off.Objects always respond to a request.
  • Agents have their own thread of control , 1 for each agent. Objects may have their own thread, but most objects don’t. – Agents are pro-active, and seek to improve their lot , according to pre-defined goals.
  • Agents are ‘Coarse Grained’ that is, a system will probably have a few agents will a lot of normal , dependent , objects. It’s similar to the way Enterprise Java Beans are used : not everything is an EJB , and there a still lots of Plain Old java Objects.
  • Objects are designed from the start to work together. Agents can be written by different people , perhaps with widely different goals in mind.

Just like C++ was a procedural language with object orientated ‘bits’ attached, Agents are currently implmented in languages like Java , with agent-y bits attached. Probably the most useful set of bits is Cougaar. Cougaar is an open source project with a live community at Cougaarforge and an Eclipse based IDE. Cougaar gives you the basic infrastructure for creating and managing agents.

Of course , there’s nothing to stop you building your own agents. According to the above definition, most systems that have workflow tieing together entities making decisions according to their own business rules are not far off being agents. Especially when they have a scheduler (i.e. their own ‘thread of control’).

What do you think? Leave a comment below.

It's been very quiet over here (aka what has Paul been up to) – Enterprise Web 2.0

It’s all been very quiet over here , too quiet. And not just because of the hosting issues (the people at Netbunch, you know that I’m talking about you)

It’s been very quiet , because I’ve been very busy. On top of all this , we’re coming to the end of the year for the (part time) Masters at UCD in Dublin, so we’ve also got exams coming up. Thankfully it’s the last year, the downside being I have a dissertation to write.

Being a blogger , I’m not happy putting together a weighty tome that will sit gathering dust on a shelf. Instead , I want something that will solve some business problems , and that I can use as interesting content. So after much thought , the proposed title of my dissertation will be …… cue drum roll ….. ta-da!

Enterprise Web 2.0

Now, if you’ve talked to me , you know I spend a lot of my working day as an Enterprise Java Consultant , working for various banks. The idea is to take some of the Web 2.0 ideas (and you don’t need me to repeat them) and apply them to the sort of problems large companies have. Or , if you want the catchy subtitle , ” it’s all about sucking the knowledge out of people’s brains and putting it onto (ugly) websites”.

So an obvious topic to cover is the use of Ajax , which while big on the web at the moment , is going to be huge once companies realise what it can bring to their internal applications. The rest of the topics cover knowledge management (what is web 2.0 if it’s not about sharing knowledge), but also some tools and techniques that will all Enterprise Java (with all it’s robustness and scalability) compete with the nimbleness and tricks of Ruby.

Business Problem 1: How to present this information to people in a easy to deploy, but powerful way.
Solution: Update to Sun Java article – this one on how to do Web 2.0 / Ajax ‘right’ in Enterprise Java (i.e. not worrying about legacy code)

Business Problem 2: Where you have documentation, but don’t know how to find it.
Solution: Write up of the Red-Piranha Adaptive Search engine that ‘learns’ what the team wants , and finds more of it.

Business Problem 3: Where you have information in Excel sheets, but can’t do much with it.
Solution: Update to previous O’Reilly Articles on JBoss Rules – this one on JBoss’ ability to ‘run’ Excel Spreadsheets.

Business Problem 4: Where you have information that people ‘know’ , but that a machine finds it hard to ‘learn’
Solution: Simple Neural Networks using Joone, applied to a ‘real life’ business problem.

Business Problem 5: Where several people have to work together on a set of information , following a strict set of steps.
Solution: JBoss workflow, with a simple online example

Day 5 of the Exoftware Agile Course at UCD Dublin

If you’re very observant , you might notice that this post is about a week late. Still , here are my notes on the slides / nodes from the final day of the Agile course.

Previous Posts from the training course are:

Exoftware Logo

Junit Introduction

  • Junit
    Framework for writing automated unit tests (don’t need to use Junit
    to write the tests , but it helps)

  • All Junit
    tests extend TestCase

  • Individual
    test methods (pre Junit 4.0) : public void testSomething() method
    signature

  • Use
    assertEquals assertTrue assertFalse assertNull assertNotNull
    assertSame and fail methods for testing.

  • SetUp() and
    tearDown() methods called before and after each test method.

  • TestSuites
    (groups of TestCases) replaced by Ant and (Eclipse) IDE
    functionality

  • Run tests via
    built in Text / Swing test running , or more likely use IDE / Ant
    integration.

  • Organise via
    same or paralell folders

  • TestMyClass
    or MyClassTest

Mock and Stubs

  • Problem: some
    classes (Collaborating Objects) can be tightly bound to system
    resources (e.g. File or Database) or to a hard to test API. Another
    example is the observer pattern. How do we unit test these?

  • Solution: use
    fake object , pass it to class under test , allows testing of the
    class at a unit level.

  • Stub : fake
    object that uses hard-coded data , often following an API. Many
    ready made stubs available (e.g. For the JDBC libraries).

    • Replace
      expensive to create objects

    • Have to
      create all objects that production code interacts with.

    • Start in
      middle and develop outwards

    • One failure
      can ripple out / appear elsewhere.

  • Crash Test
    Dummy: type of stub that deliberately fails in order to test
    exception handling – simulate database crash or I/O full.

  • Self Shunt is
    where the unit test itself implements the interface (doesn’t work
    for classes) and gets callbacks from the class under test.

  • Mock Objects
    like stubs, except that they are intelligent enought to self-verify.

    • Mock
      Secondary objects (normally ones that we build) instead of API (as
      Stub does)

    • Outside in
      development style

    • What if mock
      implementation is incorrect?

  • Interaction vs. State: In
    state-based testing you check the tests by examining the state after
    the stimulus. In interaction based testing you check the right
    interactions were triggered by the stimulus – Martijn
    Fowler,
    http://martinfowler.com/articles/mocksArentStubs.html

  • Mock objects
    tend to test more the interaction between objects rather than a unit
    test of the object itself.

  • Stub rather
    than mock external API’s

Smells and refactoring.

  • Code smells
    are bad or suspect design decisions in code.

  • Not always
    bad , but should be looked at (often will involve a trade off)

  • Smells are
    often hard to understand , hard to change code.

    • Samples :
      duplication / long methods / poor naming / tightly coupled classes
      / switch statements /

  • Refactoring :
    Small , controlled changes to codebase , so that it continues to
    compile / tests run to improve the design. The behaviour remains
    exactly the same.

  • Seperate
    refactoring and adding additional functionality.

  • Easier with
    automated tools and Unit tests to confirm that nothing has broken.

  • Refactoring
    != Rewrite. Codebase is evolved , not thrown away.

User Stories

  • In some ways
    , equivalent to Use Case from Predictive methods (but with subtle
    differences).

  • Represent
    chunk of functionality that makes sense to the customer.

  • Can be
    represented as card / conversation / paper based documents (more
    predictive approach).

  • In Agile (as
    opposed to predictive) , shift the focus from writing to talking
    i.e. Get a true understanding of what the customer wants , even if
    they haven’t expressed it very well.

  • Lessen the
    importance of requirements

  • Support
    iterative development and participatory design.

  • Even a simple
    requirement has many possible permutations (that lessen the odds of
    getting it right.

  • Not just a
    generic user – but more user roles – groups of users that do
    different things with the system , depending on their experience ,
    task at hand etc

  • User Stories
    should be Invest:

    • Independent

    • Negotiable

    • Valuable

    • Estimatable

    • Small

    • Testable

  • Templace
    (FlashCard , as used by Xplanner)

  • Be careful of
    size

    • too big and
      cannot be used for estimated (break these down into smaller
      stories). Easy for Compound , but for complex stories might need to
      do some research and then break out.

    • Too small ,
      and it is not worth the admin of estimating each. Bundle them up
      into higher level task like UI improvements or bug fixes.

  • User Story
    should give us the acceptance tests