More from JAOO - Eric Evans on DSL

(JAOO was a few weeks ago now, but still not written all my notes up as I just started on a new project which is always an intense and busy time)

The last session of the DSL track was from Eric Evans of Domain Driven Design fame, in my view one of the very few must read software books. To some extent this session felt like a bit of 'reality check' (my words). Eric spoke about the process of knowledge crunching that happens between the business people and the developers, and that business people are not necessarily very good at abstractions. This reminded me of my question during the Intentional Software session - doesn't this interaction between developers and the business people add value? Eric made the point that this knowledge crunching is how we evolve our languages, can we get unambiguous shared understanding without this process? Eric gave an example to illustrate just how hard it is to get a nice clear abstraction for even well known domains such as date and time. He made the point that these languages require careful design and it is a specialist activity, who are the best people to do this?

For me one key parallel between what Eric spoke about and some of the earlier DSL sessions is the idea of developers becoming language designers which is a (very?) non-trivial task. I wonder if the key here is the ability to evolve these DSL's over time so we can get feedback from the business and make the changes they need quickly. So we make the creation of the DSL easier by using an iterative process, I think this approach is implicit in several of the different techniques for DSL I saw presented at JAOO and it's an idea I find attractive as a firm believer in Agile methodologies.

JAOO - 'The Role of DSLs in Software Factories'

In this session Steve Cook from Microsoft spoke about DSL's and software factories.

Steve showed how visual studio could be used along with the DSL tools to graphically create new languages and then graphically 'write code' in those languages. What the demo did highlight are the issues around changing a DSL, what happens to code created in the previous version of the language? In a couple of cases Steve had to delete all the code he'd created, this was a shame because I think a lot of people came away remembering that rather than what might be quite a powerful technique. In fact it wasn't until a remark someone made later that evening that I saw past these issues.

This reminds me of the problems around managing message schema change, in the real world you don't get to delete the previous versions of a message, you have to find a migration path that allows backwards compatibility. So what happens once a DSL has been used to create a significant amount of DSL 'code' and we need to make changes to the language? Do we assume we designed the language correctly the first time and that we'll never need to change it? Business changes, the market changes, I think our DSL's are going to need to change as well.

Back to the session, the demo was all within visual studio and was largely graphical, I'm always wary of graphical tools, you often find you need expensive specialist skills or that the last 20% of the problem requires "customization". Whenever I've gone down the path it's always ended up with the generated code being dumped and us rewriting things test first, still there's always a first time but we underestimate the cost of customization at our peril.

To be honest I was a bit disappointed with the session because we didn't really get into the details, I also found myself wondering if the tool will cope with real world complexity better than some of the previous versions of visual studio features. Anyone who has tried to do complex sql queries and then generate data adapters will know where I'm coming from. That said it made me want to find out more and I'll certainly add these tools to my list of things to have a look at.

JAOO - 'Intentional Software'

For me the most interesting session of the whole conference. Charles Simonyi gave a presentation and then demonstration of his Intentional Software. At the start of the session Charles reminded us all how little the software industry has moved in the last 30 odd years, got to say I agree, it's embarrassing how ready we are to re-learn lessons of the past and kid ourselves we've invented something new. Our industry really hasn't grown up yet.

He went on to talk about what we are actually trying to achieve with software, to capture the intent of the business user, could we find a way to bring the language we use to do this closer to the user? The key idea is that developers work on generators, these take a DSL and turn it into executable code, the DSL being tailored to make it easy for business users to express their intent. So the effort switches away from developers writing code which is meant to implement those intents onto working on generators. Charles compared the way we work now to cryptography, taking something we understand and then encoding it. Given some of the code I've seen over the years I can kind of see where is coming from!

One thing which occurred to me was do we risk losing something by weakening the dialogue between the developers and the business users, I've often found the back and forth between the two groups a good way to spot inconsistencies and logical problems with a users perception of what they want and how they think their business processes work. I asked a question along these lines, Charles was very honest and said he'd not really considered it, but on the whole he seemed to think it was a risk worth taking. I guess I'm not convinced just yet, I always remember a project where if we'd not questioned how the users thought something worked we'd have had invoices going in infinite approval loops.

Anyway on to the demo - the tool we were shown seems to be pretty mature and slick. Charles was able to show us multiple perspectives of the same method: as code, as mathematical symbols, as a spreadsheet, etc. He was then able to change the code in any of these perspectives and have that change visible in all the different projections as the change was taking place in the underlying AST. The unit test perspective really caught my attention. This is technology I'd love to get my hands on!

Overall the session left me wanting to know more with a whole bunch of questions and things I'd like to try. Unfortunately no one asked what timescales they are running to and when we might see this in the wild. (doh!)

JAOO - 'DSL implementation at compile-time via syntax extension'

This was the first session I attended on the DSL track, which after concurrency is the thing which most interests me at the moment. As before just some comments and observations on the session, if you want more detail start here. This session was presented by Laurence Tratt who told us about a technique for building DSL's which are compiled at compile time, as opposed to being interpreted at runtime. He's built a language which demonstrates these concepts and this has been used in some real commercial projects - something which always gets my attention.

Laurence started by talking a little bit on our approach to parsing languages, we basically use algorithms from 30 years ago, better approaches have been found since but we seem to have stuck with the older ones due to perceived performance issues. In his work Laurence has used the Earley Parser which apparently makes things a lot easier - having written some simple compilers I'd certainly appreciate something easier and simpler.

The approach his converge language takes to a DSL is to embed the DSL into the rest of the code, effectively as a Macro. (Hmmm, LISP?) Converge uses the same parser for the DSL's as the main language, so all the converge syntax and features are available for the DSL as well. It also means we need to do much less work to implement a DSL. Laurence showed some real code, one key feature is that the macros generate converge sourcecode (at compile time) which in turn is compiled along with the rest of the program. Features of the macro's allow this code to easily interact with the rest of the program. Having tried to acheive something similar with c# 2.0, where I ended up having to generate byte code, I can see how much easier being able to generate sourcecode is.

Overall I really liked this presentation, I've not really done it justice here, I'm determined to see if something similar could be done in c# 3.0.

jaoo - software transactional memory

Now back in London after the JAOO conference, the next session I attended was 'Back to the Future: Functions + Transactions' presented by Satnam Singh. The key concept he presented was Software Transactional Memory which uses concepts we are familiar with from database transactions. Effectually concurrent access to the same memory is allowed but we check for consistent state before we write back any results, if the underlying memory has changed (someone got there before us) then we abort the transaction.

It's a nice idea as among other things it avoids a lot of the locking issues around normal shared memory models. The code showed was a functional language with monads used as the implementation of STM, this took me way back: my final year project an uni was about adding monadic input/output in gofer. Of course, as with the previous talk, it's not clear when or how these ideas will come across to mainstream languages. Satnam went on to show us how these idea's could be used for composing more complex interactions.

Some concerns I came away with was the retry model, so if the memory has changed and we abort the transaction when do we retry? Can we get into a situation where we retry forever? I'm sure they've got these things covered, and I just need to read up on the subject more.

On a final note there was a question about how we think about parallelism. So one school of thought holds that we find concurrency hard to deal with because we have been trained (via school and then uni) to think in a sequential way. Satnam seems to be in the other camp, that we instinctively think in a sequential way and hence we struggle with parallelism. I'm not so sure, but if I look at the abstractions we have for dealing with parallelism in mainstream languages I can easily understand why people find it difficult.

jaoo - scoop

JAOO

Currently at the JAOO, what a great conference and some amazing speakers. The next few postings are observations and notes from the sessions I attended, mistakes and misunderstandings are all my own work. I'll try and include links so you can find out more about the topics as well.

Firstly "SCOOP: Concurrent Programming" which was presented by Bertrand Meyer and was part of the concurrency track. I guess no surprise he mentioned preconditions, I do like the idea and often find myself thinking in a 'contract first' way - still not convinced very complex preconditions are manageable, I'd still think it easier write unit tests to express those behaviors.

Anyway SCOOP seems to offers a clean and elegant model of concurrency, although still based on ideas we are familiar with. Effectively they do locking at the object level and use method arguments to specify objects over which a method needs to obtains a lock. I like this idea because it's very clear and avoids the horrible synchnoised(lock) I've seen all too often in java code, at the very least we should give these (artificial) lock objects meaningful names.

It seems the language (an extended Eiffel) is able to then run this code in a concurrent fashion, I'm guessing there must be a lot of clever math going on in the compiler to avoid deadlock, schedule the locking, etc. It also seems having the locking at the object level is more likely to make us think harder about what should actually be in a class. One thing that did surprise me was the lack of object migration, I immediately found myself thinking of fault tolerance and scaling, I need to read more on SCOOP before jumping to conclusions though.

One question which came up was that if we need to block until all arguments of a method are available (i.e. we have the lock on them) is that inefficient. The answer seems to be getting a answer as late as possible and lazy waits.

Interestingly Meyer seems against Active Objects, I've certainly seen lots of problems in making the same object reentrant on event callbacks, perhaps a symptom of trying to do an event based architecture while still thinking in request/reply terms.

This is the first post

..just to get things going