These compromises usually manifest as divergence within a code base (we’ll branch this, just temporarily) or as duplication (a quick copy and paste for now). All of these are invariably made with good intentions and pretty much always to support the business – but for one reason of another they never get fixed again.

Add layer upon layer of these compromises and one factor is almost certainly inevitable – there will be a big refactor or rewrite project.

Now that we’ve got to this point and we have some understanding of the history of how we got here we seem to have 2 options (3 if we include carrying on as we are). So which one?

The Big Rewrite

This is the one that always seems so appealing to everyone. It gives us an opportunity to fix everything that we have learnt from past mistakes. We now understand the system much more and can enable all the features everyone wants.

But The Big Rewrite approach tends to fail. As developers we underestimate the size of the system (even though we may know the system very well) and we overestimate our abilities. This, coupled with the businesses need to still sell and use the system means we end up in a race condition. We are racing to make the new system as feature complete as possible while still supporting, maintaining and extending the old system. Often with dedicated teams on each system (with the ‘elite’ team being the big rewrite team).

I’ve yet to see a big rewrite successfully pay off. I have however seen a big rewrite cause many more problems than it attempts to resolve, often resulting on some customers on the new but with less features system and other customers on the older but more feature complete system. Sometimes this divergence goes even further (a branch of the not yet complete big rewrite system).

Refactor

As you can probably guess, this is the approach I favour. Rather than rewrite instead refactor the system as you go, but even then this approach can sometimes end up in a similar situation as the big rewrite approach. One of the main reasons for this is that to successfully refactor we need to make sure we have a crucial enabler in place – the migration.

If we do refactor, or we do manage to pull off the big rewrite (and I’m sure it does happen) then what? We now find we have all these customers on our old system and we want to migrate them to the new system. If we can’t do this then, after all, what’s the point?

But the migration itself is now seen as a big risk. We have to migrate customers data across to new databases, invariably stopping customer access to the system as we do it, which means picking a convenient timeframe to do it in (weekends being the obvious choice) and what do we do if it fails?

I’ve seen this big migration at the end cause many good intentions to go sour. What has happened in the past is that many customers never migrate across, for whatever reason, and now the new system and the old system continue on – dividing further and further apart as they go resulting in yet another code base to maintain.

Migrate Now

Which leads on (eventually) to the point of this post. Before attempting either rewrite or refactor, see if there’s a way of actually migrating to the new system now. Yes, right now.

It seems implausible as we haven’t even started on a new system yet but it may actually be much simpler than you realise (and in many ways is more of a mindset change than anything else).

In a web system recently, the simplest way to achieve this change would be to introduce a proxy (or what we were calling a ‘shim’) which would do nothing more than to proxy requests down to the underlying system.

But actually, it did much more than this.

The mindset change

By migrating now we are now on the new system. We have migrated all our customers with no loss of service and it just works (assuming it does that is). All we are doing is proxying the requests through.

But we now don’t have multiple teams, one on support and one on the new system, we just have the one new system team as we don’t have any customers on the old system.

We’ve also avoided the big migration at the end – with all the inherent risk, and more importantly, without leaving some customers on the old system, so no divergence. Now we have one code base going forward, with all our customers already using it.

What now?

Ok – so technically nothing has changed. We still have 2 or more systems underneath even if we have one system from above. So what is the next step? One next step would be to start identifying static content that is shared between systems. This could be just a help page, or a contact us page, or something of that nature. Whatever it is, we can now promote that duplicated static content up to the ‘shim’ itself.

This small change forces us to make some decisions. Where do we put this static content? How do we serve it up through the ‘shim’? Hopefully these will be fairly easy to solve issues. That’s the point though – think small. Small baby steps.

Once it’s promoted up we can switch the proxy to redirect to the new shim provided content. If there are any issues we should be able to redirect the proxy back to the old systems so we are only ever a small step away from rolling back to how it was before.

Once any kinks are ironed out and it is going through the proxy to the shim hosted content ok – delete the now redundant static content from the old systems. By deleting the redundant content we’ve, very slightly, reduced what we need to port across.

More baby steps

Next, look at ‘almost’ static content. Some content – again, something like help pages or contact us pages, that is almost shared but not quite the same. Look at how a basic templating engine (string template, spark or nvelocity for instance) could be used along with a basic model to hold the changes. Now again deal with the same small questions such as where the templating engine will sit, how will the content be served up, where do we get the model data from (and what is the identifier for each of the ‘below the shim’ systems). These should again be simple enough questions to find an answer for. Again, the point here is still very small changes.

Now again, when this almost static content is promoted to the shim, and the redirect through the proxy to the shim hosted content is working, delete it from the old systems.

At this point we’ve managed to nibble at the edge of our problem but by now the mindset change should have been reinforced. We are looking at how we can promote small parts of the existing systems into the shim itself, and once complete, deleting those small parts from the old systems. If we ever get a problem, we should only be a small redirect away from using the old system again until we resolve any issues and can redirect to the new parts of the system.

A bigger nibble

I was going to continue this on but its already got way too wordy as it is (and it’s close to midnight) so I will continue these thoughts on in another post. Now, given my current trend in posting, this may be in a few months time but bear with me

I’ve had my eye on KooBoo, a .net (and more importantly, an mvc) based cms which looks very impressive. I’m also looking at ruby on rails using RubyMine as the IDE. Both good excuses to try out something different. But who am I kidding, the problem with our website isn’t technology based. What it lacks is design and content.

So now I’m procrastinating by posting this instead. At least it’s content

We plan on this being a regular monthly event and the AltNetBeers format is a great way of fostering some intense discussion, all driven by the people that come along. I’ve not attended many of the London one’s (actually, only 1, which was part of the Alt.Net UK conference weekend) but it was fantastic and I hope that everyone that attends to tomorrow nights evening will have just as much fun.

We (The Agile Workshop) would also like to facilitate some open coding days as our new offices (more on that later) has the space to be able to do this. I’m aiming to discuss this at some point tomorrow evening to get an initial idea on who might be interested.

So please, if you are free tomorrow evening, come along. It should prove to be a fun evening.

Even when getting to the ‘Red’ failing test step, it is important to check that it’s the right ‘red’. It’s far to easy to produce a failing test which fails for the wrong reasons. A quick check that it’s failing for the reasons you expect is worth it.

If you don’t check the failing test and skip straight to the implementation to make the test pass then you’ll keep getting a failing test. When this happens there’s a natural tendency for tunnel vision to creep in as you go deeper and deeper into finding out why it’s not working before an ‘ahha’ moment where you realise where the error is.

I speak from experience after being at that point far too many times. Not that I want to admit it as each time is a personal head slap moment as I realise how insanely foolish I’ve been.

So please, take the time to stop at red and check it’s the right shade of red before moving on to green. Forget the time it will save in the long run, and instead just be glad that you won’t feel the same shame and humiliation that I’ve felt in the past!

I’m sure there is a ‘proper’ way to edit the IIS metabase – but we created the folder structure that represents the action under the main folder, set up the auth on that and then deleted the folder. The meta-base details persits and now everything works fine!

The intention is for this to be very much an audience driven session. Ronnie and myself will be taking on the driver navigator role of a pair and, with the navigator gathering the audiences opinions, direction, questions and feedback we will all drive the story forward.

Our goal is to take a hands on session like this and to capture and expand on the ‘ahha’ moments that come from driving with TDD. A lot of TDD and other techniques, processes and methods are very subtle and work together as a whole. Taking just one of these in isolation often fails to bring real benefits. We hope that by learning with the audience in this way that we can explore how these elements compliment each other.

Given that one of us is called Ronnie Barker I would also hope that we can inject a little humour into the session as well – even if it is a ‘And it’s good night from him’ at the end

So I thought (as did others) that one of the main culprits of the performance problems is down to straight execute of the generated sql statement as opposed to using sp_executesql. Using execute means that every single variation of a statement (the ‘parameter’ values themselves) would generate a new execution plan rather than using an existing cached one. Using sp_executesql means that the sql statement itself uses a cached execution plan and then the values themselves are passed in as named parameters.

So I set about modifying the proc to use sp_executesql. This involved building up a list of the parameters and their data types to pass in, as well as passing in the parameters themselves. It wasn’t as easy as I originally imagined but was fun (in a perverse way).

To validate the change I wrote a unit test which made numerous calls to the proc passing in different parameter values. In this way I could get some timings of the proc before I made the switch and then timings for afterwards as well.

But when I ran the tests to gather the timings the sp_executesql tests run SLOWER!! Usually not much slower but sometimes it is. This is against a shared dev database so part of the timings vary depending on what others are doing to the database but even so, over many of the runs I made the original execute version of the proc ran faster as the new sp_executesql version.

I have managed to speed up the proc though – but through the use of an index on two columns – although I still need to test this – I’m just surprised at how my initial assumption (or belief) in sp_executesql over execute has been dashed right now.

I’m pretty sure sp_executesql is still the way to go for various other reasons – but would ultimately like to use NHibernate and get rid of 95% of the existing procs but that’s not really an option right now.

Full Details here: http://bbits.co.uk/tddmasterclass

bbits are holding a raffle for a free ticket for the event. To be eligible to win the ticket (worth £2395!) you MUST paste this text, including all links, into your blog and email Ian@bbits.co.ukwith the url to the blog entry.  The draw will be made on September 1st and the winner informed by email and on bbits.co.uk/blog

The past 3 months have been a learning experience with all the usual highs and lows. On the low side we haven’t been able to grow according to our original plan but on the high side we have been involved with some really interesting projects and some joint ventures.

Our new offices are located within the Social Enterprise Centre in Cambridge which is part of a social charity for incubating start-ups. This is something close to our hearts and is something we want to enable ourselves at some point. The joint ventures we have already entered into are a sign of this and even agile itself eludes to this with the close collaboration with clients.

Once we have settled into our new offices we would like to start focusing on our grander vision of providing world class training, mentoring and coaching agile approaches and of course the ‘doing’ side of it of working directly on delivering projects for our clients.

So as always, if you are ever in the vicinity then please stop by and see us.

The talk starts with a very staged performance involving Richard talking to a pre recorded self over a webcam support call. It is a tad cheesy but heck, I like it.

The talk then moves on to the 2D perspective manipulation. This makes it very easy to take a 2D image and manipulate it in 3D space. This would make doing a carousel effect very easy and it also works with all content. This is proved by taking a business application form and quickly transforming it by spinning the form round and adjusting the perspective. The form still works but typing text in backwards and at an angle isn’t exactly usable

GPU acceleration is then discussed and demonstatred along with pixel shading. The GPU acceleration only applies to video (and on the mac only applies to full screen video). I thought the GPU acceleration also applied to pixel shader effects in Silverlight 3 as well. Wouldn’t mind digging into that a bit more.

However, the pixel shader effects are still cool. Way beyond me but the thought that I could find a water ripple effect and bring it in is a very tempting prospect.

Next up is caching and compression. The caching and compression story in Silverlight 3 sounds very compelling. Currently the Silverlight 3 install is actually 40k LESS than the Silvberlight 2 install even though it has much more crammed in. This is quite an achievement by the silverlight team.

The xap files are now much more compressed as well so delivering your silverlight files to your end users is going to take less bandwidth. The figure quoted is a 20% reduction in size.

Also, your own assemblies can now be cached. In Silverlight 2 it would only cache Microsoft assemblies so this is good news. Microsoft has done a good job of compressing as much as possible.

It seems you also get control of the actual caret in a text box. And to demonstrate this, a 1 pixel wide image of a ‘carrot’ for the caret . I know – more cheese but it got the chuckles.

Moving on to styling and the Silverlight 3 story is much better. It feels like there’s real separation in regards to the design elements which has also caused the most questions from everyone so this seems to be a big win for everyone. Styles can also be applied programmatically (and probably with databinding directly) so it is possible to change the style on the fly which will be good for providing more dynamic theme support. Styles can also be applied located in external assemblies so providing customer specific styles will be much more streamlined.

After a break for pizza we move on to the out of browser experience. This is one of the areas where I think Silverlight 3 will be strongest and can’t wait to get the chance to implement this in some of our projects. If this can be linked with a good synchronised storage solution (like Microsoft synchronisation services) then this will be fantastic as Silverlight 3 will detect when the network is connected. Occasionally connect computing here we come.

It also detects new versions so not only is deployment simplified, but upgrading is also simplified. I’m not sure on the extent of this (for instance, whether you get the chance to hook into the upgrade before and after to be able to migrate stored data). All in all it sounds very promising.

Deep linking also looks very nice so it is possible to link directly to a point within silverlight. This also enables back button support within the browser so no more hassle with the user clicking back expecting to just go back one small step and instead goes all the way to the start of the application (or beyond).

There was also a demonstration on how different silverlight islands on a page can talk to each other. This not only works at the page level but also between tabs and even between different browsers which is quite impressive.

In general the talk focused a lot on the presentation whizzy side of silverlight rather than on the line of business application side of it. Although silverlight is gaining more market share, flash is still the dominating technology for more marketing driven rich content so the spinning video type elements don’t feel as persuasive in that light.

Where silverlight will win more is the rich line of business side of things and also the out of browser based applications so in this regard silverlight is competing in the adobe air space more. It will be interesting to see how silverlight progresses in that space but I didn’t see enough from this talk that really emphasised that point.

All in all it was an enjoyable evening.