Dave's WebLog

Welcome to my WebLog. Hopefully this will be my little corner of the web where I can be honest and say what I want to say. Please bear that in mind. This is me.

Disclaimer: This weblog in no way reflects the views of my employer. This weblog, like any other opinion piece is intended to be read as such and not taken seriously at all.


I have found a new place to do my Blog (HERE - davespace.blog-city.com). I will try to keep this somewhat up to date so I can continue to post in other Blogger Blogs but there are no guarantees.


Arnold Ross: “Think deeply of simple things.”

Hanlon's Razor: “Never attribute to malice that which is adequately explained by stupidity.”

Gunther Rall: “So every airplane has some problems in some areas, and if you know it, you can overcome it.”

Location: Moncton, New Brunswick, Canada

Friday, July 30, 2004

Nothing like it..

There is nothing like the pressure and stress of post-install outage investigation. There's all that finger pointing, blaming, scurrying around and covering your butt - it almost makes it a sport. (except for the high salaries and there are not nearly enough fans cheering you on).

One of the biggest issues is the outage call. For us it involves about 20 managers from all levels of the organization on one call at the same time. Add to the mix an unhealthy helping of developers who are occupied with covering their butts and actually trying to solve the problem presented to them. The bigger problem is that most of the managers are pissed and they want someone's ass right then. Rather than wait patiently, stalking their prey, they demand to know right then who and what caused the outage and what can be done to fix it in the future.

I'm sorry. Piss off.

My solution to this mess is to have 2 calls going at the same time: the political call and the resolution call. The key is to have someone experienced in crisis management handling the in-between stuff. Let me give you an example. The outage today was caused by an unknown problem in the code that was installed last weekend. My boss and I are on the political call, fielding questions and blame as best we can. At the same time, the developers are working together through instant messenging and phone calls to investigate, troubleshoot and resolve the call. My boss and I on the political call intercept questions and orders to get the developers on the call and, instead, act as intermediaries between the two.

Q: How long until a fix is in place?
*pause as I ask over my shoulder to the huddled group*
A:15 minutes.

Q:What caused the outage?
*pause as I listen in on the converstation the developers are having and ask a quick question to someone who is involved in the discussion but not active at a keyboard or explaining something to the group*
A: Problem with one of the database tables and the load script.

This leaves the troublshooting team free of the politics of a dozen miffed managers and the fingerpointing that is sure to follow. It keeps the team focused and the information flowing through the pipeline in an orderly manner. Requests and responses down one managed channel.

Monday, July 26, 2004

Founding Fathers of Unix & C

It is always interesting to take a look back at the development of technology that is often taken for granted in the computer indistry today. Unix and C were both major accomplishments in an industry that was in it's infantcy but would grow and prosper under the guidence of it's early founding fathers.

For me, C was the first 'real' language that I began under some level of self study. I had cut my teeh during my high school years on the Commodore 64 using basic and some entry level of machine language near the end. C represented a milestone in my development as a developer, moving from a simple language into the realm of modern, complex and detailed development. Even today, rare 1st edition copies of the C Bible by Brian Kernighan and Dennis Ritchie bring a smile to my face.

Unix began with my early days in university as the only system containing the tools necessary to tinker around with the developing internet. It was a difficult OS to an uneducated beginner but soon enough it began to show it's strength and character.

If you have some time, check out this article at the Economist - a detailed history of the development of Unix and C, including some insite into the lives of those who worked their magic on machines that were so limiting when compared to todays modern machines.

Consider this post my effort to thank these fine people for thier effort. It does not go unnoticed.

Sunday, July 25, 2004

Install ... Completed :-)

The installation of our code completed yesterday; and arduous 12 hour process for me that started at 8:30 in the morning when the support phone rang and ended at 8:20 at night. Almost following the philosophy of some military commanders, I was the first on 'battlefield' and the last to leave.

All in all the installation went pretty well. Some install tasks fell out of place in the timeline causing some problems; the java makefile on production failed and we had to manually compile each code module separately; and a scattering of small bugs in some of the code appeared, indirectly causing other bugs in the install process to slow us down. My Java code also was installed, though functionality was not tested until late in the night (due to the location of the code and the other offices installed, it ended up not being part of the general DW install checklist).

Now, as I sit here, UCT is beginning. Connectivity and sanity testing has already been completed and it is now up to the end users to certify the code as installed and operating according to the specs. I expect some consternation today as things progress. This is mostly due to a lack of actual understanding on the part of the end-users as to what they are getting and how they are to operate it; like a caveman asking for some method of transportation to the stars and then immediately being placed in a rocket ship and told: Here you go. :-)

Friday, July 23, 2004

Software Install Weekend...

... I hate software install weekend. All the pressure comes due. It is time to put the project into the production environment and see what happens. In this case we have hundereds of modules (ColdFusion, Java and Unix code) and one application available to the end users was subject to a complete re-write.
For every single change to existing code there is the potential for, what I figure, 20 or so bugs to crop up directly or indirectly related to the change implemented. The number of bug goes up or down dependingon the quality or level of complexity of the code involved of course - up for bad code or high complexity, down for good code and low complexity. In this case we are talking about hundreds (if not thousands) of changes to some moderate-low complexity web code and a new Java module (3000+ lines of code) which a large amount of fairly complex business logic.
Needless to say, with the level of change and complexity of the code has me worried, agrivated by the fact that I seem to be 'in charge'.

Well, it's another start...

... let's see if I can keep this one up and running and maybe even show a little commitment.