Re: Definition of a functional spec

["Steven J. Owens" <puff -at- NETCOM -dot- COM>]

     I started a kneejerk reply to this post based on the first few
paragraphs, then I realized Ben is at least as familiar with the topic
as I am, though we have different points of view.  So I'll leave out
the bulk of my reactions and throw in a few general comments.

Ben Kovitz writes:
> [...] The first two questions to answer when writing any technical
> document are Who's going to read it? and What job are they going to
> do with the information?
>
> For some reason that I'm sure is of profound anthropological and
> psychological importance, there's a deep, human temptation to
> avoid asking these questions when writing scaffolding documents
> like requirements and functional specs (or whatever people call
> them).  The temptation is to write to an abstract standard of
> correctness, not to real people and a real situation.

     I'm not sure how much of that is related to writing and how much
of it is simply part of the whole conundrum of the software
development process.  Most programming is done on a "craftsman" level
at best; places like the SEI (Software Engineering Institute) have
spent millions of dollars trying to figure out the general issues, and
entire fields like CHI/HCI/Human Factors and Usability Engineering
have arisen to address the core issue of figuring out how to make the
software "right".  Which is usually what a functional spec is
supposed to be about (but not what a functional spec is really about
- see below).

> The problem that the document needs to solve *here and now* never
> seems to be addressed, only the abstract problem of conforming to
> some pre-existing standard that may or may not be relevant on this
> one project.

     You expect people to be reasonable, think clearly and consider
the issues before they express their needs?  What planet are you from?

     Seriously, I'd suggest that any writer who gets handed a simple
request to "write a functional spec for this software project" has
either an uphill battle in front of them, or an opportunity to really
affect the software development process at their company.

> that software development for many years has been and remains in
> a state of conceptual confusion.  People don't yet know exactly
> what problem programmers solve or how to characterize it.

     Yup, and most programmers don't know how to articulate what they
do and what they need - because the skills, talents, aptitudes, and
the very mindset needed to both do the work, manage the work, and
communicate about the work are often at opposite ends of the spectrum.
This is not merely an issue of needing a multitalented person - the
effort to cultivate the proper state of mind to perform these
different functions is often contradictory.

     I was going to go off on an extended exploration of this topic,
but I'll skip that and simply comment that there are a lot of
parallels between the writer's state of mind and the programmer's
state of mind.  Often with a different emphasis, but writers have a
lot more in common with programmers than either typically realize.

     In that sense, programming is more like when you're truly trying
to write creatively - not just trying to figure out how to say it
(which is a lot of what technical writing is about), but trying to
figure out what to say.  As the recent thread on interruptions showed,
this issue is difficult to work with at the best of times.

> If you ask an electrical engineer to design a transformer for you,
> you know exactly what to tell him: the input voltage, the output
> voltage, the frequency, and the power consumption and heat
> generation characteristics.  More importantly, if you don't know
> everything to say, you can just ask the engineer.  He can tell you
> exactly what information he needs in order to design the
> transformer.

     Sure, and if you ask a programmer to design a method/function/
subroutine/module to sort strings he'll know exactly what to ask you.
Ask the electrical engineer to design a missile control system.
That'll get you a response much more analogous to asking a programmer
to develop an application.

     I continually get the feeling that the key fact 80-90% of the
people miss - including most programmers - is how totally unlike
almost every other human endeavor programming is, on an organizational
scale.  People keep trying to come up with metaphors to get a handle
on software project management.  The closest thing to software project
management is any large engineering effort to design and engineer
completely new systems to address completely new needs.  Not simply
building/designing a VCR or a cellular phone from scratch, but
designing a new kind of video recording and displaying technology from
scratch, or a new kind of communication system from scratch.

> In other words, don't expect to talk to the programmers and testers
> once and then have a template that you can reuse.  You need to talk
> with them about these things and make major adjustments to the
> document on *every project*--not refining a template to an ever more
> perfect, universal standard, but making each document fit the unique
> needs of each individual project.

     Truer words.  All you can do is get a better handle on the
process of developing a functional spec, not on how the textof the
spec itself should be written and organized.

     In general, in theory the specification process is straightforward:

     You start by developing high-level design requirements.  This is
where you define the scope of the project domain and the general goals
("we're writing a database system that needs to be industry standard
compliant, highly reliable, and run fast enough to be feasible for use
in point of ale systems").  In reality, most people just skip right
past this.

     Then comes the functional specification, where you nail down what
you want (but not how it should be built), testing it against the
requirements defined in the high-level design document.  In theory this
is developed as a cooperative effort between marketing, in figuring out
what is needed for the product, and engineering, in figuring out what
can be built and for how much.  In reality, this is the substance of
a haggling process between engineering and marketing, and the process
is usually dominated by one side or the other.

     Typically in the industry, companies are either technology-driven
or market-driven.  This results in a corresponding dominance by
engineering or marketing.  In theory, the way it's *supposed* to work,
upper management keeps the two in balance and helps to find the
balance point that results in a successful product and a succesful
company.

     Once a first draft functional spec is hammered out, the
programmers go off and figure out how to break that up into pieces of
work they can individually and cooperatively work on.  As they work on
them, they figure out that some things are easier, some things are
harder, some things are better done one way than another.  The spec
shifts to fit - keeping the spec up to date is an important part of
the process.  In practice, this almost never happens.

     Most often people just don't pay attention to keeping the spec
updated, or one side or the other howls bloody murder when a
significant change has to be made, or the organization goes totally
overboard in this direction and the spec and the software development
process becomes fairly moribund and sluggish (this is what it's like
at most really large IS organizations).


     Let's take a look at this from another angle.

     An important point to understand is that, software engineering
being a much broader domain than civil engineering, for example, the
spec is *not* like a blueprint.  By the time it's accurate enough and
detailed enough to qualify as a blueprint, the program is almost done
(and in fact that's a pretty good software develpoment methodology).

     A spec is not like a contract or a work order that defines
exactly what product is to be delivered and how.

     A spec is not the answer, it is the question, and it is a very
complex question that evolves as the asker and answerer hash out what
exactly the asker is looking for.

     Think of it this way; the conversation between marketing and
engineering is a very complex one.  You start writing things down to
keep track of them as you figure out what you're talking about, and
building an outline of the conversation.  Occasionally, as the
conversation progresses and you start to have something resembling a
document, you rewrite the whole thing from scratch.  Finally, you
start to fill in the individual points in the outline, and as you do
this you come across changes that need to be made in the outline
itself; things that are easier than you thought, things that are
harder, things that need to be organized differently than you first
thought.

     Typically in the industry, people impose an arbitrary and usually
incorrect demarcation, a point where they say "this is what the spec
is, go build it." To a certain degree this is necessary and
appropriate.  At some point you have to start limiting the scope of
the conversation, or you'll be there forever.  A good, albeit brute
force, tactic to do that is to draw a line around the conversation to
date and say it's done, now on to the next stage.  This is necessary
because typically neither side in the conversation has a really good
idea of the other side's point of view.  This makes controlling the
conversation and keeping the conversation on track and making forward
progress difficult.

     If you have a specification team that is lead by good marketer
who has also been a good engineer, and a good engineer who has also
been a good marketer, then you're in software development heaven and
it's time to stop smoking that funny tobbaco.


     There are a few excellent books written on these topics for
programmers.  The ones I usually recommend are Steve McConnel's _Code
Complete_ and _Rapid Development_.  I also highly recommend Jakob
Nielsen's _Usability Engineering_.

Steven J. Owens
puff -at- netcom -dot- com

 From ??? -at- ??? Sun Jan 00 00:00:00 0000=