Re: cert discussion

[Gene Rackow <rackow@mcs.anl.gov>]

Elizabeth;

zwicky made the following keystrokes:
 >Here are some questions of interest about certification:
 >
 >	0) Which of the following best describes your attitude towards 
 >certification?
 >		a) It's the best idea since sliced bread.
 >		b) Sure, why not?
 >		c) It's an OK idea, but there are more important things SAGE
 >		could be doing.
 >		d) It's fundamentally meaningless and silly.
 >		e) It's actively harmful to the profession.
 >		f) It's morally wrong.
 >
 >	If you answered E or F, please justify your answer. If you answered
 >anything else, SAGE might as well go on working on certification.
I'd vote d, but have heavy leanings towards e.  All that certification means
is that you've passed a test.  Some of the best systems people I know of
can't deal with tests.  Put them in the real world and they shine.  Also
passing a test does not mean you know the subject.
As to it being harmful, it's just another method of judging people.  If
companies come to depend on "certification" as a method of judging if someone
is "qualified", then they are looking in the wrong area.  Instead of 
certifying people, maybe a better thing to do would be to put together a
series of guidelines that employers can use to develop their own
qualifying proceedures to see if someone is capable to do the job they
need done.  SA covers a very broad range of subjects, systems, and
specilties.  A ceritifcation test is either going to be to general to be
useful, or so specialized that people would need to take several to
show what they really know.

 >
 >	1) How much would you be willing to pay for a SAGE certificate?
 >(This to some extent controls possible testing options -- reliably
 >grading essay questions is *expensive*.)
As little as possible.  Again this is going to depend on what the test
is really made up of, and how many there are.  Also, how long is a
certificate good for?   Before really determining what the cost is
going to be, it should be determined as to how many tests one is
going to need to take.  $500/test would not be to bad if it was only
the 1 test you needed to take.  $500 would be crazy if there were 10++
of them to take.
  basic os standards
  networking
  kernel workings and device drivers
  i/o systems
  database
  security
  ....
  OS specialties? solaris2, aix, irix, hpux, osf/ultrix, linux, other??

In many cases, what certification is aimed at is the entry level person.
These are the people that don't have lots of money to get started in the
first place, and do not have a resume/vita that speaks for them.  They
need a certificate that claims they at least get in the door.  Having
to much to cost involved with that is going to limit them too.
  
 >
 >	2) Which of the following best describes your attitude towards
 >multiple-choice testing for certificates?
 >		a) Sounds like a fine idea to me.
 >		b) It's pointless, but since the whole thing is pointless,
 >		why not?
 >		c) Not only is it pointless, it's offensive and possibly
 >		immoral. 
b.  But in place of why not, why?  If you are going to have testing, about the
only way to go is going to have to be multiple choice.  Essay tests become
way to subjective.  While the answer may be correct, the writting style
may not carry a convincing arguement.  English is not the primary language
of all admins.  Plus, like you said above, it's expensive.
 >
 >	3) What process should SAGE use to develop and test the tests? Who
 >do we know who knows how other people do this?
To come up with a real testing situation, for each test there are going to
need to be in the area of 5k, preferably more questions that make up the
test suite.  The "system" would need to choose X questions from that set
to make up the test.  Anything less becomes an effort in memorizing the
test questions, not the subject.  A friend of mine just passed her
nursing tests.  There it was a computer based timed system with a random
number of questions.  Some of it was "essay" based, with a rather
good AI behind it.  Depending on answers you gave, it chose other
questions.  It could tell if you knew the subject or not.  Development of
such a system took a very long time, and the questions were fine tuned
over years of giving the same sort of tests by hand.

Also, how relevant are the questions.  Things vary from OS to OS, and in
some cases release to release.  The formats of files vary widely, etc.
While I don't have the full format of every option in the AIX version of
the /etc/passwd /etc/security/passwd stanza files memorized, I know the
ones that are important, and can find the others in moments.

Sun has, or had, a certification program for Solaris.  IBM has one for AIX,
as well as having specialty fields on top of that.  Then there is MS.

 >
 >	4) Which of the following models do you prefer?
 >		a) You must be a SAGE member to get a SAGE certification.
 >		b) You don't have to be a SAGE member, but we charge 
 >		extra.
 >		c) It doesn't matter at all whether or not you're a member.
It depends on what the group is attempting to do.  Personally I'd think the
certification board should be a seperate issue, if there is going to be
one.  Forcing membership is not the way to go.  It is clear that there are
people that are not happy with Usenix, so forcing membership upon them
is not going to make them happy.  Do we want real unbiased certification,
or a multi-card carrying club?

Side issue:
 This is something that is rather confusing.  If it costs $X to give the
 tests, why should a member of a group get it for <$X, just because
 he is a member.  If paying $Y gets you membership to the club, and $Y
 has built in overhead to give the discounts then OK.  Is there enough
 overhead in the Sage membership costs to justify the discounts to something
 that currently does not exist?  If there currently is that much excess, then
 why aren't the costs of membership currently lower?

For example AARP has tons of members and a very strong lobby, yet if you
went to the majority of the membership, the ONLY reason they are members
is for the discount card they get for being a member.  If you ask them about
issues that AARP lobby is promoting, the strongly disagree with such
standards, yet their name is on the roster.  While I don't think Usenix is
at that level, forced membership is not pleasant.  I wouldn't take a
position where I needed to be a union member as I've been there in the
past and found too many things that being a union member added weight to
their cause that I disagreed with.  I don't want to be in the same boat
with Usenix.
 >
 >	5) Can you give one example of a testing/certification program you
 >think SAGE's should be like, and why?
No.
 >	6) Can you give one example of something you think is horribly
 >wrong with an existing testing/certification program, which SAGE
 >should avoid doing?
Some of the issues are above.  One thing that I mentioned already in other
mail on this thread, is that according to "certification standards"
I'm passed the tests for FCC Class 1 radio/telegraph systems about 20years
ago.  It takes very little to keep that certification valid, and the
fact that I haven't touched or even looked at one of those systems in 15
years does not matter.  In our field, the sytems change even more often
than radio.  most radio transmission stations are using the same transmitter
towers, amps, etc they have for many years.  Most computer systems get
replaced in 3 to 5 years.

 >
 >	Elizabeth Zwicky
 >	zwicky@neu.sgi.com
 >

   Gene Rackow                    email: rackow@mcs.anl.gov
   Math & Computer Science        voice: 630-252-7126
   Argonne National Lab           FAX:   630-252-5986
   9700 S. Cass Ave. / Argonne, IL  60439