... Dramatic change in the testing protocol ...

Discussion in 'MCSA' started by MattewH, Feb 2, 2005.

  1. MattewH

    MattewH Guest

    Q&A: Microsoft Improves Certification for IT Professionals with
    Performance-Based Testing

    An innovative new testing system for Microsoft Certified Professionals
    uses simulation technology to more effectively evaluate the real-world
    skills of IT professionals.

    REDMOND, Wash., Feb. 1, 2005 -- The Microsoft Certified Professional
    (MCP) program is one of the most successful certification programs in the
    industry with over 1.7 million individuals credentialed all over the world.
    The MCP credential is for professionals who have the skills to successfully
    implement a Microsoft product or technology as part of a business solution
    in an organization.

    Microsoft wants to offer certification that helps assure MCPs are
    experienced and qualified to help with their IT problems. But traditional
    multiple choice tests don't always do a complete job of testing IT
    professionals' ability to perform practical, real-world tasks end to end. So
    the Microsoft Learning team has developed a new system of testing that
    evaluates on-the-job skills of an IT pro.

    Reinforcing its commitment to the MCP community, Microsoft is about to
    incorporate performance-based testing in all core Windows 2003 exams.
    Through simulated environments, IT professionals taking exams for
    certification will now be required to demonstrate their knowledge of
    products and processes they may encounter on the job. This dramatic change
    in the testing protocol is designed to increase the value and credibility of
    the MCP credential.

    To learn about the new testing methodology, PressPass spoke with Al
    Valvano, Lead Product Manager with Microsoft Learning, and representatives
    of the two exam delivery partners participating in the program -
    psychometrician Paul Jones, an expert on testing with Thomson Prometric, and
    Randy Trask, VP of Market Development for Pearson VUE.

    PressPass: What is performance-based testing?

    Valvano: It's testing by doing. That's the definition used by the
    Performance Testing Council, and it's a good one. It tests for an
    individual's ability to perform a task as opposed to recalling information.
    Think of it in terms of driving a car. If everybody who got their driver's
    license only had to take a knowledge-based written test without the
    behind-the-wheel road test component, they would not be proving their
    ability to drive a car. The driving test adds validity to the written exam.

    Training in IT today often employs lab-based exercises to reinforce
    learning. It's evolved to the point where we use a lot of simulations. And
    now we want to insure that when an individual takes an exam to be certified
    in a particular technology, they're being tested by doing. It's a validation
    of their ability to accomplish given tasks.

    Jones: With only multiple-choice items in an exam you can't adequately
    address certain constructs, which is the knowledge in the person's head that
    you're trying to assess. You leave out components of the ability that you're
    trying to measure. For that reason, simulation nicely complements the
    multiple-choice exams.

    In multiple-choice you have four or five items from which to choose.
    But in a performance-based item, you have free play in an environment that
    works like the real world. You solve a problem and you show how to get from
    a starting point and navigate through a process. In addition, you must show
    how to manage the whole problem-solving effort and demonstrate the results.
    What performance-based testing adds is the components of self-management,
    monitoring, problem solving and planning a solution - which gives the
    testing program enhanced validity.

    PressPass: Why is Microsoft incorporating performance-based testing
    (simulation) into its certification program now?

    Valvano: Because we've found that performance-based testing produces
    more skilled candidates. And we're hearing from both Microsoft Certified
    Professionals and IT hiring managers that there is greater benefit to them
    in performance-based testing over traditional testing. We're always looking
    for ways to add value for our customers and including simulations in our
    exams increases the value of certification to the MCP community.

    Testing with simulations raises the bar by requiring candidates to
    actually perform tasks to accomplish a specific goal. Candidates have to
    prepare for certification by really using the products. Memorization won't
    cut it - you've got to develop the necessary skills. And that will help
    differentiate you from other professionals and improve your career
    opportunities. Hiring managers gain confidence that our certifications mean
    that potential employees can do the jobs they need them to do.

    Jones: The good thing about testing with simulations is that it
    stimulates candidates to understand how to do things. And if that's your
    focus as a student, your preparation is going to help you in the workplace.
    Exam preparation is more appropriate to what you're trying to learn. In the
    measurement literature, that's sometimes called "systemic validity." That
    is, it's pushing the whole certification system in the right direction.
    Candidates are focusing on the right kind of training and preparation.

    PressPass: Is Microsoft taking any kind of unique approach to
    performance-based testing?

    Valvano: We're using a very robust simulation technology. Most other
    programs that use simulations as an integral part of their testing
    experience are for high-end certifications. They're typically live
    applications. In other words, you come into a lab and you're given a server
    and some hardware and specific problems to solve. You're probably evaluated
    by an observer as well as with some automated scoring.

    The problem with live application testing is that has limited scale.
    It's very difficult to roll out on a global basis. So you're limiting your
    audience. The Microsoft certification program has much greater reach, with
    thousands of people taking exams every single day all around the world. Now
    simulations will be available to everybody that can currently take our exams
    and they're offered in eight languages: English, simplified Chinese, French,
    Spanish, German, Korean, Japanese and Brazilian Portuguese. Everybody who
    wants to attain a Microsoft certification will have the opportunity to have
    the same innovative testing experience.

    PressPass: Where do people go to take the certification exams?

    Valvano: Customers can take the exams from either of two exam delivery
    partners, Pearson VUE and Thomson Prometric. The simulation exams are going
    to be available at more than 6,000 test centers around the world.
    Implementing simulations into exams that are globally available has proven
    to be a challenging endeavor for us. We've made the commitment to
    performance-based testing and working with our partners, we've assembled an
    incredible team to do it.

    Jones: It's no cakewalk to do performance testing. It's more
    complicated to develop and more expensive in every way. It sounds really
    great to people who have never done it but any company that is implementing
    performance testing technology wants to be sure that it's being targeted at
    a place where there's going to be a good return on investment.

    PressPass: How are you getting the word out to IT professionals about
    the enhanced testing program?

    Valvano: A page will be posted for our customers on the Microsoft
    Learning Web site describing the simulations and the exam technology. In
    March, they can also access a demo of the simulations and preparation
    guidance. The information is also going to be on the secure MCP Web site
    that is only available to those who are credentialed and in the monthly MCP
    newsletter. And then there are the thousands of customers currently going to
    Certified Partners for Learning Solutions (CPLS) for training and
    certification. We've already communicated with the CPLS network about our
    testing improvements and provided them with resources including videos to
    talk about simulations with their customers.

    PressPass: Which tests incorporate simulations?

    Valvano: We're initially introducing simulations in the core exams for
    both our Microsoft Certified Systems Engineer (MCSE) and Microsoft Certified
    Systems Administrator (MCSA) certifications. These two exams - 70-290
    Managing and Maintaining a Microsoft Windows Server 2003 Environment and
    70-291 Implementing, Managing and Maintaining a Microsoft Windows Server
    2003 Network Infrastructure - will be offered with simulations by March 31,
    2005. Simulations will launch in many other exams throughout 2005. Look for
    a release schedule on the Microsoft Certification Web page in March.

    PressPass: Is there any incentive for existing MCPs and other
    experienced IT professionals to take the exams with simulations?

    Valvano: MCPs who have already taken a non-simulated version of 70-290
    or 70-291 are not required to take the revised test but they may choose to
    refresh their credentials with the performance-based criteria. Retaking the
    exam with simulations can help an MCP refresh his skills and perhaps make
    him a bit more marketable.

    PressPass: If a person has already completed some of the exams leading
    to certification, do they need to retake a version of the test with

    Valvano: If they have passed 70-290 or 70-291 by the launch date of
    March 31, they don't need to retake the exams. But if they have not yet
    taken these exams they must take the version with simulations. That's the
    only form of the test that will be available after launch.

    PressPass: What's the best way to prepare for a performance-based

    Valvano: First, it's important to note that we will still have a range
    of item types to measure the full spectrum of a subject: case studies,
    multiple choice, hot area, drag and drop, and others that an MCP must know
    thoroughly - so hitting the books is still something test takers will want
    to do.

    For the performance-based items, we suggest looking at the "Skills
    Measured by this Exam" section of the exam prep guidelines. Make sure that
    you can perform the tasks noted in the objectives such as "Monitor, manage,
    and troubleshoot access to files and folders."

    Simulations will generally be associated with objectives that require
    the candidate to perform a specific task. Spend time in the environment
    setting up situations that are called out in the exam objectives. Lots of
    interaction in the environment will significantly contribute to a
    candidate's ability to understand the environment and accomplish specified
    tasks. So practice, practice, practice!

    Trask: Certification exams have always been about demonstrating that
    you have what it takes to perform a job. This really means that once
    certified, you should be able to hit the ground running in a new job. If you
    have the confidence you can get behind a computer and get the job done -
    you're prepared, because that is exactly what Microsoft is going to ask you
    to do!

    PressPass: How will this performance-based testing initiative impact
    IT organizations?

    Valvano: IT professionals who achieve certification where
    performance-based testing is part of the exam have demonstrated real world
    competency and the ability to perform tasks required by a specific job role.
    This makes IT professionals better qualified to resolve problems faster and
    that translates into real cost savings and improved technical readiness.
    Performance-based testing hones and validates skills. The IT organization
    benefits because with highly skilled IT professionals it is better
    positioned to provide a higher level of service to its internal and external

    Trask: It's all about instilling confidence in IT managers that
    Microsoft certification is a true and meaningful indicator of someone's
    ability to perform a job. Whether used as a barometer for smart hiring, or
    with existing staff to increase the likelihood of having successful IT
    projects, like an Active Directory deployment, adding the simulations will
    give IT managers the added assurance that Microsoft certification can be
    used as a factor in making decisions.

    PressPass: What's it like to take one of the exams with simulations?

    Valvano: It's a "wow" experience. I've taken a lot of exams throughout
    my life - from school, into college, and then in my professional career -and
    this is a very different thing. Simulations incorporate many domains of
    skills in a single question. You're in a simulated environment and the
    simulation records all the steps you take to solve the problem. Your score
    is based on the path you take and the outcome you achieve. It's directly
    connected to what you do on a day-to-day basis in IT. What's really being
    tested is: Do I know how to do this?

    Trask: Microsoft is to be applauded for including real-world
    simulations within their certification exams. Demonstrating that you can
    complete tasks offers a level of assurance we believe IT managers will
    welcome with open arms.
    MattewH, Feb 2, 2005
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael T. Anderson
    Michael T. Anderson
    Sep 29, 2005
  2. MattewH
    Feb 2, 2005
  3. Nod

    Dramatic skies in Photoshop?

    Nod, Jul 27, 2003, in forum: Digital Photography
    David Eppstein
    Jul 27, 2003
  4. Ryadia@home

    Dramatic evidence - film v digital

    Ryadia@home, Jun 20, 2005, in forum: Digital Photography
  5. Metamucil

    America's #1 comedian makes dramatic film debut!

    Metamucil, May 27, 2007, in forum: Digital Photography

Share This Page