• Discoverability Visible
  • Join Policy Restricted
  • Created 07 Jul 2014

article:1910

Can Graduating Students Design Software Systems?

Contributions: Carol Zander

The work is based on written designs produced by near-graduating seniors.
The research question asked: how do students design when they are at the
end of an undergraduate computing program?

The designs were collected under stringent interview conditions as part
of a larger multi-national, multi-institutional project that looked at the
approach students take to software design.  It included 314 subjects from 21
institutions in the US, UK, Sweden, and New Zealand.  For the design task,
subjects were given a one-page design brief that described the behavior of the
desired system: a "super alarm clock" for college students.  Subjects were
given as much time as they wanted to perform this task during which they
could ask questions of the researcher.

To organize this data for analysis, we categorized them into groups of similar
designs using a data-driven approach.  We grouped designs based on their
semantics. Based on this approach, we identified six categories:

Nothing - Designs with little or no intelligible content.

Restatement - Designs that merely restate requirements from the task
description.

Skumtomte - Designs that add a small amount to restating the task.
(The Swedish word Skumtomte refers to a pink-and-white marshmallow
Santa Claus, a traditional Christmas confection.  It looks like there is
something there, but it is only shaped and colored marshmallow fluff.)

First step - Designs that include some significant work beyond the description:
either a partial overview of the system, or the design of one of the system's
components, such as the GUI or the interface to the database.

Partial design - Design provides an understandable description of each of the
parts and an overview of the system that illustrates the relationships between
the parts (incomplete or superficial or not completely described).

Complete design - Designs show a well-developed solution, including an
understandable overview, part descriptions that include responsibilities,
and explicit communication between the parts.

Our categaorization lines up with Soloway's phases on teaching design:
  Phase 1: Understand problem specification
  Phase 2: Decompose problem into programmable goals and objects (decompose the
           problem and identify the solution components)
  Phase 3: Select and compose plans to solve problems (components are composed
           to form a working whole)
  Phase 4: Implement plans in language constructs
  Phase 5: Reflect-Evaluate final artifact and overall design process (look
           back over what has been done and learn from both the successes
           and failures)

The overall performance of the near-graduating students was quite negative:
-- 21% of the designs were restatements or less
-- 41% of the designs were Skumtomte
-- 29% of the designs were in the First step category
-- 9% produced reasonable designs (Partial design or Complete)

Over 60% communicated no significant progress toward a design.

Considering other factors, we found that while most students were between
20 and 27 years old, age did not affect performance. The number of languages
the students knew was also not a factor. There was a positive correlation
between the number of CS courses taken and the category of the design.
The number of females in the study was small, but they did better with 46% of
their designs in the top three categories, while the males had 36% of theirs
in the top three categories.  This difference is nearly all attributable to the
females producing relatively more "First-step" designs, and the males producing
relatively more "Skumtomte" designs.  The males created all the designs at the
extremes, in the "Nothing" and "Complete" design categories. There was a
positive correlation between the number of computer science courses taken and
the category of the design.

An overview is of great importance for a good understanding of the design
produced. The design brief explicitly stated that "someone (not necessarily
you)" should be able to work from the design produced.  Quite significantly, a
number of designs in the Skumtomte and First step groups are long relative to
the substance that they communicate.  Despite considerable effort put into the
design, these students end up with little. If we could understand what it was
that these students were trying to achieve, we might better understand what
they consider significant.

Author 1: Anna Eckerdal Anna.Eckerdal@it.uu.se

Author 2: Robert McCartney robert@cse.uconn.edu

Author 3: Jan Erik Mostrom jem@cs.umu.se

Author 4: Carol Zander zander@u.washington.edu

Author 5: Mark Ratcliffe mbr@aber.ac.uk

Article Link: http://portal.acm.org/citation.cfm?id=1121468&coll=portal&dl=ACM&CFID=9122936&CFTOKEN=32438184


: Back to 2009 Spring/Summer Issue Vol. 4, No. 3

: Back to List of Issues

: Back to Table of Contents

Created on , Last modified on