Session 8

Data Sources & Analysis

Boulmetis & Dutwin, Ch6-7; Emison, Ch4; Taylor-Powell & Steel; GAO Quantitative Data Analysis Introduction

Topics:

• Four levels of data: nominal, oridinal, interval and ratio.
• Approaches to data collection.
• Program objectives - who, what, when and how.
• Descriptive and inferential statistics.
• Emison's basics of "content practices."
• Sources of data:(Examples)
  • Survey: collecting standardized information through structured questionnaires to generate quantitative data. Surveys may be mailed (surface and electronic), completed on-site or administered through interviews, conducted either face-to-face, by telephone or electronically. Sample surveys use probability sampling that allows you to generalize your findings to a larger population, while informal surveys do not.
  • Case study: an in-depth examination of a particular case—a program, group of participants, single individual, site, or location. Case studies rely on multiple sources of information and methods to provide as complete a picture as possible.
  • Interviews: information collected by talking with and listening to people. Interviews range on a continuum from those that are tightly structured (as in a survey) to those that are free-flowing and conversational.
  • Observation: collecting information by “seeing” and “listening.” Observations may be structured or unstructured.
  • Group assessment: collecting evaluation information through the use of group processes such as a nominal group technique, focus group, Delphi, brainstorming, and community forums.
  • Expert or peer review: examination by a review committee, a panel of experts or peers.
  • Portfolio review: a collection of materials, including samples of work, that encompass the breadth and scope of the program or activity being evaluated.
  • Testimonial: a statement made by a person indicating personal responses and reactions.
  • Test: use of established standards to assess knowledge, skill, or performance such as a pen-and-pencil or skills test.
  • Photograph, slide, video: uses photography to capture visual images.
  • Diary and journal: recording of events over time revealing the personal perspective of the writer/recorder.
  • Log: recording of chronological entries that are usually brief and factual.
  • Document analysis: use of content analysis and other techniques to analyze and summarize printed material and existing information.
  • Action cards: use of index cards on which participants record what they did—the “action”— and when they reached their goal; primarily used in self-assessment.
  • Simulation: use of models or mock- ups to solicit perceptions and reactions.
  • Problem story: narrative account of past, present, or future situations as a means of identifying perceptions. Using fictional characters externalizes the problem situation.
  • Creative expression: use of art forms to represent people’s ideas and feelings through stories, drama, dance, music, art.
  • Unobtrusive measures: gathering information without the knowledge of the people in the setting; for example, examination of record books to identify areas of greatest activity; unobtrusive observations of playground interactions to record aggressive behaviors.
• Goals in choosing a data collection method - trustworthy, authentic and credible.
• Generic types of quantitative questions.

Homework:

1. Complete the questions and excercises found at the end of Chapters 6-7. of The ABC's of Evlauation. Likewise complete the discussion questions found at the end of Chapter 4 of the Emison text.
2. Distinguish between descriptive and inferential statistics and their uses.
3. Distinguish between nominal, oridinal, interval and ratio and describe under what circumstances you might use each type.
4. Identify the guiding principles of quantitative analysis.
5. Identify the measures of central tendency for ordinal, interval and ratio data.
6. Identify measures of spread or distribution for ordinal, interval and ratio data.
7. What is a "normal" distribution?
8. Identify measures of association for ordinal, interval and ratio data.
9. What is a "point estimate" and how does it differ from an "interval estimate"?
10. When is a Chi-Square and an ANOVA statistic employed and for what purpose?

NOTE: Submit your assigned homework in World format as attachments to Canvas email and submit this homework - complete with APA citations and references - to the instructor by no later than 5 pm on the last day of Session 8 (as designated on the course schedule page that is linked above).