1.

2.

• Problem Approach: I.E. client has housing problem, drug problem etc. is good for diagnosis
• Quality-of-Life Approach: focuses on end states and attempt to move client toward one of a number of desirable end states.

3.

Numeric Counts

• used to measure client conditions, status; behavior

Standardized Measures

• used to measure client feelings, attitudes, etc.

Level of Functioning Scales

• measures client, family functioning or condition

Client Satisfaction

• measures client perceptions

4

Intermediate Outcome Performance Measures

• Numeric Counts
• Standardized Measures
• LOF Scales
• Client Satisfaction

Ultimate Outcome Performance Measures

• Numeric Counts
• Standardized Measures
• LOF Scales

5.

• Step 1: Use focus group which includes a representative group of stakeholders to discuss how to assess quality outcomes
• Step 2: As many outcome performance as possible should be identified
• Step 3: Group should arrive at a consensus on the best two outcome performance measures for the particular program

6.

• In many cases cause & effect relationships between program activities and outcomes cannot be established
• In other cases cause & effect relationships may exist to the extent that programs contributed, and maybe even contributed significantly to an outcome
• However, in most cases we avoid the word "cause" and speak of "program contributions" toward a change etc.

7.

• Definition: Data that enable evaluative judgements to be made about social problems in a community or state (Miller, 1991)
• Social Indicators as Ultimate Outcome Measures: Examples
• Benchmarking in Oregon (Oregon Options)
• Goals 2000
• Healthy People 2000

8.

• Utility: percentage of information considered useful
• Validity: measures what it purports to measure
• Reliability: measure produces same result repeatedly
• Precision: capturing incremental changes
• Feasibility: implemenability of measure
• Costs: relative start up & maintenance costs of measure
• Unit Cost Reporting: ability to generate cost per outcome data

9.

• Include demographics and characteristic data related to client flow
• Nominal measures relating to client flow
• critical events that reflect,
• an undesirable occurrence that an agency is trying to prevent or avoid
• a desirable occurrence that the agency is attempting to achieve.

10.

I&R

• intermediate output performance measure (unit of service): one referral
• output with quality dimension: one appropriate referral
• outcome performance measure
• intermediate: one client receiving assistance

11.

Counseling

• intermediate output performance measure (unit of service): one hour
• output with quality dimension: one hour with counselor of record
• outcome performance measure
• intermediate: one client demonstrating improved behavior
• ultimate: one client no longer needing service

12.

Developmental Disabilities: Behavior Management

• one client reported to be exhibiting maladaptive behaviors
• one client with a current behavior checklist in his/her file

Aging & Adult Services: Adult Day Care Services

• one client returned to independent living status
• one client prevented from entering a long-term care facility
• one client entering a nursing home facility

13.

• SEA reporting standards promotes use of numeric counts
• Government Performance & Results Act of 1993 also promotes use of numeric counts
• Governmental human service programs also prefer numeric counts

14.

• Utility High
• Precision Low
• Validity Low to Medium
• Reliability Feasibility High
• Cost Low to Medium
• Unit Cost Reporting High

15.

• Standardized measures are validated, reliable and normed pre- post-test measures used to assess quality-of-life changes in clients

16.

• Barthel Index
• Caregiver Strain Index
• Index of Clinical Stress
• Beck Depression Scale
• Folstein Mini-Mental Assessment

17.

• Population
• behavior
• attitude
• problem
• intra-personal or interpersonal functioning
• development
• personality
• achievement, knowledge, aptitude
• services

18.

19.

• Number of clients demonstrating measurable improvement
• proportion of clients who demonstrate measurable improvement relative to the number of clients receiving a full complement of services
• number of clients demonstrating clinical improvement (comparing pre- to post-treatment)

20.

• proportion of clients who demonstrate clinical improvement to the total of clients who completed treatment
• Proportion of Clients who achieve a target level of improvement
• proportion of clients who achieved a target level of improvement to the total number of clients who completed treatment

21.

• Utility Low to High
• Validity High
• Reliability High
• Precision Medium to High
• Feasibility Low
• Cost High
• Unit Cost Reporting Low

22.

Before & after client assessment tools designed for use with a particular human service program that attempts to capture an important dimension of client functioning

 

23.

• LOFs are typically focused on only one dimension of client functioning
• Consequently, LOF’s are typically used in combination to assess multiple dimensions of client functioning
• LOFs are typically (but not always) ranked from very low to very high

24.

Conceptual Framework: Dimensions of functioning and descriptors anchoring the assessment scale must be based upon a thorough understanding of the program, clients, and the underlying problem

Developing Descriptors:

• should describe levels of functioning
• should discriminate between different function levels
• should accurately reflect client behaviors

25.

Respondent Considerations:

• Developing Scales with Client in Mind
• observe client
• ask client
• get information on client from third party

26.

• Identifying Behavior to be Observed: Involves becoming familiar with scales first, then observing clients
• Site of Observation: Behaviors change with setting so choose setting or settings
• Frequency of Observation: Utilize a discrete of settings in which the respondent uses to complete an LOF scale.

27.

• Avoid Influencing the Observation Situation: Influencing situation make the observed behavior less typical of the client
• Reliability of Observation: Procedures should be setting for making observations and collecting data, so that different observations are comparable to one another

28.

• Step 1: Select the Functions to Be Rated: All functions selected should be expected to change as a consequence of participating in the program
• Step2: Select the Number of Points on the Scale: Minimum of 3 points (still problematic) to a maximum of 8 points. Usually use 5 point scale
• Step 3: Write the Descriptors: Should be based on typical, observable verified client behaviors

29.

• Step 4: Field Test the LOF Scales: Use on a small scale to obtain experience in observation and coding
• Step 5: Test Reliability of LOF Scale: When used across many observations of the same client or client group are comparable observation results obtained.

30.

• Utility Low to High
• Validity Medium to High
• Reliability Medium to High
• Precision Medium
• Feasibility Low
• Cost High
• Unit Cost Reporting Low

31.

• Generates personal attitudes, opinions, feelings and choices
• Typically scaled from "Not Helpful at All" to "Very Helpful" or "Extremely Helpful"

32.

• Utility Medium
• Validity Low to Medium
• Reliability Medium
• Precision Low
• Feasibility Medium
• Cost Low to High (start-up)
• Unit Cost Reporting High

33.

• Failing to relate performance measures to a program’s mission
• Relying too heavily on existing data
• Excluding stakeholders from the process
• Selecting too few quality performance measures

34.

• How often should performance measurement data be reported? At least annually, semi-annually to quarterly is more functional
• How much time does it take to collect and aggregate performance measurement data? If performance data is collected regularly and automated then time required ranges from 10 to 20 hours to collect and aggregate a year’s worth of data

35.

• Inputs: Financial Resources in $; personnel in FTE
• Outputs: intermediate (time, material); final outputs (service completions)
• Quality: client satisfaction %; Outputs with quality dimensions
• Outcomes & Ultimate Outcomes: intermediate outcomes (numeric counts of quality outcomes

36.

• Cost Efficiency Ratios: Cost per intermediate outputs, & final outputs per FTE; cost per final output
• Cost Effectiveness Ratios: Cost per intermediate outcome, costs per intermediate & final outcomes per FTE, and cost per ultimate outcome.