Program evaluation and process improvement are related processes, with a shared goal of gathering data to answer some question related to how a program or process is going. In its most basic form, process improvement involves looking at how your clinic can do what it does even better. Specifically, it consists of utilizing a systematic framework for improving some aspect of an organization. Unlike process improvement, the focus of program evaluation is usually on the whole program, as opposed to just one aspect of it. So, if you wanted to examine how well your clinic/program is working overall, you might look at doing a program evaluation. For most types of program evaluations, they are typically done after the program has been stable for some time, meaning it is no longer adjusting the type and amount of services it provides. This module mainly focuses on process improvement, specifically how to apply process improvement process and tool to clinic optimization. One reason that we emphasize process improvement is that it tends to be focused on one aspect of a clinic at a time. This makes it a very good model for looking at how well some of the clinic optimization efforts are going, such as improving utilization of group therapy or decreasing wait times.
There are numerous benefits to a clinic having a robust process improvement process (see callout box on right). In general, using an ongoing process improvement process can help improve processes in the clinic (e.g., decreasing wait times), track new initiatives (e.g., increased use of group therapy), or improve outcomes (e.g., patient satisfaction or return to duty rate).
We recommend you follow an established model or framework when undertaking process improvement efforts. The module covers two process improvement frameworks that are commonly used in DoD settings, the FOCUS-PDCA and LEAN Six sigma. Since these frameworks involve a step where your team will need to identify the causes for a problem, the module also includes information on several common process improvement tools, including tools for conducting a root cause analysis (Handout: Tools for Process Improvement ). This handout introduces flow-charting, affinity diagrams and fishbone diagrams, as well as several graphs used in PI processes. We also include a Fishbone Diagram worksheet.
Since process improvement and program evaluation involve collecting data to answer questions, you or your staff may have questions regarding whether these activities fall into the category of research, and would need additional approvals within your command (e.g., IRB). We have two handouts that help address such questions, a FAQ document that covers common Myths About Process Improvement, and a more detailed document that covers the differences between process improvement, program evaluation and clinical research: Questions to Distinguish between Process Improvement, Program Evaluation, and Research Projects
The module also includes examples of how to apply these models to projects linked to the Clinic Optimization toolkit, such as:
- Decreasing wait times for SPEC or FTR
- Expanding the use of group therapy
- Conducting a Clinic Gap Analysis
- Expanding the use of EBPs for key disorders
- Setting up a Sub-clinic for PTSD
- Utilizing BHTs to support EBPs
There are several sources of data for informing PI efforts. One key source is your clinic’s own data on appointments. The handout for how to request and analyze your clinics appointment data from your Command’s M2 representative (the Guide to Creating Clinic Level Reports in Excel ) provides step-by-step guidelines on how to create graphs and reports of many different kinds of data. This handout is covered in depth in the Metrics module.
Another data source for process improvement that you have available is information on EBP Training and Utilization. Through using a simple one-page survey of providers once a quarter, you can assess levels of training, use, and level of fidelity to EBPs within your clinic. The spreadsheet that comes with this also produces pre-made graphs. This set of tools is covered under the Gap Analysis module.
Yet another data source that will help you with process improvement is the Patient Disposition Tracking Tool. It involves collecting data on dispositions using a one-page form (Behavioral Health Patient Disposition Tracking Sheet). This data is entered into an easy-to use database which can generate various pre-built graphs and reports. This set of tools is covered in depth in the Metrics module.
The toolkit also has a template for using FOCUS-PDCA framework to report out on the results of a PI project. It also has several completed examples of this template, each detailing how a clinic might have approached an improvement and documented the results. These templates are listed in the table below.
Relevant toolkit items:
| Item Name | Description | Audience |
|---|---|---|
| Evaluating the Clinic Optimization Process | Training presentation on how to evaluate clinic optimization. Differentiates between program evaluation, process improvement, and research. Provides examples for completing process improvement projects to evaluate the level of a clinic’s optimization. | Clinic Manager |
| Myths About Process Improvement | Fact sheet that addresses common misconceptions regarding implementing performance improvement projects within military treatment facilities. | Clinic leaders/Providers |
| Process Improvement Tools | Handout describing common process improvement tools, including flowcharts, affinity diagrams, fishbone diagrams, 5 whys exercise, and common charts for displaying process improvement data. Handout provides simple and easy to follow steps for using these tools. | Clinic leaders/Providers |
| Example Affinity Diagram Process: Low Use of Group Psychotherapy | Handout showing the use of an affinity diagram to examine factors that contribute to low use of group therapy. | Clinic leaders/Providers |
| Example Fishbone Diagram: Long Wait Times for Follow- | Handout showing the use of a fishbone diagram to examine factors that contribute to long wait times for follow-up appointments. | Clinic leaders/Providers |
| Fishbone Diagram | Blank template for creating a fishbone diagram for a process improvement project. | Clinic leaders/Providers |
| Distinguishing between Process Improvement, Program Evaluation, and Research Projects | Handout that summarizes the differences between process improvement, program evaluation and clinical research projects across a range of dimensions . | Clinic leaders/Providers |
| Process improvement templates | (Focus-PDCA) | |
| Improving Levels of Group Therapy Utilization | Example FOCUS- PDCA template for a clinic that attempted to improve group therapy use | Clinic Manager |
| Expanding the use of EBPs for PTSD and Depressive Disorders | Example FOCUS- PDCA template for a clinic that attempted to improve the use of EBPs for two disorders | Clinic Manager |
| Setting up a Sub-clinic for PTSD | Example FOCUS- PDCA template for a clinic that started a sub-clinic for PTSD | Clinic Manager |
| Conducting a Clinic Gap Analysis | Example FOCUS- PDCA template for a clinic that completed a Gap Analysis | Clinic Manager |
| Expanding the Use of BHTs in Clinical Care | Example FOCUS- PDCA template for a clinic that attempted to improve the use BHTs clinical skills | Clinic Manager |
| Decreasing Wait Times for Follow-up Appointments | Example FOCUS- PDCA template for a clinic that attempted to decrease wait times for follow-up appointments | Clinic Manager |
To return to the Clinic Optimization Toolkit, click here.