I found this document in my inventory of Project Management templates. I apologize that I can't credit the original author. If someone knows who created this template let me know and I will edit this post and include the author's name.
Project Scorecard Overview
1. Identify criteria for success. Review the objectives and deliverables in the Project Definition, as well as any other existing information that is relevant to the project. Based on this existing documentation, define what information is needed to show that the project was successful. This can be from two perspectives:
• Internal – These characteristics indicate that the project was managed and executed effectively and efficiently. This might include having deliverables approved with no more than two review iterations, hitting major internal milestone dates on time and having a minimum amount of errors uncovered in user acceptance testing.
• External – These characteristics indicate that your project objectives were completed successfully. Examples here include completing the project within approved budget and timeline, ensuring your deliverables meet approved quality criteria and customer satisfaction surveys.
2. Assign potential metrics. Identify potential metrics for each success criteria that provide an indication whether or not the criteria is being achieved. These can be direct, quantifiable metrics, or indirect metrics that give a sense for success criteria For each metric, briefly determine how you would collect the information, what the effort and cost of collection would be, and what value would be obtained.
3. Look for a balance. The potential list of metrics should be placed into categories to make sure that they provide a balanced view of the project. For instance, you do not want to end up with only a set of financial metrics, even though they might be easiest to obtain. In general, look for metrics that provide information in the areas such as:
• Quality of deliverables
• Customer satisfaction with the deliverables produced
• Project team performance
• Business value delivered
4. Prioritize the balanced list of metrics: Depending on how many metrics you have identified, prioritize the list to include only those that have the least cost to collect and provide the most value to the project. There can certainly be as many metrics collected as make sense for the project, but there may end up being no more than one or two per category. In general, look to provide the most information with the least amount of work.
5. Set targets: The raw metric may be of some interest, but the measure of success comes from comparing your actuals against a predefined target. The target may be a single value you are trying to achieve, or it may be a range. For instance, you may need to complete your project by a certain fixed date, but your actual cost might need to be +/- 10% of approved budget.
6. Add workplan detail: For each metric that remains, determine the specific information necessary to add the appropriate activities to the project workplan. This will include:
• What specific data is needed for the metrics?
• Who is responsible for collecting the metric?
• When will the metric be collected and reported?
• How will the metrics be reported (status reports, quarterly meetings, metrics reports)?
Note: Define your success criteria upfront and get project sponsor sign-off.