Common menu bar
Breadcrumb
Understanding the Early Years (UEY) Initiative - July 2007
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, please contact us to request an alternate format.
Table of Contents
- TITLE PAGE
- EXECUTIVE SUMMARY
- 1. INTRODUCTION
- 2. REVIEW FINDINGS
- 3. CONCLUSION
- APPENDIX A APPROVED FILE REVIEW RESULTS
- APPENDIX B REJECTED FILE REVIEW RESULTS
- APPENDIX C REVIEW OBJECTIVES, CRITERIA AND METHODOLOGY
- APPENDIX D MANAGEMENT ACTION PLAN
Title Page
Project No: 19003/06
| Project Team | ||
| Director General: | E. Murphy-Walsh | |
| Senior Audit Director: | C. Lepage | |
| Audit Director: | R. Patel | |
| Team Leader: | S. Lynch (up to December 2006) | |
| Review Team: | S. Mihailov | |
| K. Jevons | ||
| E. Tremblay | ||
| N. Portelance |
APPROVED:
DIRECTOR:
DIRECTOR GENERAL:
July 2007
SP-796-07-07E
(également disponible en Français)
Paper
ISBN: 978-0-662-46814-1
Cat. No.: HS4-37/2007E E
PDF
ISBN: 978-0-662-46815-8
Cat. No.: HS4-37/2007E-PDF
EXECUTIVE SUMMARY
Protected Internal Audit Services conducted an Early Implementation Review of the Understanding the Early Years (UEY) Initiative in 2006-07. This review is intended to provide assurance to senior management that program delivery has been established appropriately in order to meet its objectives and highlight any areas that require focused management attention for improvement. For a detailed description, see Appendix C: Review Objectives, Criteria and Methodology.
UEY is a national initiative designed to provide communities across Canada with locally-specific information and knowledge on the development of children under age six in their communities; on family and community characteristics; and on programs and services supporting them. It helps communities assess their children's needs and equips them with information to make informed resourcing decisions about the best policies and programs for young children and their families in these communities. The focus of UEY is community capacity building using local research on children, with broad objectives of:
- strengthening the capacity of communities to make decisions that are informed by quality local research to give their children the best start in life/enhance the lives of their young children;
- mobilizing community coalitions to support the needs of young children; and
- enhancing knowledge at the national level about how communities can use research to address children's issues.
The UEY Initiative is managed nationally and delivered by the Income Security and Social Development Branch of Human Resources and Social Development Canada (HRSDC). It is based on a pilot research initiative that began in 1999 and included 12 communities. The success of the pilot served as the basis for UEY becoming a formal national initiative in 2004, when the Government announced that it would provide $68 million over seven years. A portion of those funds, $33.5 million, was allocated for Contribution Agreements with a three year project cycle under the Social Development Partnerships Program. Of the remaining funds, approximately $18 million in non-salary dollars was to support data collection and analysis carried out under contract using Operating and Maintenance (O&M) funds.
Up to 100 communities were to be funded through four annual Call for Proposals (CFP) from 2005 to 2008, with the first group of 21 new communities receiving funding in fall 2005. Protected.
Fieldwork was performed from September to December 2006 at National Headquarters and also included five on-site visits to projects. The review included: an assessment of general program management; process control mapping and analysis; and a file review of all proposals received in response to the 2005 CFP.
This review was conducted in accordance with Treasury Board's Policy on Internal Audit and the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing.
Management Control Framework Objective — In our opinion, management practices and systems are developed and aligned to support program policy and objectives, identify and manage risks and are operating as intended. However, a concern exists that the performance measures contained in the SDPP Results-Based Management and Accountability Framework, which currently includes the UEY Initiative, are very broad and may not be adequate to report on the effectiveness of UEY specifically. Although some of the performance measures for the entire SDPP apply to UEY, it is important to have specific performance measures for the Initiative itself in order to effectively measure results of UEY.
Information Management Objective — In our opinion, information is appropriate for the assessment of policy and program design and the administration of the Understanding the Early Years Initiative, ensuring program integrity for reporting purposes.
Operational Compliance Objective — In our opinion, the Understanding the Early Years Initiative is managed in accordance with appropriate acts, regulations, policies, directives and guidelines, with a few notable exceptions. Overall, the files are well maintained and documented. However, there are key concerns requiring immediate management attention, which are noted below. Although there are some other instances of non-compliance, in the opinion of the auditors, projects were administered in compliance with program requirements and would not put the integrity of the overall process in jeopardy. The areas of concern related to:
- the use of contribution funding to support service contracts; and
- some essential work processes pertaining to the assessment and negotiation of contribution funding are not evident.
Overall, the Understanding the Early Years Initiative is well managed and controlled in support of program objectives. Good communication and cooperation is evident within the UEY program area, as well as with recipients, and other stakeholders such as community coalitions, provincial and territorial representatives. Feedback from these groups as well as departmental representatives has led to improvement of tools and processes in time for the second round of CFP, as well as the overall management of the UEY Initiative.
Recommendations from this review are:
- Protected.
- Protected.
- The Understanding the Early Years Management Team should ensure that appropriate parental permissions are obtained to allow the sharing of each community's Early Development Instrument data file with their recipient organization.
- The Understanding the Early Years Management Team should establish policies and procedures for retention, storage and disposal of UEY documentation particularly in light of privacy concerns.
- The Understanding the Early Years Management Team should amend its contracting and funding practices for service contracts and Contribution Agreements to ensure compliance with the Treasury Board Contracting Policy and the Treasury Board Policy on Transfer Payments.
- The Understanding the Early Years Management Team should ensure that a copy of the proposal with the date received stamp is retained in the project file.
- The Understanding the Early Years Management Team should ensure consistent application of specified criteria to all proposals.
- The Understanding the Early Years Management Team should strengthen the assessment function as a critical element of project administration/negotiation and provide staff with the necessary training and support to perform their tasks with due diligence.
1. INTRODUCTION
Background
Protected, an Early Implementation Review of the Understanding the Early Years (UEY) Initiative was conducted by Internal Audit Services in 2006-07. This review is intended to provide assurance to senior management that program delivery has been established appropriately in order to meet its objectives and highlight any areas that require focused management attention for improvement.
UEY is a national initiative designed to provide communities across Canada with locally-specific information and knowledge on the development of children under age six in their communities; on family and community characteristics; and on the programs and services supporting them. It helps communities to assess their children's needs and equips them with information to make informed resourcing decisions about the best policies and programs for young children and their families in these communities.
UEY attempts to achieve regional representation across Canada, with a mix of urban and rural communities, and aims to involve young children from all backgrounds. These include Aboriginal children, children of new immigrants, children in official language minority communities and disadvantaged children. The focus of UEY is community capacity building using local research on children. The broad objectives of UEY are to:
- strengthen the capacity of communities to make decisions that are informed by quality local research to give their children the best start in life/enhance the lives of their young children;
- mobilize community coalitions to support the needs of young children; and
- enhance knowledge at the national level about how communities can use research to address children's issues.
The UEY Initiative is managed nationally and delivered by the Income Security and Social Development Branch of Human Resources and Social Development Canada (HRSDC). It is based on a pilot research initiative that began in 1999 and included 12 communities. The success of the pilot served as the basis for UEY becoming a formal national initiative in 2004, when the Government announced that it would provide $68 million over seven years. A portion of those funds, $33.5 million, was allocated for Contribution Agreements under the Social Development Partnerships Program (SDPP). Of the remaining funds, approximately $18 million in non-salary dollars was to support data collection and analysis activities, carried out under contract using Operating and Maintenance (O&M) funds.
Up to 100 communities were to be funded through four successive annual Call for Proposals (CFP) from 2005 to 2008. Approximately 25 communities were to be introduced in each annual intake, with each community participating in a three year project cycle. The first CFP was held in 2005 in which 21 new communities were selected. Protected.
Objectives and Scope
Recognizing the short timelines between the announcement and launch of UEY, program delivery mechanisms and tools were modelled on the design of the 1999 UEY pilot initiative. The results of this Early Implementation Review, therefore are presented in the context of a learning experience that gives management early feedback on UEY achievements to date and potential gaps between intended and actual delivery approaches.
Objectives of this review were to assess:
- The Management Control Framework — To determine whether management practices and systems are developed and aligned to support program policy and objectives; identify and manage risks, and are operating as intended.
- Information Management — To determine whether information is appropriate to assess policy and program design and the administration of UEY, thereby ensuring program integrity for reporting.
- Operational Compliance — To determine whether the UEY Initiative is managed in accordance with appropriate acts, regulations, policies, directives and guidelines.
Fieldwork was performed from September through December 2006 at National Headquarters and also included site visits to five projects: UEY Greater Victoria (British Columbia), UEY Lorette (Manitoba), UEY Northern Region of Ontario, UEY Pointe-de-l'Île de Montreal (Quebec), and UEY Cumberland County (Nova Scotia). These sites were chosen on several factors including size, number of young children and geographical coverage.
Review components included: an assessment of program management; process control mapping and analysis; and a file review of all proposals received in response to the 2005 CFP. The auditors developed a checklist from relevant legislation, policies and program procedures which was validated by program officials, prior to beginning the file review. Program officials were debriefed on all relevant findings as the review unfolded, with clarification from UEY staff sought by the auditors when necessary.
Please refer to Appendix A for a copy of the file review checklist containing a summary of the approved files review results. Appendix B contains a summary of the rejected files review results, while Appendix C gives a detailed description of the review objectives, criteria and methodology.
This review was conducted in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing.
2. REVIEW FINDINGS
All significant review findings presented in this section follow review objective(s) and criteria, detailed in Appendix C: Review Objectives, Criteria and Methodology.
2.1 Objective 1: Management Control Framework
To determine whether management practices and systems are developed and aligned to support program policy and objectives; identify and manage risks, and are operating as intended.
Criteria
2.1.1 Leadership
Leadership is demonstrated by program management in implementing the UEY Initiative. UEY program management is a small unit, concentrated in a single locale, which has facilitated their effectiveness in providing strategic and functional direction. In spite of many challenges including departmental reorganizations and changing government priorities, management has successfully provided clear messaging and guidance. At this early stage of UEY implementation, it is not possible to assess management's effectiveness in communicating and reporting results.
2.1.2 Strategic Planning
Strategic Planning for UEY builds on experience with the pilot projects, protected. Various elements of strategic planning: UEY's mandate, objectives and intended outcomes support the branch vision, goals and priorities, and are communicated consistently, permeating program documentation at all levels. Provincial-territorial counterparts and regional Service Canada representatives are involved in UEY through consultation on assessments of proposals and coordination of activities to broaden coverage, while avoiding duplication of effort.
2.1.3 Performance Measures
Identifying meaningful performance measures will be challenging for UEY, given the nature of social programs. The Social Development Partnership Program is a broad-based grants and contributions program. The logic model, included in SDPP's Results-Based Management and Accountability Framework (RMAF), is intended to capture the activities and outputs for all components, including UEY, operating underneath the program's Terms and Conditions. It outlines how activities and outputs are intended to eventually lead to the ultimate outcome of full inclusion of vulnerable populations in all aspects of Canadian society. Although UEY fits into this broad based logic model, it is not specifically tailored to UEY, making performance measurement for UEY difficult. Performance indicators for measuring outputs and outcomes identified in the SDPP RMAF, are very broad. They are qualitative and quantitative in nature and focus mainly on procedural activities, reports generated and stakeholders' views, in order to be applicable to all programs and initiatives operating under SDPP.
A second round of data collection, which was part of the five year pilot projects, was dropped from the expanded Initiative's three year projects, eliminating the use of comparative data as a means of measuring progress within UEY communities. Ultimately, to demonstrate success, performance measures will need to provide an indication of the degree to which UEY was a catalyst for mobilizing communities to improve early childhood development outcomes for their children; and the degree to which it was cost-effective in having an impact on social policies and programs that address the needs of children and families. A formative UEY evaluation is scheduled for 2007-2008.
Recommendation
1. Protected.
2.1.4 Monitoring and Performance Reporting
Before contribution agreements are entered into, monitoring requirements and schedules are determined for each project, using the risk assessment module in the Common System for Grants and Contributions (CSGC). In addition to the formal monitoring schedules UEY Research Staff and Program Officers (PO) are routinely in contact with recipients through phone calls and e-mails. Recipients are required to submit quarterly activity and financial reports using the template provided by the program, which outlines their progress and outputs, as well as financial activities for the period. These are reviewed by the POs to determine whether they are in keeping with requirements and timelines as stated in Schedule A of the Contribution Agreement; any needed clarification or corrections are sought from recipients. The Financial Monitoring Unit (FMU) reviews the financial reports and authorizes payments. Additionally, the FMU is tasked with performing one on-site financial monitor at a minimum, for each contribution agreement. Given the early stages of the UEY Initiative, the audit team was not able to assess the on-site financial monitoring activity.
Projects' activities and outputs are then aggregated into a single report for the UEY Initiative using three themes: knowledge development and dissemination; community capacity and community action. Although still early in the projects' life cycle, the UEY program team has already developed and distributed a final report template to recipients. Additionally, a community handbook on reporting results/evaluation and subsequent training on its content has been delivered which provides guidance on collecting and reporting results in line with departmental accountability frameworks.
2.1.5 Resource Allocation Models
The resource allocation model for UEY evolved from experience with the pilot research sites in terms of contribution agreement funding levels for each project, estimated research contract requirements dependant on the number of children involved and internal program staffing requirements. An original estimate of $350,000 per contribution agreement, based on costs for the pilot, has since grown to $375,000 due to additional costs associated with hiring a researcher.
In assessing which proposals will be accepted, efforts are made to achieve regional representation across Canada, ensuring a mix of urban and rural communities, while also endeavouring to include children from marginalized or disadvantaged communities. Gaps in regional coverage, as well as groups targeted as priorities, were identified after the 2005 CFP, and subsequent steps have been taken to involve these groups in the next round of UEY projects.
Protected
Recommendation
2. Protected.
2.1.6 Program and Financial Authorities
Responsibilities for program and financial authorities are clearly identified and communicated. Delegations of authorities are in place and understood by all individuals and file reviews confirmed they are consistently followed.
2.1.7 Policies
Policies for research and program delivery are in place for UEY and updated and adapted as required. Given the short time frame between the announcement of the expanded UEY Initiative to full uptake, some policies and procedures had to be refined during implementation. For example, departmental guidelines and acts regulating privacy and data sharing affect the degree to which research information may be used and shared with stakeholders. To address the problem, the UEY team sought advice from departmental representatives in Legal Services and Access to Information and Privacy (ATIP) and worked diligently to clarify requirements and limitations, communicating results and providing recipients with vetted parental consent forms to be used in any further rounds of data collection.
2.1.8 Operational Planning
Operational planning has been carried out and clear linkages to strategic priorities, branch objectives, resource allocation, expenditures, accountabilities and timeframes exist. An annual workplan provides an overview of UEY priorities, related activities and assigned responsibilities.
2.1.9 Program Tools
The UEY team has developed extensive program tools and processes to assist in the consistent delivery of UEY throughout the various stages of the Initiative. Several orientation and training workshops, application guides, reporting templates and communication tools have been provided to focus efforts on desired outcomes and support a streamlined approach. Tools have been added or improved in response to identified deficiencies to benefit current and subsequent rounds of UEY. Some examples of improvements made include: the provision of templates for comprehensive release forms for parents of young children involved in the Parent Interviews and Direct Assessments of Children Survey (PIDACS) data collection activities; and addition of a financial reporting component added to the orientation session provided for new UEY recipients.
Complications arising from privacy issues (inadequate parental release permissions with ensuing data restrictions), as well as challenges faced by the delay in contracting for a research provider for the PIDACS portion of the research, may have resulted in difficulties such as lost momentum with community partners and risk to the department's reputation. However on review, the audit team noted that the cost savings and improved transparency of tendering the contract through PWGSC was warranted. This situation has been offset by strong working relationships on the part of program management, particularly fostered by POs with recipients. See section 3.2.8 for further details.
2.1.10 Effective Communication Strategies
Communication strategies were developed and implemented to ensure delivery of clear, consistent messaging to staff, and gathering of feedback from staff, partners and stakeholders. The UEY team has worked with the Communications Branch to develop clear and consistent messaging both internally and externally. The UEY unit is a small group concentrated within a single office and this close proximity has facilitated the regular exchange of ideas informally between management, research, and program delivery staff.
UEY program management and staff have made extensive efforts to provide information and transfer knowledge to UEY stakeholders in a proactive manner in order to enhance communication. To date, there have been several forums for exchanging ideas with UEY communities:
- Understanding the Early Years Orientation Session (Ottawa, November 2005);
- Community Mapping Workshop (Vancouver, May 2006);
- Early Development Instrument (EDI) and Parent Interviews and Direct Assessment of Children Survey (PIDACS) Workshop (Burlington, September 2006); and
- Reporting on Results/Evaluation Training (three sessions in Vancouver, Halifax and Ottawa, January/February 2007).
Other external communications include publishing information on the UEY website, using Infosecure (a web-enabled forum for discussion among UEY co-ordinators), and giving presentations to UEY communities. UEY is a complex Initiative; each project involves multiple players including a sponsoring organization, a community coordinator, a researcher, community coalition and contractors. The UEY team has demonstrated its ability to be adaptable and flexible in addressing the needs of individual recipients, even when presented with departmental reorganizations, privacy legislation and staff turnover, combined with short timeframes between the announcement of the UEY expansion and the start of program delivery. Often, grants and contribution programs require reactive responses from officials to questions/issues of recipients. To ensure timely and consistent communication, the UEY team sends group e-mails directly to sponsors and co-ordinators.
2.1.11 Recruiting, Selecting and Training
Human resource planning is evident and clearly documented with linkages to departmental priorities, internal and external environmental risk factors, and staffing requirements. UEY integrates both research and program management components, and as such, requires staffing in both areas. For the most part, research and program officer positions are kept separate; but there are a few instances where Program Officers (POs) perform a combined role of administering projects while performing research duties. This gives staff interesting development opportunities and also provides a creative approach to broadening the unit's capacity. The incumbent POs are relatively new to their positions. Staff administering UEY have received Program Delivery training and instruction in CSGC. However, file reviews indicate that POs may benefit from further guidance in delivering grants and contribution programs, particularly in project development and negotiation. See section 3.3.3 for elaboration.
2.1.12 Work Processes
Work processes were built on experience with the UEY pilot, with workflows and procedures updated to address identified issues. Figure 1: "Understanding the Early Years, Overview Process Chart", provides a broad summary of UEY activities and the associated roles and responsibilities of the various groups involved.
Each UEY community is selected through a Call for Proposals process. An initial screening and two-tier assessment is conducted to determine which proposals will receive funding. Regional representatives of Service Canada and provincial/territorial government representatives contribute their knowledge of regional and local non-profit organizations that apply for UEY funding during the assessment phase. After ministerial approval has been given, HRSDC enters into a three year Contribution Agreement with the successful project Sponsor who then hires a Community Co-ordinator to manage all aspects of their UEY project, and a Researcher to conduct an inventory of the community's programs, resources and services.
Two independent research organizations under contract to HRSDC, McMaster University's Offord Centre for Child Studies and R.A. Malatest & Associates Ltd., gather and compile early childhood development information in each UEY community using the "Early Development Instrument" (EDI) and the "Parent Interviews and Direct Assessments of Children Survey" (PIDACS), respectively. The Community Co-ordinator hired under the Contribution Agreement helps facilitate the data collection process in their community.
Click here to enlarge
Using research results provided by the contractors and the data from the Inventory of community programs, services and resources, the Community Researcher overlays both sources of information into a mapping report that contains socio-economic census data specific to geographic areas within their designated communities. Compilation and interpretation of this information forms the basis for the UEY coordinator to work with their sponsor and community coalition in developing a Community Action Plan. This Action Plan, a key deliverable of the Contribution Agreement, is intended to provide the impetus for community stakeholders to work together beyond the life of the UEY Initiative, mobilizing resources to offset identified risks and deficiencies and address the needs of their young children and families.
At a more detailed level, the following process map in Figure 2, captures the UEY team's internal workflows for administering the contribution program. The UEY Initiative relies on HRSDC's Operations Guide on Grants and Contributions to support program administration and where needed, has developed its own tools to guide the process within this operational framework. Formal documents and procedures were developed by the UEY team to guide intake, screening, assessment and decision making processes that ensure fairness and transparency of project recommendations.
The conventional practice under SDPP is to not contact applicants prior to receiving ministerial approval. This practice was maintained by the UEY Initiative. The UEY team recognized that in many cases, successful proposals were completed by contractors with little input from persons who would be responsible for carrying out the project's activities. In response, a decision has been taken to meet with the second round of successful project sponsors before entering into Contribution Agreements. This direct contact will allow UEY staff to familiarize recipients with the intricacies of HRSDC's fiscal year and financial reporting requirements, as well as expected project work plans. This will in turn allow recipients to more realistically project funding requirements across fiscal years and adjust their work plans accordingly.
The UEY Initiative relies on the expertise of a separate Financial Monitoring Unit (FMU) for financial administration of UEY Contribution Agreements. This unit, located in the same division as UEY also has responsibility for administering other contribution agreements under the Social Development Partnerships Program (SDPP).
Click here to enlarge
2.1.13 Internal Controls
For the most part, effective internal controls are in place for the UEY Initiative. The process map in Figure 2 illustrates key steps in processing UEY proposals, showing control points. A description of these controls and their effectiveness is discussed below.
| a. | Documented Procedures and Tools — Clearly documented procedures are a preventive, manual control that provides some assurance of consistent processes. UEY work processes, CFP documents (templates and application guides) and assessment tools are derived from HRSDC's Grants and Contributions Operations Guide and the Common System for Grants and Contribution Training Manual. Procedures have been updated to address issues that were identified during the first round of funding and in the Voluntary Sector Accord and Codes of Good Practice. Tools have been developed to ensure that proposals are consistently and completely screened and assessed for eligibility. However, some essential work processes outlined below were not evident: |
- date stamps showing date of receipt of proposals;
- consistent application of criteria for required supporting documentation; and
- negotiation of the budget and development of Contribution Agreements as outlined in the Grants and Contributions Operations Guide.
Training has been provided to UEY program officers for using CSGC; however, further guidance and training in project development and administration would strengthen the effectiveness of this control. See section 3.3.3 for further details.
| b. | Authorizations — Appropriately designed and communicated delegation of authority is a critical component of a sound internal control framework, which provides specific authority for staff to perform certain actions. Delegation of authorities is in place and clearly understood by all individuals. |
| c. | Segregation of Duties — Segregation of duties is a critical preventive control designed to reduce the risk of error, abuse or fraud, as well as ensure payment accuracy. Segregation of duties exists for UEY, where staff (POs) responsible for recommending and administering projects is different from those responsible for approving the agreement (Minster) and issuing payment (FMU). Approvals for payment in the Corporate Management System (CMS), are also performed by individuals separate from those who process the payments in accordance with the Financial Administration Act (FAA). The effectiveness of this approach is underscored by the file review finding of a 100% compliance rate for appropriate authorizations and financial coding. The FMU is also credited with finding and correcting a few signature irregularities with recipients before they became a problem. |
| d. | External Consultation — Independence of external assessors is a preventive control designed to reduce risk of fraud or bias. Assessments conducted by provincial/territorial government representatives and regional early childhood development experts ensure objectivity in decision making and minimize the risk of error, abuse or fraud regarding recipient eligibility in the proposal approval process. |
| e. | Internal Consultation — Internal assessments are an integral component of UEY's intake process. They are conducted in a group situation that allows for peer review, a preventive control designed to reduce the risk of inconsistent eligibility assessments and errors. |
| f. | Review of outstanding debts — Reviewing outstanding debts before entering into a Contribution Agreement is a preventive control designed to reduce the risk of waste by continuing to fund an applicant organization that already owes money to the department. As part of the work process, a review of the Departmental Accounts Receivable System (DARS) is done for each proposal, prior to finalizing the Contribution Agreement. |
| g. | Access Controls — Appropriate access controls are preventive controls to ensure that only authorized individuals can process or review the proposal information, process payments and ensure that sensitive information is secure and safeguarded from accidental loss or deliberate theft. Electronic information stored in CSGC and CMS is controlled in a secure manner through USER ID and passwords. Paper files and documentation are also appropriately secured by controlled card access to the office building and floor. |
| h. | System Edits — System edits are effective automated controls that prevent processing incomplete, inaccurate or unauthorized transactions. Several automated controls exist in CSGC, including: |
- mandatory fields;
- payments cannot exceed commitments;
- approval of projects must be done by a second individual which demonstrates segregation of duties for security; and
- payments cannot be made before the project's start date.
| i. | Audit Trails — Audit trails that record information and events are an effective tool for investigating and/or substantiating actions taken. USER IDs are captured and tracked in CSGC and CMS for all financial transactions. The importance of the audit trail is heightened, given that there are few automated controls in CSGC over changes made to project information. |
| j. | Automated data feeds — Automated data feeds reduce the risk of error from manual intervention in the payment process. Payment data is transferred automatically from CSGC into the department's financial system, CMS, as authorized. |
| k. | Reporting — Reporting in CSGC is the departmental system of record and is sufficient for UEY management, given the limited number of projects. CSGC, maintains the key information on individual UEY Contribution Agreements. CMS reports are used by the FMU to assess budget commitments and expenditures. |
| l. | Monitoring — Monitoring activities are a critical control used to detect abuse, errors and/or fraud. Project monitoring requirements are determined by a risk assessment module in CSGC; a monitoring schedule is generated and placed on file before contribution agreements are finalized. UEY staff are routinely in contact with recipients through phone calls and emails. In this manner they are able to monitor the appropriateness of project activities and progress, as per the terms of the contribution agreements. The importance of financial monitoring visits is heightened, given that supporting documentation required for claims processing consists only of general ledger entries and there is no requirement for recipients to submit original documents to support their claims. It should be noted that as projects were still in the early stages, no on-site financial monitoring had yet been performed, therefore the auditors were unable to assess the adequacy of financial monitoring at this time. |
2.1.14 Improvement Processes
Plans and program delivery mechanisms are revisited regularly with risks and opportunities for improvement identified and action taken as needed in a proactive manner. Examples include:
- developing tools and workshop components to educate and alleviate confusion among recipients over fiscal reporting requirements;
- modifying CFP submission deadlines to increase accessibility by reflecting the reality of school calendars; and
- providing extensive support to project recipients through templates and training, in addition to frequent contact for generic project maintenance.
Also, UEY program management recognized that some groups were under-represented in the first round of 21 projects and have taken steps to rectify this for the 2006 CFP where the UEY team worked with its federal counterparts, Communications and Aboriginal organizations to increase uptake among Aboriginal communities.
2.2 Objective 2: Information Management
To determine whether information is appropriate to assess policy and program design and the administration of UEY, thereby ensuring program integrity for reporting.
When addressing the topic of information management, it is important to remember that in UEY, there are two distinct aspects to information. As in any operation, information is a key dimension of UEY's management framework and is addressed under the criteria headings below. Also for UEY, early childhood development information is a critical input as well as the ultimate product. This relationship is illustrated in Figure 1: "Understanding the Early Years, Overview Process Chart". During this review the audit team became aware of concerns relating to the research information; however while the audit team was able to determine that the contracting process was appropriate; an evaluation of the research is beyond the scope of this Early Implementation Review.
Criteria
2.2.1 Accuracy, Completeness, Timeliness and Relevancy of Information
Program records are well maintained: POs keep comprehensive paper files for each project and the FMU maintains a separate paper file of all financial records for each project. All files were consistently and thoroughly documented and the co-location of the FMU on the same floor as the rest of the UEY team has facilitated easy information sharing and workflows.
CSGC is a standard system used to capture all grants and contribution information within HRSDC. Use of mandatory fields in CSGC ensures that appropriate steps are followed in the required sequence and key program information is captured.
2.2.2 Availability of Information
Program delivery information is available at any time from data in CSGC and more detailed records contained in project and financial files are easily accessible to program staff. Narrative progress reports are compiled from quarterly reports submitted by recipients and, given the relatively small number of projects, this appears appropriate. Financial quarterly reports are also provided by recipients as required by the terms of their Contribution Agreements. CMS reports are used for assessing budget commitments and expenditures.
2.2.3 Data Integrity and Validity of Information
The file review found that the validity and data integrity of the information generated in CSGC is high. Information entered into CSGC was supported by the applications and other authoritative documents.
2.2.4 Management Information for Program Performance
As noted earlier, a formative evaluation for UEY is scheduled for fiscal year 2007/08. The evaluation will assess the early outcomes of the Initiative in terms of meeting the program objectives. In the interim however, management has been able to identify gaps or deficiencies and address any concerns identified in a timely manner. Given the relatively small number of projects, this is appropriate.
2.2.5 Relevant Information to Manage Resources
Based on file reviews, program documentation and reports, it was found that information is available and used well to manage UEY resources. Financial information in CMS, used to track both committed and actual spending against budgets, is updated daily from CSGC data.
2.2.6 Information to Assess Operational Efficiency and Effectiveness
Operational efficiency and effectiveness are assessed using formal and informal information. Through national conferences and workshops, feedback on operational efficiency and effectiveness is gathered and shared.
2.2.7 Information Provided to Recipients
Program information provided to organizations was timely, accessible, understandable and transparent. CFP information, e-mail addresses and links to pilot sites were available on the website. The Guide for Applicants was clearly written and matched the requirements on the proposal template. Clarification and improvements were made to proposal requirements and supporting documents for the second round of CFP, including an extension of the closing date to improve accessibility. Transparency of the decision making process was achieved; letters sent to unsuccessful applicants provided sufficient information to indicate the reason for the decision. Documentation on files indicates that when more information was requested, the UEY team responded promptly with details of assessment results, so that applicants could address deficiencies to improve their chance of success for the next round of funding. Where possible, information in the form of workshops, training, reporting templates and other tools, were provided in a comprehensive, timely, and proactive manner. When circumstances have demanded redirection, the UEY team excelled in seeking clarification and has been highly responsive to recipients, providing information and guidance in a timely manner.
2.2.8 Appropriate Sharing of Data
UEY operational data is shared continuously within HRSDC, made available through both CSGC and CMS. Access to the data is restricted appropriately only to those individuals involved in administering and managing the program. Suitable sharing of reports on program results within HRSDC and with other stakeholders cannot be adequately assessed at this early stage of UEY implementation.
Sharing of research data is one of the main elements of the UEY Initiative and is a means to developing an evidence-based action plan. However, restrictions governing the federal government's sharing of information due to privacy issues have presented challenges not fully understood at the outset of the Initiative. Proper release permissions were not always obtained prior to the collection of data to allow for comprehensive data sharing arrangements to the degree that had been initially been intended. The UEY team is working with the department's Databank Review Committee (DRC), Access to Information and Privacy (ATIP) and Legal Services, and steps have been taken to mitigate the problem. For the PIDACS portion of the research, a comprehensive parental consent form has been developed that will eliminate future reoccurrences. A similar parental consent form is not in place at this time for the EDI portion of the UEY initiative.
Recommendation
3. The Understanding the Early Years Management Team should ensure that appropriate parental permissions are obtained to allow the sharing of each community's Early Development Instrument data file with their recipient organization.
2.2.9 Record Retention, Storage and Disposal
At this early stage in the UEY Initiative, retention, storage and disposal of UEY documentation is not an issue beyond the policies and procedures outlined by Financial and Administrative Services (FAS) and incorporated into the research contracts. Management intends to follow departmental requirements for archiving files when the need arises.
Recommendation
4. The Understanding the Early Years Management Team should establish policies and procedures for retention, storage and disposal of UEY documentation particularly in light of privacy concerns.
2.3 Objective 3: Operational Compliance
To determine whether UEY is managed in accordance with appropriate acts, regulations, policies, directives and guidelines.
Criteria
2.3.1 HRSDC Operations Guide on Grants and Contributions
UEY relies on HRSDC's Operations Guide on Grants and Contributions to support its administration of the Initiative. Upon review, we have determined that it adheres to the Treasury Board Transfer Payment Policy and the Financial Administration Act (FAA) and Regulations, and does not exceed the intent of the policy nor the statutory framework for managing grant and contribution programs.
2.3.2 Departmental Policies Reflect Appropriate Legislation and Treasury Board Policies
Based on our review of relevant documentation, we have concluded that with the exception of the irregularities noted below, policies are in line with appropriate legislation and Treasury Board policies.
The overview of UEY activities and the associated roles and responsibilities previously provided in Figure 1: Understanding the Early Years Overview Process Chart, also illustrates the funding vehicles used to support these activities. Service contracts, by definition, have deliverables to the Crown, as compared to Contribution Agreements which cannot result in any direct benefits to the Crown. Contracting for PIDACS and EDI data collection is appropriate, as HRSDC retains control of the data and related information generated.
UEY is a unique program in that it is necessary for recipients of contribution agreements to interact and work with service contract providers. This adds a level of complexity to the UEY Initiative in terms of ensuring that these funding vehicles are used appropriately. During the data collection periods, co-ordinators are required to spend quite a bit of time working with the contractors for EDI and PIDACS and facilitating their efforts. In addition, reimbursement of school boards' teacher replacement costs estimated at $350,000, was funded through amendments to Contribution Agreements, instead of being handled as an amendment to the EDI service contract. Teacher replacement costs are reimbursements to school boards for training and data collection activities performed by the teachers, which are essentially in support of the EDI contract. While these arrangements do not represent any additional charges to Canadian taxpayers, combining the two funding instruments in this manner to support UEY delivery, overrides parliamentary authority and does not comply with the Treasury Board Contracting Policy and the Treasury Board Policy on Transfer Payments. For the 2006 CFP, UEY Management has assured the audit team that teacher replacement costs will be funded directly through the EDI service contract.
The UEY Initiative uses the SDPP Terms and Conditions, which contains a clause in section 11 that requires monthly forecasts to be included in all applications. The Initiative's CFP documents however, only require budgeted amounts broken down on a fiscal year basis, which is more relevant and functional. Section 11 also contains a clause requiring the disclosure of former public servants under the conflict of interest and post-employment guidelines at the application stage, which again, is not included as a requirement in UEY's CFP documents. As this is a standard clause in all Contribution Agreements, there would appear to be no additional benefit derived from this requirement.
Recommendation
5. The Understanding the Early Years Management Team should amend its contracting and funding practices for service contracts and Contribution Agreements to ensure compliance with the Treasury Board Contracting Policy and the Treasury Board Policy on Transfer Payments.
2.3.3 Adherence to Departmental Policies, Procedures, Guidelines
Generally, project files were processed according to departmental policies, procedures, guidelines and the program's terms and conditions for administering UEY. Files were consistently and thoroughly documented and staff conscientiously complied with their understanding of program requirements. Assessments to support decision rationales were well documented; approval authorities were correctly applied. All files demonstrated that recipients had been checked in DARS for outstanding debts owed to the department, prior to entering into Contribution Agreements. Communications with recipients were frequent and well documented, demonstrating strong working relationships. Financial requirements were conscientiously followed and projects monitored for progress.
However, some areas of concern were identified and are described below. The results of the file review are summarized in Appendix A for approved and Appendix B for rejected files. The checklists used to examine the files were developed from Treasury Board's Policy on Transfer Payments, HRSDC's Operations Guide on Grants and Contributions, SDPP's Terms and Conditions and UEY's 2005 CFP documents. The checklists were accepted by UEY program management prior to commencement of the file review.
Documentation of Date Received
The UEY Initiative clearly identifies a closing date for CFPs. Applications are to be date stamped as evidence that they were received prior to the closing date. Date stamps showing the date the proposal was received were not evident on 76% of accepted files, and 67% of rejected files. While the UEY team confirmed that they date stamped one of the five copies of each proposal received, they were unable to find the stamped copy. Without sufficient evidence to support the date received, the transparency and fairness of the application process could be questioned. The program area has advised the audit team that for the current CFP, all copies of proposals are date stamped to rectify this situation.
Recommendation
6. The Understanding the Early Years Management Team should ensure that a copy of the proposal with the date received stamp is retained in the project file.
Decision to Accept or Reject
In all cases where proposals were rejected, the audit team concurred that the decision was appropriate. In some cases however, the audit team believed that the decision to reject the proposal ought to have been made earlier: at the initial screening stage, rather than after a full assessment was completed. It should be noted that in all cases reviewed, the UEY team took a more generous approach to screening for required documentation.
In one case, although ultimately successful in receiving funding, the level of support from school boards did not adequately meet the required proposal standards, and no rationale or explanation documenting any special circumstances was evident. Therefore, the auditors assessed the proposal as having been screened in, inappropriately. Beyond the issue of efficiency, this raises questions regarding the transparency of the process if the criteria for required documentation to demonstrate eligibility are not enforced.
The program area has on its own initiative refined the required criteria for the current CFP to better reflect the components of UEY that are essential for projects' success.
Recommendation
7. The Understanding the Early Years Management Team should ensure consistent application of specified criteria to all proposals.
Appropriate Assessment of Budgets and Claims
A key area of concern for the audit team relates to the adequacy of budget assessments performed by POs and ensuing documentation of justifications. Of the accepted files, 95 % did not contain sufficient evidence to demonstrate that the proposed budgets had been assessed to determine whether the requested amounts were reasonable and contained sufficient detail, prior to entering into contribution agreements. Based upon SDPP practices, UEY management decided that there was to be no contact with prospective recipients until projects have received ministerial approval.
The potential impact of the lack of budget assessment and appropriate documentation is significant, with financial as well as reputational implications for HRSDC. Thoroughness in developing contribution agreements at the outset is essential for successful projects, setting the tone for the duration of the project life cycle and impacting the viability of UEY as a whole. A key component of any proposal assessment is to strike a balance between adequate financial support and prudent investment decisions that could include scoping down funding requests or seeking alternate funding sources. This concept is addressed comprehensively in the Operations Guide. The program area acknowledged this shortcoming in the first round of funding and has committed to addressing this in future CFPs.
It should be noted that the current POs were not involved in developing the 2005 CFP projects; all were brought into the process to administer the projects after the contribution agreements were underway. Discussions with staff revealed a lack of understanding of their role in assessing budgets and verification of claims. For example, standard items such as Mandatory Employment Related Costs (MERCS) versus benefit packages, or accepting budget items by default if they are not specifically excluded as ineligible costs, were not generally understood.
Recommendation
8. The Understanding the Early Years Management Team should strengthen the assessment function as a critical element of project administration/negotiation and provide staff with the necessary training and support to perform their tasks with due diligence.
3. CONCLUSION
Overall, the UEY Initiative is well managed and controlled in support of program objectives. Good communication and cooperation is evident within the UEY program area, as well as with recipients and other stakeholders. Feedback from recipients and departmental representatives has led to improved tools and processes for the second round of CFP, as well as overall management of the UEY Initiative.
Management Control Framework Objective 1
In our opinion, management practices and systems are developed and aligned to support program policy and objectives, identify and manage risks and are operating as intended.
Information Management Objective 2
In our opinion, information is appropriate for the assessment of policy and program design and the administration of the Understanding the Early Years Initiative, ensuring program integrity for reporting purposes.
Operational Compliance Objective 3
In our opinion, the Understanding the Early Years Initiative is managed in accordance with appropriate acts, regulations, policies, directives and guidelines with a few notable exceptions discussed in this report. Project files are well maintained and documented and although there were a few instances of non-compliance, projects were usually administered in compliance with program requirements and would not put the integrity of the overall process in jeopardy.
In our professional judgement, sufficient and appropriate review procedures have been conducted and evidence gathered to support the accuracy of the conclusions reached and contained in this report. These conclusions are based on a comparison of the situations as they existed at the time against the relevant criteria. These conclusions are only applicable for the Understanding the Early Years Initiative.
This review was conducted in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing.
APPENDIX A APPROVED FILE REVIEW RESULTS
| Yes | No | N/A | Incidence Rate | ||
| Screening Requirements | |||||
| 1 | Was the proposal postmarked and/or date stamped by the closing date? (4:00 p.m. EST April 11, 2005) | 5 | 16 | 0 | 76% |
| Organization eligibility (non-profit and actively involved in social development issues) Is there: | |||||
| 2 | evidence of non-profit status, i.e. letters patent or corporate charter? | 1 | 0 | 5% | |
| 3 | evidence of organization's mandate, i.e. official mission statement? | 21 | 0 | 0 | 0% |
| 4 | If Quebec project, is a completed M-30 submitted? | 2 | 0 | 19 | 0% |
| Is there evidence to support that the community fits program criteria: | |||||
| 5 | Place based? (i.e. Contiguous geography) | 21 | 0 | 0 | 0% |
| 6 | Have an existing community coalition? | 21 | 0 | 0 | 0% |
| 7 | Support of all participating schools and school boards? | 20 | 1 | 0 | 5% |
| 8 | Have suitable potential candidates for UEY coordinator? | 20 | 1 | 0 | 5% |
| 9 | At least 300 5-year-olds entering S Kindergarten in 2005-06? | 21 | 0 | 0 | 0% |
| 10 | Given the above, was the proposal screened in/out appropriately? | 20 | 1 | 0 | 5% |
| Proposal Assessment | |||||
| Does the proposal: | |||||
| 11 | include a description of community and key partners | 21 | 0 | 0 | 0% |
| 12 | address one or more program objectives? | 21 | 0 | 0 | 0% |
| 13 | fit within Program priorities? | 21 | 0 | 0 | 0% |
| 14 | describe the activities to be undertaken that are compatible with UEY? | 19 | 2 | 0 | 10% |
| 15 | identify expected results within a specific timeframe, and impact on objectives that are compatible with UEY? | 21 | 0 | 0 | 0% |
| 16 | identify products or outputs that are compatible with UEY? | 21 | 0 | 0 | 0% |
| 17 | identify process for evaluating results that are compatible with UEY? | 21 | 0 | 0 | 0% |
| 18 | outline process for dissemination of products/results that are compatible with UEY? | 21 | 0 | 0 | 0% |
| 19 | identify targetted beneficiaries that are compatible with UEY? | 21 | 0 | 0 | 0% |
| 20 | contain a completed budget forecast using the budget template? | 21 | 0 | 0 | 0% |
| 21 | provide a monthly cashflow forecast? | 0 | 21 | 0 | 100% |
| 22 | disclose involvement of former public servants under Conflict of Interest and Post-employment Guidelines? | 0 | 21 | 0 | 100 % |
| 23 | Is the project for a 3 year period? (some exceptions allowed) | 21 | 0 | 0 | 0% |
| 24 | Is there a budget worksheet for each fiscal year? | 21 | 0 | 0 | 0% |
| 25 | Is there evidence of contribution of "in-kind" funds? | 21 | 0 | 0 | 0% |
| 26 | Are all expenses included in the budget eligible? | 21 | 0 | 0 | 0% |
| 27 | Are wages broken out on the basis of an hourly rate and # of hours? | 19 | 2 | 0 | 10% |
| 28 | Are costs for implementing the knowledge dissemination plan & results based reporting included in the budget? | 21 | 0 | 0 | 0% |
| 29 | Does the proposal contain a completed narrative workplan covering the duration of the project which is compatible with UEY activities? | 21 | 0 | 0 | 0% |
| Does the workplan include: | |||||
| 30 | activities in order to be followed to achieve objectives and results? | 21 | 0 | 0 | 0% |
| 31 | key milestones? | 20 | 1 | 0 | 5% |
| 32 | start and end dates for each activity? | 21 | 0 | 0 | 0% |
| 33 | human resource requirements for each activity? | 21 | 0 | 0 | 0% |
| 34 | dates progress and final reports? | 21 | 0 | 0 | 0% |
| 35 | explanation of knowledge dissemination plans and required budget? | 20 | 1 | 0 | 5% |
| 36 | explanation of results-based reporting plans and required budget? | 21 | 0 | 0 | 0% |
| Does the proposal contain a complete list of legal signatures, combinations and thresholds required for: | |||||
| 37 | contract purposes? | 21 | 0 | 0 | 0% |
| 38 | cheque purposes? | 21 | 0 | 0 | 0% |
| 39 | financial and/or other reports submitted to HRSDC? | 21 | 0 | 0 | 0% |
| 40 | Are the accounting practices adequately disclosed in the proposal? | 21 | 0 | 0 | 0% |
| 41 | Does the proposal contain a declaration of amounts owing in default to the Govt. of Canada? | 21 | 0 | 0 | 0% |
| 42 | Is the proposal signed and certified by the applicant? | 21 | 0 | 0 | 0% |
| Project Development and Assessment | |||||
| 43 | Is a completed Internal Review Assessment Scoring Document (conducted by UEY staff & SDC regional rep.) on file? | 21 | 0 | 0 | 0% |
| 44 | Is a completed External Assessment Package (conducted by external stakeholders) on file? | 21 | 0 | 0 | 0% |
| 45 | Is a combined final ranking on file? | 21 | 0 | 0 | 0% |
| 46 | Is sufficient evidence on file to demonstrate that the Budget Forecast & Workplan were assessed/challenged to determine appropriateness, including other possible sources of funds? | 1 | 20 | 0 | 95% |
| 47 | Are all purchases of capital assets of $500 or more justified/negotiated and included in the agreement and amendment? | 21 | 0 | 0 | 0% |
| 48 | Are all shortcomings in the proposal assessment rectified at the agreement negotiation stage before proceeding with the contribution agreement? | 7 | 14 | 0 | 67% |
| 49 | Was an environmental assessment carried out and placed on file? | 21 | 0 | 0 | 0% |
| 50 | Is there proof on file that the applicant was checked in DARS for outstanding debt to HRSDC? | 21 | 0 | 0 | 0% |
| Recommendation & Approval | |||||
| 51 | Is there a recommendation on file? | 21 | 0 | 0 | 0% |
| 52 | If there is a recommendation, does it contain a rationale for funding? | 21 | 0 | 0 | 0% |
| 53 | If the project was approved, is a copy of the appropriate Approval on file? | 21 | 0 | 0 | 0% |
| 54 | Was approval obtained on or prior to the project start date? | 21 | 0 | 0 | 0% |
| 55 | Is a copy of the decision letter to the recipient on file? | 21 | 0 | 0 | 0% |
| Contribution Agreement | |||||
| See App. C TB Policy on Transfer Payments | |||||
| 56 | Is the Contribution Agreement identifying the recipient on file? | 21 | 0 | 0 | 0% |
| 57 | Does it include a schedule of project activities (including timeframe and milestones)? | 21 | 0 | 0 | 0% |
| 58 | Does it include measurable and quantifiable objectives (i.e., expected results are clearly described); | 21 | 0 | 0 | 0% |
| 59 | Does it include eligible costs? | 21 | 0 | 0 | 0% |
| 60 | Does it include an appropriate expenditure breakdown? | 21 | 0 | 0 | 0% |
| 61 | Does it include a schedule and basis of payment? | 21 | 0 | 0 | 0% |
| 62 | Does it include the duration of the agreement which is consistent with the program terms and conditions? | 21 | 0 | 0 | 0% |
| 63 | Does it include terms for termination with and without cause? | 21 | 0 | 0 | 0% |
| 64 | Does it include a clause to limit the liability of the government in the case where the recipient is entering into a loan, a capital lease or other long term obligation in relation to the project for which the contribution is provided? | 21 | 0 | 0 | 0% |
| 65 | Does it include procedures to be followed to recover payments should the recipient be in default of the provisions of the agreement? | 21 | 0 | 0 | 0% |
| 66 | Does it include an indemnification clause for the benefit of the Crown? | 21 | 0 | 0 | 0% |
| 67 | Does it include a clause that requires the recipient not represent itself, including in any agreement with a third party, as a partner or agent of the Crown? | 21 | 0 | 0 | 0% |
| 68 | Does it include a requirement that no member of the House of Commons shall be admitted to any share or part of this Funding Agreement or any benefit arising therefrom? | 21 | 0 | 0 | 0% |
| 69 | Does it include a requirement that it is a term of this Funding Agreement that no current or former public office holder or public servant who is not in compliance with the Conflict of Interest and Post-employment Code for Public Office Holders or the Conflict of Interest and Post-Employment Code for the Public Service shall derive a direct benefit from this Agreement? | 21 | 0 | 0 | 0% |
| 70 | Does it include a requirement that any Payment by Canada under the Agreement is subject to there being an appropriation by Parliament for the fiscal year in which the payment is to be made? | 21 | 0 | 0 | 0% |
| 71 | Does it include terms for agreement amendments? (ie. changes to $, scope of activities, capitals assets, duration etc.) | 21 | 0 | 0 | 0% |
| 72 | Does it include terms for capital assets and their disposal (ie. HRSDC retains the right to approve disposal of capital assets of $500+ at project end)? | 21 | 0 | 0 | 0% |
| 73 | Does it include a requirement for the recipient to provide a declaration regarding the registration of lobbyists? | 21 | 0 | 0 | 0% |
| 74 | For contributions in excess of $100,000, does it include a requirement for the recipient to declare any and all sources of proposed funding for the project before and/or shortly after the commencement of the agreement, as well as upon completion of the project? | 21 | 0 | 0 | 0% |
| 75 | Does it include a provision for repayment should total government assistance exceed the amount anticipated? | 21 | 0 | 0 | 0% |
| 76 | Does it include a requirement for the recipient to declare amounts owing to the federal government, and states that HRSDC funding may be offset against amounts owing? | 21 | 0 | 0 | 0% |
| 77 | Is it for an amount that does not exceed the amount approved by the Minister? | 21 | 0 | 0 | 0% |
| 78 | Was it signed by HRSDC after it was signed by the recipient? | 19 | 2 | 0 | 10% |
| 79 | Is it signed on or after project approval date? | 21 | 0 | 0 | 0% |
| 80 | Is it signed before the start date? | 19 | 2 | 0 | 10% |
| 81 | Is it signed before expenses have been incurred? | 21 | 0 | 0 | 0% |
| 82 | Does it include a forecast of cash flow for the original agreement and any financial amendments? | 21 | 0 | 0 | 0% |
| 83 | Does it include all required amendments? Note that an amendment to an agreement is required: if there are any changes in HRSDC's overall contribution; where there are significant changes in the scope of activities; when capital assets are acquired that we | 21 | 0 | 0 | 0% |
| 84 | Are the amendments reasonable in both $value and number of amendments? (Rule of thumb: 50% of original value is excessive) | 17 | 2 | 2 | 10% |
| 85 | Does it include the appropriate authorizations for the original agreement (ie, HRSDC delegation of authority & signatures as well as the recipient)? | 17 | 2 | 2 | 10% |
| 86 | Does it include the appropriate authorizations for any subsequent amendments (ie, HRSDC & recipient. Also note that HRSDC authorization requirements are different if amendments of 25% or less of original funding vs. more than 25%) | 19 | 0 | 2 | 0% |
| 87 | Does it include a clause stating HRSDC may do an audit of costs claimed? | 21 | 0 | 0 | 0% |
| 88 | Does it include a clause stating HRSDC funds are not to be used to pay for persons assisting the recipient in obtaining HRDC contribution funding? | 21 | 0 | 0 | 0% |
| 89 | Where the recipient further distributes contribution amount does the agreement include all provisions as stated in Appendix C of the TB payment policy? | 0 | 0 | 21 | 0% |
| 90 | Does it include a requirement for the recipient to repay any overpayments, unexpended balances and disallowed expenses? | 21 | 0 | 0 | 0% |
| 91 | Was commitment completed prior to HRSDC signing the agreement? | 21 | 0 | 0 | 0% |
| 92 | Was commitment of funds pertaining to any financial amendments completed prior to HRSDC signing the amendments? | 19 | 0 | 2 | 0% |
| Monitoring | |||||
| 93 | Is monitoring carried out in accordance with the Risk Assessment and Monitoring Plan on file that was completed prior to the start of the agreement? | 21 | 0 | 0 | 0% |
| 94 | Question not used | 0 | 0 | 21 | 0% |
| 95 | Does evidence suggest that project activities and costs were monitored for compliance with the agreement? | 20 | 0 | 1 | 0% |
| 96 | Where applicable, are contributions from other partners and in-kind contributions verified? | 6 | 0 | 15 | 0% |
| 97 | Are follow-ups to the monitoring report(s) initialled by both parties or completed and copy left on site? | 5 | 1 | 15 | 5% |
| Payments | |||||
| 98 | Did the Program Officer responsible for administering the agreement ensure prior to expenditure certification that: | ||||
| 99 | the payment is in accordance with the agreement/cash flow? | 21 | 0 | 0 | 0% |
| 100 | charges not payable have been eliminated? | 19 | 2 | 0 | 10% |
| 101 | purchases of capital assets of $500 or more were pre-approved? | 21 | 0 | 0 | 0% |
| 102 | the account has not previously been paid in whole or in part? | 21 | 0 | 0 | 0% |
| 103 | the calculation of the payment is mathematically correct? | 21 | 0 | 0 | 0% |
| 104 | the payment does not exceed the amount to be withheld, as specified by the terms and conditions of the program and the contribution agreement? | 11 | 0 | 10 | 0% |
| 105 | the proper expenditure coding has been applied? | 21 | 0 | 0 | 0% |
| 106 | the Requisition for Payment or approved claim form is completed? | 21 | 0 | 0 | 0% |
| 107 | before final payment, all over payments from the agreement have been recovered or arrangements have been made with the recipient to collect the overpayment(s) of the agreement? | 0 | 0 | 21 | 0% |
| If advance payments were made: | |||||
| 108 | was the sponsor eligible for an advance? | 9 | 0 | 12 | 0% |
| 109 | were the advance amounts & timing compliant with TB and FAS Policy? | 9 | 0 | 12 | 0% |
| 110 | were only April forecast expenditures paid out of previous year funds? | 12 | 0 | 9 | 0% |
| 111 | Did claim amounts for April support the advance amount issued from the previous fiscal year? | 12 | 0 | 9 | 0% |
| 112 | Were all payments certified following the correct delegation of authority according to Section 34? | 21 | 0 | 0 | 0% |
| 113 | Were the recipient's signing requirements as per the agreement respected? | 21 | 0 | 0 | 0% |
| Compliance with Ts&Cs | |||||
| 114 | Does the project and project activities meet the program Terms and Conditions? | 3 | 18 | 0 | 86% |
| Close-out | |||||
| 115 | Were assets disposed of in accordance with the agreement and the Terms and Conditions? | 0 | 0 | 21 | 0% |
| 116 | Was the final contribution paid after receipt of the final claim form, unless early payment was justified on file and allowable by Ts and Cs (i.e., appropriate holdback provisions were utilized)? | 0 | 0 | 21 | 0% |
| 117 | Is it fair to say that nothing in the file indicates of a lack of open accessibility to the Program? | 21 | 0 | 0 | 0% |
APPENDIX B REJECTED FILE REVIEW RESULTS
| Yes | No | N/A | Incidence Rate | ||
| Screening Requirements | |||||
| 1 | Was the proposal postmarked and/or date stamped by the closing date? (4:00 p.m. EST April 11, 2005) | 5 | 10 | 0 | 67% |
| Organization eligibility (non-profit and actively involved in social development issues) Is there: | |||||
| 2 | evidence of non-profit status, ie letters patent or corporate charter ? | 9 | 5 | 1 | 33% |
| 3 | evidence of organization's mandate, ie official mission statement ? | 10 | 4 | 1 | 27% |
| 4 | If Quebec project, is a completed M-30 submitted? | 2 | 0 | 13 | 0% |
| Is there evidence to support that the community fits program criteria: | |||||
| 5 | Place based? (ie. Contiguous geography) | 14 | 0 | 1 | 0% |
| 6 | Have an existing community coalition? | 11 | 3 | 1 | 20% |
| 7 | Support of all participating schools and school boards? | 5 | 9 | 1 | 60% |
| 8 | Have suitable potential candidates for UEY coordinator? | 9 | 5 | 1 | 33% |
| 9 | At least 300 5-year-olds entering S Kindergarten in 2005-06? | 7 | 7 | 1 | 47% |
| 10 | Given the above, was the proposal screened in/out appropriately? | 8 | 7 | 0 | 47% |
| Does the proposal: | |||||
| 11 | include a description of community and key partners | 9 | 1 | 5 | 7% |
| 12 | address one or more program objectives? | 6 | 4 | 5 | 27% |
| 13 | fit within Program priorities? | 5 | 5 | 5 | 33% |
| 14 | describe the activities to be undertaken that are compatible with UEY? | 4 | 6 | 5 | 40% |
| 15 | identify expected results within a specific timeframe, and impact on objectives that are compatible with UEY? | 5 | 4 | 6 | 27% |
| 16 | identify products or outputs that are compatible with UEY? | 4 | 5 | 6 | 33% |
| 17 | identify process for evaluating results that are compatible with UEY? | 3 | 6 | 6 | 40% |
| 18 | outline process for dissemination of products/results that are compatible with UEY? | 5 | 4 | 6 | 27% |
| 19 | identify targetted beneficiaries that are compatible with UEY? | 7 | 3 | 5 | 20% |
| 20 | contain a completed budget forecast using the budget template? | 7 | 3 | 5 | 20% |
| 21 | provide a monthly cashflow forecast? | 0 | 10 | 5 | 67% |
| 22 | disclose involvement of former public servants under Conflict of Interest and Post-employment Guidelines? | 0 | 10 | 5 | 67% |
| 23 | Is the project for a 3 year period? (some exceptions allowed) | 8 | 2 | 5 | 13% |
| 24 | Is there a budget worksheet for each fiscal year? | 8 | 2 | 5 | 13% |
| 25 | Is there evidence of contribution of "in-kind" funds? | 5 | 5 | 5 | 33% |
| 26 | Are all expenses included in the budget eligible? | 3 | 7 | 5 | 47% |
| 27 | Are wages broken out on the basis of an hourly rate and # of hours? | 6 | 4 | 5 | 27% |
| 28 | Are costs for implementing the knowledge dissemination plan & results based reporting included in the budget | 5 | 4 | 6 | 27% |
| 29 | Does the proposal contain a completed narrative workplan covering the duration of the project which is compatible with UEY activities? | 5 | 5 | 5 | 33% |
| Does the workplan include: | |||||
| 30 | activities in order to be followed to achieve objectives and results? | 8 | 1 | 6 | 7% |
| 31 | key milestones? | 7 | 2 | 6 | 13% |
| 32 | start and end dates for each activity? | 6 | 3 | 6 | 20% |
| 33 | human resource requirements for each activity? | 4 | 4 | 7 | 27% |
| 34 | dates progress and final reports? | 6 | 3 | 6 | 20% |
| 35 | explanation of knowledge dissemination plans and required budget? | 6 | 3 | 6 | 20% |
| 36 | explanation of results-based reporting plans and required budget? | 2 | 7 | 6 | 47% |
| Does the proposal contain a complete list of legal signatures, combinations and thresholds required for: | |||||
| 37 | contract purposes? | 10 | 0 | 5 | 0% |
| 38 | cheque purposes? | 10 | 0 | 5 | 0% |
| 39 | financial and/or other reports submitted to HRSDC? | 10 | 0 | 5 | 0% |
| 40 | Are the accounting practices adequately disclosed in the proposal? | 10 | 0 | 5 | 0% |
| 41 | Does the proposal contain a declaration of amounts owing in default to the Govt. of Canada? | 10 | 0 | 5 | 0% |
| 42 | Is the proposal signed and certified by the applicant? | 10 | 0 | 5 | 0% |
| Project Development and Assessment | |||||
| 43 | Is a completed Internal Review Assessment Scoring Document (conducted by UEY staff & SDC regional rep.) on file? | 10 | 0 | 5 | 0% |
| 44 | Is a completed External Assessment Package (conducted by external stakeholders) on file? | 10 | 0 | 5 | 0% |
| 45 | Is a combined final ranking on file? | 10 | 0 | 5 | 0% |
| 46 | Is sufficient evidence on file to demonstrate that the Budget Forecast & Workplan were assessed/challenged to determine appropriateness, including other possible sources of funds? | 1 | 0 | 14 | 0% |
| 47 | Are all purchases of capital assets of $500 or more justified/negotiated and included in the agreement and amendment? | 0 | 0 | 15 | 0% |
| 48 | Are all shortcomings in the proposal assessment rectified at the agreement negotiation stage before proceeding with the contribution agreement? | 0 | 0 | 15 | 0% |
| 49 | Was an environmental assessment carried out and placed on file? | 0 | 0 | 15 | 0% |
| 50 | Is there proof on file that the applicant was checked in DARS for outstanding debt to HRSDC? | 0 | 0 | 15 | 0% |
| Recommendation & Approval | |||||
| 51 | Is there a recommendation on file? | 15 | 0 | 0 | 0% |
| 52 | If there is a recommendation, does it contain a rationale for funding? | 0 | 0 | 15 | 0% |
| 53 | If the project was approved, is a copy of the appropriate Approval on file? | 0 | 0 | 15 | 0% |
| 54 | Was approval obtained on or prior to the project start date? | 0 | 0 | 15 | 0% |
| 55 | Is a copy of the decision letter to the recipient on file? | 13 | 0 | 2 | 0% |
APPENDIX C REVIEW OBJECTIVES, CRITERIA AND METHODOLOGY
1. REVIEW OBJECTIVES AND CRITERIA
1.1 Management Control Framework Objective
To determine whether or not management practices and systems are developed and aligned to support program policy and objectives; identify and manage risks; and, are operating as intended.
Criteria
1.1.1 Leadership should demonstrate:
| a. | effectiveness in setting strategic direction; |
| b. | effectiveness in integrating the Voluntary Sector Accord and Code of Good Practice on Funding and the Code of Good Practice on Policy Dialogue into program policy, design and processes; |
| c. | effectiveness in communicating program results that are clearly defined and reported to the appropriate audience; |
| d. | effectiveness in exercising functional direction; |
| e. | commitment and involvement in priority setting initiatives (including internal/external stakeholder consultative processes); and |
| f. | clear communication of strategic and functional direction to all levels of the organization and to program recipients. |
1.1.2 Strategic Planning should be carried out which:
| a. | outlines the initiative mandate, objectives and intended outcomes in line with the branch vision, goals and priorities; |
| b. | establishes and tracks delivery on priorities, including improvement initiatives, in line with the stated objectives and available resources; |
| c. | involves internal/external partners and stakeholders; |
| d. | regularly considers present and future internal and external environmental aspects and risk factors; and |
| e. | is communicated to, and understood by, staff at all levels of the organization. |
1.1.3 Performance measures should be:
| a. | aligned with objectives, priorities and key processes; |
| b. | balanced between qualitative and quantitative measures; |
| c. | designed to be aggregated to the appropriate level of management; |
| d. | supported by reliable and timely data and information; |
| e. | sufficient to report on results at all levels; and |
| f. | linked to accountability frameworks. |
1.1.4 Periodic monitoring and reporting on program performance and program delivery costs should be carried out.
1.1.5 Resource Allocation Models should be developed which are:
| a. | aligned with priorities; |
| b. | based upon sound assumptions; |
| c. | applied consistently; and |
| d. | reviewed/updated as appropriate. |
1.1.6 Program and financial authorities, accountabilities and responsibilities should be clearly defined, co-ordinated to achieve objectives of the Initiative and communicated to all appropriate levels of the organization.
1.1.7 Policies of the Initiative should be developed in a timely manner, clearly formulated and updated as required. They should be communicated throughout the organization in sync with applicable changes to operational procedures.
1.1.8 Operational planning should be carried out that considers linkages to strategic priorities, the objectives for the business unit are aligned with branch objectives, allocated resources, forecasted expenditures, accountabilities, timeframes and availability of data and other essential information.
1.1.9 Program tools, operational policy, and processes should support a streamlined approach to the application of departmental accountability requirements.
1.1.10 Effective communication strategies should be developed and implemented to ensure clear and consistent messaging to and feedback from, recipients, staff, partners and stakeholders.
1.1.11 Procedures should exist to ensure effectiveness in recruiting, selecting, training, and managing the performance of people.
1.1.12 Work processes should be effectively designed, streamlined and automated to providing quality service for funded organizations thereby ensuring efficient use of resources. Proposal intake and review processes should be appropriate to ensure fairness and transparency. They should be reviewed regularly and reengineered when appropriate. Changes should be documented and promptly reflected in operational procedures.
1.1.13 Effective internal controls should be in place to ensure:
| a. | appropriate and consistent decisions regarding recipient eligibility; |
| b. | payment accuracy; |
| c. | security and protection of personal information; |
| d. | deterrence of error, fraud and abuse; |
| e. | achievement of client service; |
| f. | accuracy of reporting; and |
| g. | identification and management of risks. |
1.1.14 Improvement processes should be implemented which:
| a. | are linked to strategic priorities and major business lines; |
| b. | allow for input by staff at all levels of the organization and by external stakeholders; and |
| c. | are monitored and measured for success. |
1.2 Information Management Objective
Information should be appropriate for the assessment of policy and program design and the administration of UEY, ensuring program integrity for reporting purposes.
Criteria
1.2.1 Information should be accurate, complete, timely and relevant for understanding business risks and development of strategies to manage the program.
1.2.2 Information and reports should be available on a regular basis for effective management.
1.2.3 Data integrity and validity should be maintained for all information and reports generated.
1.2.4 Management information should be used for program performance.
1.2.5 Relevant information should be used to manage resources.
1.2.6 Information should be used to assess operational efficiency and effectiveness.
1.2.7 Information provided for recipients should be timely, easily accessible, understandable and transparent.
1.2.8 Where applicable, data should be shared appropriately within HRSDC and with other partners/ government departments.
1.2.9 Effective record retention, storage and disposal strategies should be used.
1.3 Operational Compliance Objective
To determine whether or not UEY is managed in accordance with appropriate acts, regulations, policies, directives and guidelines:
Criteria
1.3.1 HRSDC's Operations Guide on Grants and Contributions to support the administration of funding for UEY should adhere to the Treasury Board Transfer Payment Policy and the Financial Administration Act and Regulations and seek to clarify, not exceed the intent of the policy and statutory framework for managing grants and contribution programs.
1.3.2 Departmental policies to guide the administration of UEY should be developed and updated supporting line with appropriate legislation and Treasury Board policies.
1.3.3 Staff administering UEY, should follow departmental policies, procedures, guidelines and the program's terms and conditions.
2. SCOPE
There were three components of this early implementation review:
- General Program Management;
- Process Control Mapping and Analysis; and
- File Review.
This review entailed work primarily at National Headquarters; there was some interaction with Service Canada representatives in the Regions regarding their role in the assessment of UEY proposals. Recipients of program funding were contacted to elicit their input on program design and administration. Specific recipients were chosen based on a number of factors. Program officials, rather than auditors, made the necessary arrangements with recipients. This review encompassed all HRSDC UEY activities from December 13, 2004 to the present.
3. METHODOLOGY
The first component of this review primarily entailed document review and interviews with management, staff and contribution recipients.
The second component involved the development of detailed process maps identifying all internal control points and an analysis of the control framework.
The third component involved reviewing:
- a sample of UEY proposals (approved and not) to assess the effectiveness of the control framework in place for the assessment, recommendation and approval phases; and
- all contribution agreements in place to assess the effectiveness of the control framework throughout the project life cycle. Files had not undergone a complete project life cycle; they were assessed to the point possible.
This combination of various methodologies will provide management with a high level of assurance and inform future work.
APPENDIX D MANAGEMENT ACTION PLAN
| Review Recommendations | Corrective Management Action Plan | Expected Completion Date | Responsibility |
| 3. The Understanding the Early Years Management Team should ensure that appropriate parental permissions are obtained to allow the sharing of each community's Early Development Instrument data file with their recipient organization. | UEY has been working with Access to Information and Privacy (ATIP) since early 2006 on proper informed consent for parents and data sharing requirements. As a first step, UEY used an ATIP- approved parent consent letter for the Parent Interviews and Direct Assessments of Children Survey (PIDACS) for the 21 UEY communities. For the next 15 communities, an ATIP- approved informed consent letter will be used for the EDI and PIDACS data collections scheduled in 2009. | Fall 2008 | Liz Nieman |
| 4. The Understanding the Early Years Management Team should establish policies and procedures for retention, storage and disposal of UEY documentation particularly in light of privacy concerns. | Discussions with HRSDC's Security and Archives will be undertaken this fiscal year to discuss issues surrounding procedures for retention, storage and disposal of UEY documentation. A security protocol will then be developed in compliance with implementing a Records and Documentation Management System and its implementation is expected to be rolled out in 2007-2008 for the Income Security and Social Development Branch. | 2007/08 | André Bordeleau |
| 5. The Understanding the Early Years Management Team should amend its contracting and funding practices for service contracts and Contribution Agreements to ensure compliance with the Treasury Board Contracting Policy and the Treasury Board Policy on Transfer Payments. | The issue identified in the Early Implementation Review has been rectified. For the 15 communities approved under the latest Call for Proposals, teacher replacement costs and related data collection activities will be covered under a service contract, in compliance with the TB contracting and transfer payments policies. | In effect New contribution agreements signed in May and June 2007 |
André Bordeleau UEY project officers |
| 6. The Understanding the Early Years Management Team should ensure that a copy of the proposal with the date received stamp is retained in the project file. | For the last Call for Proposals, five copies of a proposal had to be submitted. To ensure that a date stamped copy was placed in the project file; all five copies were date stamped. This practice will continue for the last Call for Proposals process as well. | Completed - will continue for next CFP | Hoda Sawaya UEY project officers |
| 7. The Understanding the Early Years Management Team should ensure consistent application of specified criteria to all proposals. | For the most recent Call for Proposals and as part of the initial screening process, UEY project officers used a revised template to guide decisions on whether to screen in or out any proposal according to the mandatory requirements of the UEY Initiative. UEY project officers then presented their assessments to UEY management as a way to confirm whether the mandatory requirements were met or not. The completed screening template, with comments, has been placed in each project file. A proposal had to meet all requirements in order to move forward to next stage of the assessment process. Staff who conducted the screening of the most recent Call for Proposals were trained on this matter. |
Screening Process and Template completed in September 2006 | UEY management and project officers |
| 8. The Understanding the Early Years Management Team should strengthen the assessment function as a critical element of project administration/negotiation and provide staff with the necessary training and support to perform their tasks with due diligence. | Greater emphasis is being placed on the assessment function for budgets and workplans that are assessed as part of the UEY Call for Proposals process and on-going financial administration of contribution agreements. For example, before signing the contribution agreements resulting from the Call for Proposals (which closed September 29th 2006), UEY project officers reviewed proposals for areas that needed to be improved, informing the recipients of these changes and negotiating the terms of the contribution agreement, including the budget and workplan. In addition, training on the art of negotiation is being provided to Project Officers to strengthen the negotiation function to perform their tasks with due diligence. | Contribution agreements signed May and June 2007 | Donald Nadeau UEY Project Officers |