Audit of the Time and Attendance Processing System Development
Description
February 17, 1999AUDIT OF THE TIME AND ATTENDANCE PROCESSINGSYSTEM DEVELOPMENT PROJECT (II)Audit Report No. 99-011OFFICE OF AUDITSOFFICE OF INSPECTOR GENERAL912142582918531116104144BACKGROUNDDIRM'S CONTRACT MANAGEMENT NEEDS IMPROVEMENTContract InitiationRecommendationAPPENDIX I – CHRONOLOGY OF KEY TAPS DATESAPPENDIX II – CORPORATION COMMENTSAPPENDIX III – MANAGEMENT RESPONSES TO RECOMMENDATIONS14 CORPORATION COMMENTS AND OIG EVALUATIONDIRM'S INTERNAL CONTROL PROCESS WAS NOT EFFECTIVE13 RecommendationsContractor Oversight Not EffectiveRecommendationsRecommendationsInformationKey Assumption for Proceeding with TAPS Based on InaccurateFeasibility and Cost-Benefit of Alternative Solutions Not Considered KEY TAPS DECISIONS NOT BASED ON SDLC METHODOLOGYRESULTS OF AUDITOBJECTIVES, SCOPE, AND METHODOLOGYTABLE OF CONTENTSFebruary 17, 1999 W TO: Demitros, DirectorJohn Lynn, Acting DirectorFROM: David H. SUBJECT:Development Project (II)(Audit Report No. 99-011)Corporation’s (FDIC) Time and Attendance Processing System (TAPS) development project. resources.BACKGROUNDAudit of the Time and Attendance Processing System (TAPS)2with management and identified three issues that the FDIC needed to address to improve the (Audit Report No. 97-106). This report paralleled our earlier discussions Development Projectfinal audit report entitled regarding the TAPS development process to that point. On September 29, 1997, we issued ...
Sujets
Informations
| Publié par | Nucheg |
| Nombre de lectures | 61 |
| Langue | English |
Extrait
AUDIT OF THE TIME AND ATTENDANCE PROCESSING SYSTEM DEVELOPMENT PROJECT (II)
Audit Report No. 99-011 February 17, 1999
OFFICE OF AUDITS
OFFICE OF INSPECTOR GENERAL
TABLE OF CONTENTS
BACKGROUND OBJECTIVES, SCOPE, AND METHODOLOGY RESULTS OF AUDIT KEY TAPS DECISIONS NOT BASED ON SDLC METHODOLOGY Feasibility and Cost-Benefit of Alternative Solutions Not Considered Key Assumption for Proceeding with TAPS Based on Inaccurate Information Recommendations DIRM'S CONTRACT MANAGEMENT NEEDS IMPROVEMENT Contract Initiation Recommendations Contractor Oversight Not Effective Recommendations DIRM'S INTERNAL CONTROL PROCESS WAS NOT EFFECTIVE Recommendation CORPORATION COMMENTS AND OIG EVALUATION APPENDIX I CHRONOLOGY OF KEY TAPS DATES APPENDIX II CORPORATION COMMENTS APPENDIX III MANAGEMENT RESPONSES TO RECOMMENDATIONS
2 3 4 4 5 8 9 9 10 11 12 13 14 14 14 16 18 25
Federal Deposit Insurance Corporation W ashington, D.C. 20434
DATE: TO:
FROM:
SUBJECT:
February 17, 1999 Donald C. Demitros, Director Division of Information Resources Management John Lynn, Acting Director Division of Administration
David H. Loewenstein Assistant Inspector General
Office of Audits Office of Inspector General
Report EntitledAudit of the Time and Attendance Processing System Development Project (II) (Audit Report No. 99-011)
The Office of Inspector General (OIG) has completed an audit of the Federal Deposit Insurance Corporation’s (FDIC) Time and Attendance Processing System (TAPS) development project. This report presents a summary of the TAPS development project and serves as a “lessons learned document for the FDIC's use in managing future development projects, including the Corporation’s current efforts on a system to support the processing of personnel information. Our report includes eight recommendations for incorporating needed controls into the Division of Information Resources Management’s (DIRM) system development and contracting processes. FDIC’s lack of adherence to established and generally accepted system development life cycle (SDLC) procedures and DIRM’s ineffective contractor oversight practices contributed to the failure of TAPS and resulted in the unnecessary expenditure of significant corporate resources.
BACKGROUND The OIG initiated an audit of the FDIC’s TAPS development project in November 1996. In June 1997, we met with management to discuss our concerns and preliminary recommendations regarding the TAPS development process to that point. On September 29, 1997, we issued a final audit report entitledof the Time and Attendance Processing System (TAPS)Audit Development Project(Audit Report No. 97-106). This report paralleled our earlier discussions with management and identified three issues that the FDIC needed to address to improve the
2
TAPS development process. First, FDIC management did not have the information needed to make informed decisions regarding the development approaches for TAPS because the project team did not adhere to generally accepted system development methodologies when developing cost-benefit and feasibility analyses. In addition, FDIC management and project personnel did not have the information needed to properly manage the TAPS development effort because progress reports did not compare results being achieved to projected costs, benefits, and risks. Finally, the project team increased the risks associated with a successful completion of the project by deviating from accepted SDLC methodologies and performing design and development work before functional requirements were finalized. These issues seriously impaired management’s decision-making ability regarding the viability of the project and resulted in additional costs and resource consumption to re-perform many efforts already completed.
FDIC management agreed with our findings and recommendations and committed to following a structured approach for developing TAPS. On October 22, 1997, the FDIC's Audit Committee requested that the Office of Internal Control Management (OICM) perform a review to determine the effectiveness of the project’s internal controls and identify where internal controls may have broken down in the SDLC process. On March 18, 1998, OICM issued its report, which reiterated the issues identified by our office and contained several additional recommendations.
OBJECTIVES, SCOPE, AND METHODOLOGY
The objectives of the audit were to determine whether (1) the TAPS development was adhering to established and generally accepted SDLC procedures, (2) user requirements had been adequately defined, (3) system deliverables satisfied user requirements in a cost-effective and timely manner, and (4) adequate internal controls were incorporated into the design of the system. Because management discontinued the TAPS development effort before finalizing requirement and development activity, we were unable determine whether adequate internal controls had been incorporated into TAPS.
To accomplish our other audit objectives, we interviewed DIRM, Division of Administration (DOA), and contractor personnel responsible for developing TAPS. We also analyzed documentation prepared during the development process, including planning documents, project status reports, draft requirements documents, and design documents. In addition, we reviewed current policies and procedures related to the FDIC’s SDLC methodology and attended TAPS Steering Committee meetings and other TAPS project meetings. The TAPS Steering Committee was comprised of senior management officials who made decisions on approaches regarding TAPS. Because of the time-sensitive nature of the TAPS development project, we met with DIRM and DOA personnel frequently throughout the audit to discuss our preliminary recommendations.
We conducted our audit between November 1996 and August 1998 in accordance with generally accepted government auditing standards.
3
RESULTS OF AUDIT
Although management committed to improving FDIC's development practices related to TAPS in response to recommendations made by our office and OICM, DIRM and DOA continued to deviate from FDIC’s SDLC process. Throughout our fieldwork, we advised TAPS program personnel and the TAPS Steering Committee about the project's lack of adherence to the FDIC's SDLC process. Specifically, we raised concerns about the quality, completeness, and accuracy of cost-benefit information being provided to management for decision-making purposes. We informed DIRM and DOA management that the lack of current, accurate, and complete feasibility and cost-benefit information on TAPS was seriously impairing senior management's decision-making ability regarding the project. However, management disregarded our concerns and deviated from generally accepted SDLC approaches throughout the life of the project.
Following our earlier report, DIRM and DOA again proceeded with design and development work before fully defining user requirements. In addition, the FDIC did not effectively manage the development of TAPS, and contractor oversight was not effective. These actions resulted in the unnecessary expenditure of at least $6.5 million and ultimately contributed to management's decision to discontinue the project.
In June 1998, the FDIC discontinued the TAPS development effort because of design complexities caused by DIRM’s failure to freeze requirements for the system. Shortly after the project was discontinued, we met with the Directors of DIRM and DOA to discuss our final conclusions regarding TAPS and to provide these management officials with our proposed recommendations for managing future information technology (IT) efforts. These recommendations, which are contained in this report, are aimed at ensuring that (1) management has the information needed to make informed decisions regarding whether and how to proceed with future development efforts, (2) DIRM disciplines itself to completing initial development phases before proceeding to subsequent phases of development projects, and (3) project status information and contractor oversight is improved so that management is aware of changes in schedule, cost, and risk. Many of the recommendations contained in this report are similar to recommendations contained in earlier OIG reports. We are restating the recommendations in this report because of DIRM’s failure to effectively address the recommendations in the past.
KEY TAPS DECISIONS NOT BASED ON SDLC METHODOLOGY
Shortly following our initial involvement with the TAPS project in 1996, we began raising concerns about the quality, completeness, and accuracy of cost-benefit information provided to management for decision-making purposes. The lack of current, accurate, and complete feasibility and cost-benefit information seriously impaired management's decision-making ability regarding TAPS and resulted in the unnecessary expenditure of significant corporate resources. Despite management’s commitment to improve its adherence to accepted SDLC methodologies and, thereby, improve information supporting management decisions, the FDIC continued to deviate from accepted practices throughout the project. The FDIC’s actions throughout the TAPS development process continued to increase the risk associated with the project, resulted in
4
ever-increasing expenditures of unnecessary funds, and ultimately resulted in the discontinuance of TAPS development efforts.
Feasibility and Cost-Benefit of Alternative Solutions Not Considered
Throughout the development process, DIRM and DOA repeatedly took actions and expended funds toward the in-house development of an automated time and attendance system without formally evaluating the feasibility or cost-benefit of alternative solutions. Despite encountering significant problems throughout the project and committing to improve the planning process related to TAPS, DIRM and DOA did not re-evaluate their original course of action.
The purpose of a feasibility study is to provide senior management with: (1) an analysis of the project's objectives, requirements, and system concepts; (2) an evaluation of alternative approaches; and (3) a recommended approach. The purpose of a cost-benefit analysis (CBA) is to provide management with adequate cost and benefit information to analyze and evaluate alternative approaches. Because the structures of feasibility studies and CBAs are so similar, FDIC's SDLC Manual allows them to be combined.
DIRM and DOA developed an initial risk assessment, dated June 1995, and a CBA, dated July 1995, to support their decision to proceed with the in-house development of TAPS. However, these analyses did not use full life cycle cost data or formally evaluate alternative solutions, such as implementing only the Corporate Time and Attendance Worksheet (CTAW), modifying a commercial-off-the-shelf system, or modifying an existing system developed by another federal entity. In addition, TAPS cost-benefit information did not evaluate technical, cost, or schedule risks associated with the project or revisit original assumptions when significant changes took place in the project's scope, cost, and schedule. We also noted that estimated cost savings attributed to the development and implementation of TAPS were overly optimistic.
We met with DOA's TAPS program manager on May 2, 1997 to discuss our concerns regarding the limitations of the TAPS risk assessment and cost-benefit analysis. We reiterated our concerns to DOA's TAPS program manager on June 11, 1997 when significant changes were taking place in the project's scope, cost, and schedule. We advised the DOA project manager that alternative solutions should be formally evaluated and presented to senior management before proceeding with further TAPS development activities. Despite a verbal commitment to address our concerns, DIRM and DOA management awarded a $1.9 million contract to continue the in-house development of TAPS on July 24, 1997 without the benefit of a thorough and enhanced CBA or feasibility study.
On September 29, 1997, we reported on our concerns regarding the limitations of the TAPS CBA and risk assessment in our report entitledAudit of the Time and Attendance Processing System (TAPS) Development Project(Report No. 97-106). noted that the TAPS CBA was We not supported by adequate documentation and that the assumptions underlying the analysis were based on inaccurate and outdated information. We recommended in the report that DIRM and DOA revisit the TAPS CBA and review and update it, as necessary, throughout the development life cycle. We also recommended that DIRM and DOA evaluate the cost-benefit of alternative
5
solutions to TAPS before continuing with additional development work. DIRM and DOA formally agreed to implement our recommendations and committed to following a structured approach for developing TAPS.
In November 1997, OICM initiated a review of the TAPS development project to determine the effectiveness of its internal controls and to determine where internal controls may have broken down. OICM's report, dated March 18, 1998, reiterated the concerns expressed by our office. OICM also determined that DIRM and DOA had informally considered three alternatives to TAPS before the project was initiated, but that this effort was cursory in nature and not adequately documented.
In December 1997, 5 months after awarding a contract to continue in-house development of TAPS, DIRM and DOA completed revisions to the TAPS CBA. However, these revisions did not include a formal evaluation of alternative solutions. DIRM’s Deputy Director stated that the significant cost savings projected for in-house development of TAPS would make alternative solutions non-viable. However, FDIC could not make informed decisions on the viability of other alternatives without such a study. Further, the projected cost savings for TAPS continued to be overly optimistic. The projected cost savings were outdated and based on a limited analysis performed in 1995. The projected savings were based primarily on a reduction in employee time to enter and process time and attendance information. However, the time savings projections were unsupported and optimistic. Further, some of the projected timesavings still being cited by DIRM and DOA in 1997, even if realistic, would have already been achieved through the implementation of the FDIC’s CTAW in 1996.
OICM recommended in its March 18, 1998 report that DIRM and DOA document the required components of a CBA and perform reviews of the projections and assumptions at various points during the SDLC. During this same time frame, FDIC was encountering significant problems in addressing TAPS requirements and designing a system architecture. A system architecture provides the structure for data and automated processes that the application will employ to support user requirements. However, despite these problems and management's commitment to address the concerns raised in the OICM and OIG reports, DIRM and DOA increased the value of the existing contract by 25 percent on March 18, 1998 without reconsidering the costs and benefits cited in the December 1997 CBA.
On March 31, 1998, DIRM documented a cursory review of three alternatives to TAPS that had been performed in 1995. The 1995 analysis had concluded that the alternatives were not viable solutions for the FDIC's time and attendance requirements. However, this analysis was flawed because the FDIC’s time and attendance requirements had not been defined at that time. In addition, the FDIC’s actions to merely document prior analyses did not address the status of alternatives in 1998, because TAPS requirements had been significantly modified on several occasions throughout the development effort. When FDIC documented this 1995 analysis in March 1998, it did not evaluate new potential solutions or re-evaluate potential solutions considered immature in 1995.
6
In May and June 1998, DOA began to question the assumptions underlying the projected cost savings associated with the development and implementation of TAPS. On May 12, 1998, DIRM and DOA revised the estimated cost savings attributed to TAPS from $15.2 million to $12.9 million over 5 years. DIRM and DOA further revised the estimated costs savings of TAPS on May 19, 1998 from $12.9 million to $1.5 million over 5 years. This more realistic evaluation of TAPS cost savings should have been performed as early as June 1997, when significant changes began taking place in the project's scope, cost, and schedule. Management would have had more accurate and meaningful information on which to base its decisions had such an analysis been performed in June 1997 when TAPS development efforts were being re-directed because of significant problems or at other times when major changes occurred in TAPS risks, costs, and schedules.
On May 21, 1998, DIRM and DOA awarded two additional contracts valued at approximately $1.8 million to continue development of TAPS, again without the benefit of a thorough and enhanced CBA. The TAPS Steering Committee justified the continued development of TAPS, despite the drastic reduction in estimated cost savings, on the premise that TAPS would "correct a deficiency in controls that was identified in a 1995 General Accounting Office (GAO) audit." However, as discussed in the following section of this report, this information was not accurate because GAO had no outstanding issues relating to the FDIC's time and attendance processes after 1995. We advised the TAPS Steering Committee that the deficiencies cited by GAO in its 1995 and prior year audit reports had already been corrected by FDIC in 1996. However, Steering Committee members disputed our statements.
On June 30, 1998, after expending at least $6.5 million on TAPS development and obtaining only a functional requirements document and external design document, the TAPS Steering Committee decided to discontinue the project.1 In July 1998, DIRM and DOA began researching the feasibility of an integrated personnel system to be called the Corporate Human Resources Information System (CHRIS).
FDIC management’s inability to make informed decisions regarding TAPS development can be attributed, in part, to confusion on the part of DIRM and DOA officials regarding the FDIC's own SDLC procedures. In a February 19, 1998 memorandum discussing OICM's review, the Deputy Directors of DIRM and DOA stated "There was no FDIC SDLC in 1995." The officials also stated "There are two versions of the FDIC SDLC, a March 1996 version and a July 1997 version; the March 1996 has no standard CBA format or structure and the July 1997 version does not require a CBA for any project."
Despite the assertions of these officials, the FDIC did have a SDLC process in 1995, the Electronic Data Processing (EDP) Project Guide. The FDIC's EDP Project Guide, which was based on the METHOD/1 SDLC methodology that FDIC purchased from Arthur Andersen in 1989, required a feasibility study and CBA during the planning phase of an IT project. Although DIRM updated the FDIC's SDLC process in March 1996, the March 1996 version required a feasibility study and CBA for major IT projects. The March 1996 version also required system developers to update CBAs when significant changes occurred in a project's cost, scope, or 1We were unable to determine the total costs related to TAPS because the FDIC did not track all costs incurred throughout the project.
7
schedule. DIRM again updated the FDIC's SDLC process in July 1997. The July 1997 version also required a feasibility study and CBA for major IT projects.
As the FDIC pursues a new direction to satisfy the FDIC's personnel processing requirements, we believe that DIRM and DOA should follow generally accepted SDLC practices and formally evaluate the feasibility and cost-benefit of alternative solutions. The FDIC's SDLC Manual requires that a feasibility study and CBA be completed before committing full life cycle resources. Other government and industry guidelines also stress the importance of feasibility studies and CBAs. For example,Evaluating Information Technology Investments, a practical guide issued jointly by the Office of Management and Budget (OMB) and GAO in November 1996, recommends that management evaluate the cost-benefits and risks of IT projects before making significant investments in those projects. We also believe that the results of DIRM’s and DOA's evaluation should be presented to the FDIC's IT Council for approval before investing significant life cycle resources or executing additional contracting actions. The FDIC’s IT Council is responsible for ensuring that strategic IT planning is performed from a corporate perspective.
Proposed changes to the FDIC's SDLC Manual would require that CBAs be updated and approved by DIRM’s Deputy Director when significant changes occur in the project's scope, estimated resources, or timeframes. While updating CBAs throughout a project's life cycle is consistent with sound business practices and guidelines, such as OBM Circular A-130, we believe that subsequent approvals of CBAs should be made at a higher level of management, such as the IT Council, when significant changes occur in a project's scope, cost, or schedule.
Key Assumption for Proceeding with TAPS Based on Inaccurate Information
One of the FDIC's key assumptions for continuing with the TAPS project was based on inaccurate information. Specifically, cost-benefit information used by senior management throughout the project assumed that implementing TAPS would correct certain internal control weaknesses that had been reported by GAO in prior year financial statement audit reports. In its audit of FDIC's 1995 financial statements, dated July 1996, GAO reported, "As in previous audits, our 1995 audits continued to identify deficiencies in adherence to required procedures in preparing time and attendance reports, separation of duties between timekeeping and data entry functions, and reconciliation of payroll reports to time cards."
During May and June 1998, when the FDIC drastically reduced the projected cost savings attributed to TAPS, senior FDIC management cited benefits for continuing TAPS development efforts. These management officials placed particular reliance on the assumption that TAPS would correct the internal control deficiencies noted earlier by GAO. However, the FDIC had taken other actions during 1996 to address GAO's internal control concerns related to the FDIC's time and attendance processes.
In its audit of the FDIC's 1996 financial statements, dated June 1997, GAO reported, "We found that the implementation of these new procedures effectively addressed the internal control issues we identified in the time and attendance reporting process in our prior year's audits." We spoke
8
with a GAO official and confirmed that the FDIC's implementation of time and attendance reporting procedures during 1996 had effectively addressed the internal control issues identified in GAO's prior year audits. The GAO representative also informed us that, as of June 4, 1998, there were no outstanding internal control issues relating to the FDIC's time and attendance processes. We advised the TAPS Steering Committee of our research and discussions with GAO on June 9, 1998. We informed the committee that TAPS was not needed to address earlier GAO concerns and that internal control weaknesses cited by GAO in prior years should not be used as a reason for continuing with TAPS development. However, members of the TAPS Steering Committee disputed the information provided and continued with TAPS development activities until the project was ultimately terminated in July 1998. Although management disagreed with the information we provided them regarding GAO’s lack of time and attendance control concerns, their current proposal to acquire an integrated corporate human resources system calls for postponing implementation of the FDIC’s time and attendance requirements.
Recommendations We recommend that the Director, Division of Information Resources Management: (1) Modify the FDIC’s SDLC process to require a formal evaluation of feasibility and cost-benefits for alternative solutions to satisfy the FDIC's system development requirements and present this information to the FDIC's IT Council for approval before committing significant life cycle resources to a particular alternative. (2) Maintain current, accurate, and complete cost-benefit information throughout the project and regularly compare this information to that which was relied upon by senior management at the outset of the project. (3) Revise the FDIC’s SDLC Manual to require project staff to advise senior management when significant deviations occur in the project's cost-benefit information, timelines for implementation, or risk and present this information to the FDIC’s IT Council for approval prior to proceeding with the project.
DIRM’S CONTRACT MANAGEMENT NEEDS IMPROVEMENT Despite management’s commitment to follow the FDIC’s structured development approach in response to recommendations in our initial TAPS audit report, the FDIC entered into several contracts for the design and development of TAPS without first completing and approving user requirements. In addition, the project development schedules and cost estimates used to obtain senior management approval of the TAPS contracts were not supported by detailed analyses or documentation. We also noted that the terms of the TAPS contracts were broad and did not require the contractor to provide deliverable products within specified timeframes. Such contracts typically require increased contractor oversight. However, DIRM's oversight of the
9
TAPS contractor was ineffective. Contractor concerns were not addressed in a timely manner, if at all, and were not regularly communicated to senior management.
The FDIC's SDLC process requires that user requirements be defined, documented, and approved before making significant investments in detailed design and development work. The FDIC's lack of adherence to prescribed SDLC procedures, coupled with DIRM’s ineffective contractor oversight, contributed to project delays, unnecessary costs, and the ultimate termination of TAPS development activities.
Contract Initiation In discussions during May and June 1997 and in our September 1997 report, we advised management that the project team had increased the risks associated with a successful completion of the project by performing design and development work before functional requirements were finalized. We reported that as much as 90 percent of the TAPS design work that had been completed as of June 1997 had to be re-performed because of changes in user requirements.
The FDIC awarded a $1.9 million contract in July 1997 for the design and development of TAPS, again without first completing and approving a functional requirements document (FRD). The FDIC's SDLC process requires that user requirements be defined, documented, and approved in an FRD before making significant investments in detailed design and development work. The risk in performing development work before requirements have been defined is that if business requirements change or do not receive management approval, the investment in the development work may not benefit the project or the Corporation. Validation of requirements early in a system's life cycle development is important because failure to validate requirements can result in frequent and expensive changes in later life cycle phases. Given the complexity of the proposed TAPS system, the project team could not have completed TAPS requirements definition and obtained approval of an FRD by July 1997, which is when the FDIC awarded a contract for the development of TAPS.
The FDIC's contractor recognized that TAPS user requirements had not been completely defined or approved when it submitted its contract proposal in July 1997. The contractor proposed that the FDIC's requirements first be validated for accuracy and completeness before initiating development work. The contractor also proposed that an evaluation be performed of the TAPS design to ensure that it correctly translated TAPS requirements into a system that would operate properly in DIRM's planned three-tier architecture. DIRM’s three-tier architecture comprises the hardware, communications, and operating software for applications processing in a client-server environment.
The FDIC’s TAPS Steering Committee approved the award of the TAPS contract without detailed information regarding how DIRM determined that TAPS development and implementation could be completed by February 1998 at a cost of $1.9 million. We spoke with the DIRM oversight manager for TAPS and learned that DIRM had not performed a detailed analysis supporting the projected costs and delivery dates. Subsequent extensions in the project
10
© 2010-2024 YouScribe