January 21, 2004
TO: Mayor Vera Katz
Commissioner Jim Francesconi
Commissioner Randy Leonard
Commissioner Dan Saltzman
Commissioner Erik Sten
Auditor Gary Blackmer
FROM: Tim Grewe, Chief Administrative Officer
CC: Mort Anoushiravani, Administrator, Bureau of Water Works
Dean Marriott, Director, Bureau of Environmental Services
Matt Lampe, Chief Technology Officer and Director, Bureau of Technology Services
RE: Ordinance Authorizing a Contract for a Replacement Customer Information System
Ordinance Authorizing a Contract for CIS Quality Assurance
The attached two Ordinances authorize contracts for replacement of the City’s current automated utility customer information, billing, accounting, reporting and collection system (CIS), and for independent Quality Assurance services during implementation of the replacement CIS.
Contract with Cayenta Canada, Inc. for a Replacement CIS
This contract is an integrated systems agreement that includes software licenses, implementation services and maintenance support. The total contract commitment amount is $4,103,513.00.
The contract identifies the functions that the CIS software to be purchased from Cayenta will perform, the hardware and technical environment in which the software will be operated, and the work required to install and configure the software in that environment. The contract also identifies the responsibilities of both the City and Cayenta in accomplishing this work, includes a schedule for completing this work, details the process for changes to the work and schedule that will inevitably arise, and identifies the payment amounts, payment milestones, and the acceptance criteria and terms under which the City will make payments to Cayenta.
The contract includes the warranties that Cayenta is providing to the City concerning the proper functioning of their software and their work in helping the City install it, and the maintenance and support arrangements once the replacement CIS has successfully been moved into production. The contract also includes agreements about the City’s options in the event that disputes arise with Cayenta that cannot be resolved through the change control process.
The City Attorney advises that the upper limit of Cayenta's liability under the contract is limited to the contract amount, backed by Cayenta as well as corporate guarantees of performance by its parent company Titan Corporation. It is expected that Titan will be purchased by Lockheed Martin Corporation and that Lockheed will assume the corporate guarantee initially provided by Titan. In the event that Lockheed is legally unable to assume liability for the corporate guarantee, Cayenta must replace the corporate guarantee with a performance bond in the same amount. The cost of obtaining such a performance bond, if required, will be split between the parities, with the City paying the premium for the bond, and Cayenta paying for the line of credit necessary to secure the bond. Whether the guarantee is secured by a corporate guarantee or a performance bond, this would permit the City to recover up to approximately $4.1 million in damages in the event Cayenta cannot or will not perform according to their contractual obligations. This provides the City with significant financial coverage until expiration of the corporate guarantee (or performance bond) 90 days after the replacement system goes “live.” Consultants hired by the City have advised that this limitation of damages is nearly universal in the software industry
The City Attorney also has advised that the majority of damage stemming from the Severn Trent System occurred after that system went "live." In the current case the City has engaged in significant due diligence activities over the past year in selecting CIS software that has been demonstrated to meet City business requirements and has created a detailed implementation strategy. Therefore, the City's main protection before going “live” is in the skillful management and administration of the contract.
Summary of Costs, Schedule and Change Control Procedures
Cost
$ .800 million Cayenta software licenses
$ .171 million 3rd party software licenses
$ .160 million 1st year maintenance on Cayenta software
$ .034 million 1st year maintenance on 3rd party software
$ 1.166 million Total Software Licenses and Maintenance
$ 2.488 million Cayenta implementation services
$ 3.654 million Total Payments exclusive of Expenses
$ .450 million Estimated implementation expenses to be reimbursed to Cayenta
$ 4.104 million Total Authorization Amount
$ .945 million Estimated City hardware costs
$ 5.451 million Estimated City staff and Consulting costs
$ 1.000 million Contingency
$11.500 million Estimated Total Implementation Costs
The contract amount includes an estimate for Cayenta expenses during implementation. Expenses are actual out of pocket costs to be reimbursed to Cayenta for reasonable and necessary items such as travel, accommodations, food, communications and the like.
The contract amount also includes the first year of normal maintenance and support for one year following live operation of the new system. Costs for the subsequent two years will be at the same rate as for the first year ($160,000 annually for Cayenta software, and approximately $34,000 for 3rd party software supplied by Cayenta). Three years after live operation, annual maintenance costs will not increase by more than 5% per year. All maintenance and support costs after the first year (included in this contract) will require separate authorization from Council. It is our intent to continue with annual maintenance, so this authorization will be back before Council about a year after the new system goes “live.”
It should be noted that the City is projecting significant annual operational reductions once the replacement CIS is in live operation. We expect to reduce approximately $400,000 in the first year and $1 million in year two as the transition between systems is completed. Once this transition is completed, we anticipate reductions of approximately $1.7 million per year thereafter. The Water Bureau has, however, been paying for these higher operational costs of the current system through reductions in other areas within the bureau, so although CIS operational costs are expected to decrease, the bureau can be expected to reprogram these CIS reductions to other operational priorities.
Schedule
Implementation start date February 1, 2004
Targeted “go live” date December 15, 2005
Estimated elapsed time 23 months
Change Control
• All changes to the scope of work, schedule or administrative details will be tracked by the project management team and reported at least quarterly to the CAO and CIS Executive Committee
• Any increase in the contract amount requires CAO approval, but only after discussion with the CIS Executive Committee and approval of the Mayor and the Commissioner-in-Charge of the Water Bureau and BES.
• If the City elects to complete additional “cycle simulation” testing, the CAO is authorized to amend the contract amount by up to the $160,000 estimate for that purpose.
• Any increase in the contract amount in excess of 10% of the initial contract amount (approximately $410,000) requires formal Council approval.
• Any change in schedule that increases the projected “go live” date by more than 90 days requires Council approval.
• Any other change in schedule, or any other contract amendment, requires CAO approval, but only after discussion with the CIS Executive Committee.
Total implementation costs are well within TMG Consulting’s original estimate of $6 - $16 million. However, costs or the schedule may change during then course of the implementation effort, but our commitment is to be clear in communicating why any change is required.
Due Diligence - Why we believe Cayenta can supply a replacement CIS that will meet the City’s business requirements
The first critical factor in lowering risk in replacing the City’s billing system is to obtain proven software that matches the City’s business requirements with as little modification as possible. In order to assess software options, it was therefore essential to carefully articulate City business requirements, which we did in a Request for Proposals (RFP) issue in December 2002. We received 12 responses to the RFP.
The City assembled a core group of 12 City employees, plus 3 outside advisors, as a Selection Committee to evaluate CIS options. Additional City staff participated throughout the evaluation process as required in specific areas of their expertise. City employees represented the Water Bureau, BES, the Auditor’s Office and OMF, and a variety of disciplines within those organizations, including:
• Financial management and auditing
• Customer account and property management
• Meter management
• Rates and billing management
• Bill printing
• Cashiering and payment processing
• Payment extensions, collections and debt recovery
• Service orders
• Reports
• Interfaces to other City systems (such as core financial system and meter-reading devices)
• Technical system environment
• Training
Through an increasingly detailed evaluation of proposals, the Selection Committee first assessed whether proposals met 44 minimum requirements. Five firms met the minimums, and were further evaluated based on their responses to 973 detailed business requirements specified in the RFP. Four vendors were selected to participate in interviews and product demonstrations.
Since the City’s current CIS has been problematic from an accounting and audit standpoint, as part of the demonstrations we asked vendors a separate series of detailed financial and auditing questions, including how they handle 14 difficult accounting scenarios that we typically encounter. We also discussed their technical environment, underlying software design strategies and future product directions in separate “breakout” sessions.
By the end of the product demonstrations and related financial and technical discussions, the Selection Committee determined that two vendors could meet City business requirements without significant modifications to their current base product, or that in some cases the City could adopt business practices that matched the products without difficulty. In the case of Cayenta, seven minor modifications appear to be necessary and Cayenta will make these in their base product. This means that the City will be using Cayenta's updated and now standard software without modifications, thereby eliminating concerns about upgrading to future releases of their software and “loosing” those modifications, or having to maintain them ourselves.
This is an important difference in approach from the City’s experience with OV. With OV, the City agreed to work with STS to custom develop important aspects of the system. With Cayenta we will be purchasing software that will meet City business requirements without any significant modifications.
In addition to assessing basic business requirement functionality and technical currency of the Cayenta solution, the Selection Committee reviewed implementation strategies, project management capabilities, commitment to diversity, the vendor’s financial health and experience, and cost. Cost considerations included the risk that initial costs could escalate, or the schedule for implementation could extend due to the implementation or project management strategies proposed.
The importance of this thorough evaluation work, which consumed four months of essentially full-time effort for the Selection Committee, was to verify what vendors told us about their software in their responses to the RFP by seeing actual demonstrations of that functionality.
As a result of these extensive evaluation activities the Selection Committee unanimously recommended Cayenta as the lead vendor option for the City. This recommendation was based on the fact that Cayenta can meet City business and technical requirements without any major modifications, proposed an implementation and project management strategy that will minimize cost and schedule overruns, is a financially healthy vendor that is actively participating in the marketplace and committed to evolving their product over time, has a commitment to diversity, and because Cayenta’s initial bid was substantially lower (by nearly $5 million) than the other vendor also capable of meeting City business requirements.
The CIS Executive Committee, led by the CAO, and including the Water Bureau Administrator, BES Director and the Chief Technology Officer, reviewed and discussed the Selection Committee’s recommendation. The CIS Executive Committee asked that additional investigation be conducted in the following areas:
1. Could the cost of the higher priced proposal be reduced?
2. Further investigate accounting and other functional capabilities
3. Further investigate Cayenta’s financial health and corporate structure, including its parent company
4. Further investigate issues with Cayenta customers who were unhappy with their choice
5. Further investigate aspects of Cayenta’s software development strategy
The Selection Committee spent an additional two months further investigating these issues. Cayenta’s financial health, and that of Titan (their parent company) was discussed under a non-disclosure agreement with officers from both companies. Additional discussions were held with Cayenta customers (including review of documents from the dispute where the customer appears to be the most unhappy with their Cayenta experience). Further discussions were also held concerning Cayenta’s software development architecture and strategy.
In the end the Selection Committee again concluded that Cayenta should remain the first choice of a CIS solution that the City should pursue. This recommendation, after substantial further discussion by the CIS Executive Committee, was accepted. Members of the Selection Committee briefed all members of the Council and we filed a Report that the Council accepted on August 13, 2003, concurring that contract discussions with Cayenta should proceed.
Implementation Strategy - Why we believe the strategy for installing the Cayenta solution has a very high probability of success.
In addition to identifying basic CIS software that will meet City business requirements, and concluding an extensive due diligence investigation to verify a very high probability that the software will work as represented, the second critical factor in lowering City risks is to prepare a detailed Statement of Work (SOW). The SOW included in the contract identifies the City business requirements that the Cayenta solution will address, the technical environment in which the software will operate, the work required to modify and configure the software and related database to operate in accordance with those requirements, and the responsibilities of both Cayenta and the City in accomplishing this work.
The strategy for implementation is to establish operation of minimum City business requirements as defined in the RFP with the Cayenta software. These minimum business requirements are limited in scope, and do not include functions beyond those generally now available in the current CIS with only one exception, the storm water discount program, which the Cayenta system will be configured to support when it goes “live.” However, other desirable features such as monthly residential billing, customer access to account information via the internet, or on-line payment options may not be available at the time the replacement system goes “live.” Some expanded features may be available if they do not require additional effort to configure, but they are not the primary focus of the implementation effort, and will need to be addressed as separate activities once we have accomplished the basic work of replacing the current billing system.
It should be noted that although the current CIS is relatively stable and is accomplishing basic billing tasks, it remains at risk of failure. The City’s primary risk mitigation strategy for the current system is to replace it as quickly as reasonable, and any “scope creep” in doing so beyond accomplishing the minimum business requirements has the potential to elongate the implementation schedule, and therefore extend the risk period during which we must continue to operate the current CIS
Throughout the implementation of the replacement CIS solution there are three principals we will follow. First, because of the critical nature of utility billing for the City, we will not allow CIS replacement implementation activities to inappropriately disrupt operation of the current CIS. Since many of the staff involved with operating the current system must also be involved with the replacement effort, the implementation team has been selected to minimize impacts to existing operations, while ensuring we have talented people to participate in the implementation effort. These arrangements will inevitably involve some sharing of resources, and the implementation project mangers and Water Bureau management understand and have agreed to this strategy.
The second principal we are following is to test software configurations and modifications at each and every step. We will not proceed to the next step until testing confirms that we are ready to do so. We will test the Cayenta product “out of the box” by configuring basic features and confirming that the system operates correctly with test data, and as we move through the various stages we will test individual functions, and ultimately all of the functions as they work together, first using test data, and later using actual data converted from our existing CIS databases.
The third principal we are following is to use independent Quality Assurance (QA) services to double-check the accuracy and completeness of all implementation work. QA services have been selected and an Ordinance authorizing a contract for them will also be on the Council agenda at the same time as the Cayenta contract. QA services will include review of processes, testing results, and all activities of the implementation team. The QA consultant will report findings to the CAO on an agreed-upon schedule (probably monthly or bi-monthly) throughout the implementation effort. Although normal project management strategies will routinely identify risks, issues and concerns and manage or resolve them, independent QA services will focus on items that may have escaped the project team. QA reports will therefore assist the CAO and the entire project management team in ensuring that no issues are left unattended.
Prior to moving the replacement system into production a second level of independent QA review will be conducted by the City’s outside financial auditor, managed by the Auditors Office. This review will focus on the integrity of financial transactions flowing through the replacement CIS, including internal audit controls and the ability to produce reports necessary for preparing and auditing financial statements.
The basic implementation strategy will be as follows. Date estimates for all phases of the project can be expected to change as the work progresses. All changes (including schedule) will be managed through a formal change control process:
Phase 1 - Project Initiation (Estimated for February 2004 to April 2004)
This phase is the initial period when the implementation project’s scope, objectives, assumptions, timeline, governance structure and sponsorship are formalized and approved in a Project Charter. A more detailed project plan will be completed, with staffing assignments, roles and responsibilities confirmed. Cayenta will deliver their basic software and documentation, and assist with installing it in the City’s hardware environment. Critical implementation issues will be detailed, quality and risk management plans developed, the criteria for readiness to move the replacement system into production agreed to, and change control procedures put in place. The implementation team will also receive basic training in the Cayenta product’s underlying logic and design strategies, and its user operation.
Phase 2 - Fit Analysis and Configuration (Estimated for April 2004 to August 2004)
During this phase specific information is gathered so that the Cayenta software can be initially configured to operate correctly “out of the box” with test data. This work will reveal any “gaps,” at a very detailed level, between the basic Cayenta product and City business requirements, and will identify and confirm interfaces to other City systems, City business practices that will need to adjust, and/or additional customization that is required. Detailed plans for end user training will also be developed during this phase.
Phase 3 - Deployment (Estimated for February 2004 to January 2005)
Based on the work in the previous phase, functional specifications, written in non-technical language, will be prepared that describes the nature and scope of each interface and modification. Once approved, the functional specifications will be used to prepare detailed programming and configuration specifications, and end-user training outlines will be updated as appropriate. Individual programs and routines will be tested to confirm they are operating in conformance with approved specifications.
Phase 4 - Functional Testing (Estimated for July 2004 to January 2005)
This phase will ensure that all components are tested and working properly, including billing cycle simulations. Testing activities will be repeated until expected results are achieved. Testing scenarios and data will be evolved as required to address problems and issues. Testing results will be recorded in formal testing documentation and a final functional testing report will be prepared. Training materials to be delivered to end-users will be created, and job roles that will change will be identified, along with transition plans for staff that will have new duties once the replacement system is in production.
Phase 5 - Integrated System Testing (Estimated for January 2005 to September 2005)
This phase includes end-to-end testing of all system functionality as an integrated system, including operation of automated interfaces to the City’s core financial system, meter-reading devices, bill printing routines and the like. Full billing cycle and collection process simulation testing will be completed. Later stages of testing will use data converted from the City’s existing CIS databases. This will include a series of mock data conversion runs that will provide a full set of production data to the test environment.
Phase 6 - City Acceptance Testing (Estimated for August 2005 to October 2005)
During this phase the project team will conduct another series of independent tests to determine if the replacement CIS is ready to be placed into production. Testing scenarios will be expanded and will utilize converted production data and live data from interfacing systems to simulate actual operations. Independent quality assurance personnel will thoroughly review testing methodologies and results, confirm the financial integrity of transactions, and verify that appropriate audit trails and financial reporting are working properly. A formal report and recommendation on a decision to move the new system into production will be prepared for approval by City management and the Council.
Phase 7 - Production Readiness and Cut Over (Estimated for September 2005 to December 2005)
Deploying the replacement CIS as a production system requires an organized and well-communicated strategy. Given the experience with deploying the current system, the City may require even further testing to acquire additional knowledge of the system’s performance in a simulated production environment. If that is required, this additional testing will be completed during this phase. Final end-user and technical support training will be conducted to ensure that City staff are fully prepared to use the system and its supporting business processes, and to run production operations. The final step will be to actually move the replacement CIS into production.
Phase 8 - Post-implementation Support (Estimated for December 2005 to June 2006)
Once the replacement system is successfully in production, Cayenta will support City staff in operating the system. Initially after “go live” Cayenta will retain staff on-site at the City in addition to their normal off-site support group. During this phase, system operations will fully transition to City staff, and Cayenta will prepare a report outlining their assessment of the City’s ability to operate the system on our own.
Data Conversion
Data conversion is a crucial part of implementation but cannot a separated out as its own “phase” since it is activity that has already begun (during procurement of a new system), and will not be completed until the end of final production cutover in Phase 7. The accuracy of existing data in the current CIS databases is being verified now, in preparation for conversion activities. Data conversion is iterative, in that multiple efforts will be required, first to map existing data to Cayenta’s data structure, and then to execute conversion for portions, and then for all, data. Testing of data conversion results at every stage will allow us to correct errors and test conversion routines, while supporting various stages of functional testing, and finally, of fully integrated testing with a complete data set.
Key Protections for the City - Why we believe the City will be well protected in the event of any dispute with Cayenta.
The third critical factor in reducing risk in replacing the City’s CIS is to ensure that if the software does not function as expected, or cannot be installed for the price or within the schedule anticipated, that the City has adequate protection for costs incurred and to finish the job through other means. As the City Attorney has noted, however, our primary protection is in the extensive due diligence we have conducted to ensure that we are buying software that is already in use by other customers, and demonstrations of the software’s capabilities (including accounting requirements) without major modification. We obviously do not anticipate problems, but have included a number protections in the event the unexpected occurs. City consultants have stated that many of these protections exceed “industry norms.”
Source code will be available to City in escrow if Cayenta defaults on obligations
Cayenta delays (beyond those authorized through the Change Control process)
o All payments are tied to specific deliverable products
o Cayenta delay also means a delay in City payment
o Delays that cause the “Go Live” date to extend 90 days or more allows City to collect $2,500 per day back to the first day of delay, up to 10% of Contract Amount ($410,000)
o This will be incentive for Cayenta to stay on schedule with their work
Corporate Guarantee from Cayenta’s parent company (Titan)
o Guarantee is available to hire others to do the job if Cayenta cannot
o Guarantee will be equal to the contract amount ($4.1 million)
o Guarantee expires 90 days after “Go Live”
o If Titan sells to Lockheed, Lockheed will assume Guarantee
o If Lockheed legally unable to do so, Cayenta will replace Guarantee with Performance Bond
o City will pay premium for Performance Bond
o Cayenta will pay for Line of Credit for Performance Bond
o It is highly unlikely that Lockheed will be legally unable to assume Guarantee
o Access to Guarantee or Performance Bond will likely require litigation
Payments are structured to match realized value for City during implementation
o See attached chart
o Just prior to “Go Live” City holds over $1 million in payments still due on contract
o After “Go Live” payment, City will hold $400,000 in payments still due on contract
o “Acceptance” is defined as billing errors less than 2%, 90 days after “Go Live”
o After Acceptance City will hold $100,000 still due on contract
o Final City payment will be $100,000 at 180 days after “Go Live”
On-site support will be provided by Cayenta for 6 months after “Go Live”
Warranty will continue for 1 year after “Go Live”
Consequential damages (including lost revenue)
o Cayenta will pay up to 5% of Contract Amount ($200,000) through Warranty period
o If system fails after “Go Live” City will not be covered for substantial lost revenue
o This points to the importance of due diligence in selecting system
o Requires thorough and careful implementation plan
o Requires thorough testing at each and every step before “Go Live”
o Requires skillful management and administration of contract
Termination for convenience or for cause
o Termination for convenience increases City flexibility
o Termination for convenience will require City to pay all license fees
Contract with Pacific Consulting, Inc. for CIS replacement Quality Assurance Services
This contract is in a total amount of $268,800.00. Pacific Consulting will provide ongoing independent written assessments of the process, interim and final products delivered and completed by both Cayenta and City staff, and progress as the City replaces its current CIS.
Pacific Consulting was the unanimous choice of the selection committee following an open RFP process and interviews with four finalists. Pacific is owned by former Deloitte Consulting partners who have experience successfully completing other, similar QA engagements, including for the City of Seattle’s recent CIS project.
Pacific Consulting will prepare monthly written reports on all aspects of the CIS implementation effort being completed by Cayenta and City staff. QA reports will cover topics including risk and issue management, project organization, work product and activity quality, and cost, staffing and schedule management. Written reports will be made directly to the CAO, who will share them with the CIS Executive Committee, Mayor and Commissioner in charge of Water and BES. Drafts of monthly reports will be shared with Cayenta and City project managers and discussed as necessary with the CAO and the CIS Executive Committee before they are finalized to ensure that Pacific Consulting is aware of their perspectives. However, such sharing of draft reports does not require Pacific Consulting to alter or adjust any statement they wish to make concerning the CIS replacement project, preserving their total independence.
Once the Cayenta and City project team is satisfied that the replacement CIS is ready for live operation, the City’s outside auditor will verify the financial integrity of transactions and that appropriate audit reports can be generated. Pacific Consulting will provide a final written assessment of City readiness to move the replacement CIS into production. This final assessment will include a review of the status of all necessary business practice changes, interfaces with other City systems (including the IBIS financial system), the readiness of the technical environment and its staffing levels, skills and training, and the status of transition plans for project staff and staff supporting the current OV system.
Once the Cayenta system is in live operation, Pacific Consulting will also prepare a summary of the implementation project’s successes and failures, and make recommendations the City should consider to improve implementation of other large, complex business systems.
Conclusion
Preparing to replace the City’s CIS has been a long and difficult process for the City, and for the staff and consultants engaged in this work. Water Bureau staff in particular have had to work through the dual chores of stabilizing the current system while analyzing options for its replacement. Working as a close, cross-organizational team led by OMF and involving the Water Bureau, BES, the Auditor’s Office and the City Attorney’s Office has been challenging at times, but ultimately the results have been excellent.
Implementing the replacement CIS over the next 2 years will also be difficult. Issues will arise, there will be setbacks, and there will be decisions we will need your direction on. But I am confident that the team we have in place will manage through it all successfully, and that 2 years from now we will have a replacement CIS that works as expected, and that we can all be proud of as we move forward.
I have attached the most recent version of Frequently Asked Questions about the City’s CIS replacement effort. If you have any other questions, we would be pleased to discuss them with you.
Thank you.