Commercial Off The Shelf And Its Validation Information Technology Essay

Abstract:

Now a day, Pharmaceutical Company is no longer to expand their resources for the development of computer software. Instead of they are buying the off-the shelf computer software which fulfils all kind of business requirements at very low cost. This paper mainly describes about the commercial off the Shelf software (COTS) and methods to evaluate the COTS products. It includes the regulatory requirements for the COTS system. As all software needs to be validated, COTS also need to be validated for its intended use. This paper described about the validation approach for the COTS system and Principles for validating COTS system. As, software life cycle model is very important for the step wise validation process for the commercial off the shelf software. This paper also described the validation process of COTS by using the general life cycle model.

Introduction:

Three types of the software products were there in the computers and automated system. There software was used for acquisition, processing, recording, reporting, storage and retrieval of the data.

COTS- Commercial off the shelf

MOTS- modified off the shelf

CUSTOM

These softwares can exist in local hard drives, network hard drives, embedded on integrated circuits or removable disks.

MOTS: The software is altered for the specific application for example Lab windows, Lab Tech Notebook, generic data acquisition software, etc. The part which is purchased is considered as COTS and the part which is modified is considered as CUSTOM. Testing should be performed on each part which is modified and it gives the evidence that each piece performs it works as per designed and provides satisfied each function.

CUSTOM: This software is lab written for example, C++, SQL Database design, etc. They should be gives the information about requirements, design, construction, testing and installation [1].

COTS: The software is purchased without any modification for example MS word, MS excel etc.

COTS Overview:

From last decade, the software world developed rapidly. Many changes were come in the business software field. Now, companies are no longer to expand their internal resources to develop software. Instead of that they used the software which fulfils all kind of business requirement which is delivered as “off the shelf”. COTS (Commercial off the shelf) software is used in the pharmaceutical industries and forensic laboratories. COTS refer to computer software or hardware system and it also includes free software with commercial support. This technology is ready made technology and it is available for lease, sale or license to general public. It is an alternative of in house developments or one off government funded developments. Many types of COTS software are there like resource plans applications, customer relationship management tools, quality system databases for CAPA, complaint handling, auditing, document control etc. The COTS software will be used at lower risk and it is develop without safety consideration of the software. COTS allow the usage of any software application for the specific work and it will not alter the basic program. So, it can be used by the end user without any kind of alteration and adaption.

The use of COTS system becomes mandatory in some government and business programs because due to COTS, some of the products give significant savings in procurement and maintenances. COTS software specifications are written externally so many government companies have such a fear about the future changes to the product because it is not in their under control. Usages of COTS components will reduce overall system development costs and maintenance costs [2].

Characteristics of the COTS systems:

The buyer has no contract with source code

Vendor controls the system development

The software has a nontrivial installed base [3]

Software is readily configurable to meet specific intended functions

It is ready available and easily adaptable by user

Overall cost of the system is low

Evaluation of COTS Products:

To develop COTS system, candidate Cots components will be evaluate early in the development process. At early stage, requirements are less defined and it provides only general idea to the evaluator. Some of the COTS evaluation methods proved less useful because it is based on traditional development paradigms. These methods are depending on structure requirement and specification criteria of COTS. They are not able to give the fast changing response as marketplaces changes. Other methods are depending on the prequalification of COTS components. For this, developers have to choose such components, which is highly qualified or certified. The developer depends on the in context evaluation to determine specific knowledge about each and every candidate COTS software product. The method, which is used to evaluate the COTS, is developed concurrently with the requirement process.

There are number of methods which are used to evaluate the COTS components. Each of the method points out one or more critical aspect of COTS software.

COTS based Integrated System Development (CISD) method:

There are two stages for COTS selection: Identification and Evaluation.

Identification: In this stage, candidates are identified and classified. The data are collected by vendor documentation, personal experience or by any another way. The result contains potential candidates.

Evaluation: In this stage, final candidates are selected and unsuitable candidates are eliminated. In this stage they use two techniques one of which is concrete and other prototype technique, which is widely used and it is the only way to recognize a COTS candidate within the systems context.

There are three critical stages in prototype technique.

Functionality: Candidates are tested in remote manner for the confirmation of functionality of COTS product and that product is valid for current application.

Interoperability: the candidates are evaluated for the assurance that they are able to synchronize with other components of the system.

Performance: In this, qualitative analysis of the system is performed to see the effect of the COTS system on the overall performance of the system.

Management evaluation is the final aspect of this method. Training, cost, vendor capability etc are included in this evaluation. At the end of this aspect, final selection of COTS is take place. There are two other approaches which are based on development time and cost. Comprehensive evaluation, in which there is a best COTS product set. First fit evaluation, in which there is a first product sets which completes all the requirements. It is a cost effective approach.

This method is relying on complete predefined set of the requirements.

Off-The -Shelf-Option(OTSO)

This method is concerned with the actual selection process. Three phases are there:

Search Phase: In this phase, COTS candidates are recognized. During this phase, requirements are not specified.

Screening and Evaluation: In this phase, they narrow down the field of potential candidates.

These two phases provide the additional information to understand of the product capabilities which gives the feedback to the requirements definition process. This cause improvement or alteration of known requirements and also provides some new requirements. Evaluation is always performed on the bases of known set of evaluation criteria. These new criteria come from different sources which includes requirement specification, high level design specification, the project plan etc.

Analysis phase: This is a last phase of the selection process and which leads to the final selection of the COTS product.

This process is repetitive process because the requirement is improved and defined through the course of the evaluation phases.

Checklist Driven Software Evaluation Methodology(CDSEM)

This method is used to evaluate the employs checklists. In this, they use quality metric for the checklist. This process is based on the metric so this method provides the numeric result so it is suitable for the component. This method is good for use because it qualifies the evaluation result. But this depends on the vendor documentation which causes the adoption of inappropriate candidates.

Procurement- Oriented Requirements Engineering(PORE)

In this method, requirements are defined parallel to the COTS component evaluation and selection. This method used the template approach which relies on evaluating COTS product. The template gives primary steps which are required to perform evaluation of the candidate COTS application [4].

Regulatory Requirement:

Validation of Computerized system is used for the pharmaceutical activities for creating, managing and reporting data. This system is requisite by the US Code of Federal Regulations, Title 21, Part 210 and 211 also 21 CFR part 11. The main problem in the software validation requirements is COTS applications. Government says that it is end user’s responsibility to validate the software before it is used in the production work in the FDA regulated environment.

There are two main references for computerized system validation:

PDA Technical Report No. 10

Validation of computer related systems and GAMP (Good Automated Manufacturing Practice) for validation of automated system.

These two references come from the steps in the life cycle validation approach. Vendor finishes the development, design and test requirements for the system. End user’s responsibility is to check that the system, which is made by the vendor, which provides all the application according to the procedure, which is already defined [5].

Advantages and Disadvantages of COTS System:

Advantages:

Decreased development effort: COTS software reduces the effort of development and test code and it also reduces the risk. It increases the productivity and also increases the quality. One can update and reused the COTS software without making new whole software. One can save the time because the system is already being tested prior to use.

Faster Procurement Process: This process is less formal and competitive then other procurement process. It will save the time during purchase because many reviews are not primary. COTS software provides higher quality, lower cost, faster acquisition time and provides flexibility in maintenance.

Increased Reusability: COTS software will provide such facility that one can use many projects and by sharing it gives higher reuse of the software which increases productivity and reliability.

The components are less costly than any other in house system.

Disadvantages:

Increased Configuration Control Problem: It is difficult to segregate all the applications which are needed to provide the required functionality.

Obsolete COTS software: Software development time is 12-18 months and obsolete time is 36-48 months. There may be problem in the software because when the software is in used in field, it may obsolete.

Inability to meet requirement: Sometimes the purchase of DoD can make to purchase of COTS software which is not able to meet the system’s requirements [11].

COTS Validation Approach:

Validating of COTS system is for the understanding of the systems intended use. Software must be validated if one has to achieve the compliance with rules. If the system is used as a part of production or quality system, the system must be validated by the manufacturer as per the protocol. Manufacturer must be aware that the system will perform its work as intended in their chosen application. If the software is not used for compliance with FDA regulation, validation may not be required.

Two elements are needed to be addressed during validation: One is systems actual compliance with applicable rules, standards and general guidelines and other is its formal validation.

Software vendor must give the proof that the system’s application is compliant with FDA regulations and other standards. It is company’s responsibility to verify that the system works as per the compliance. Generally, verification is done before moving the system to production area for use. System should be complies with 21 CFR Part 11 requirements. It is meaningless if the system is not properly function as per intended use and it was moved to the production use so validation is an established documented that the system works as per intended use and must be adhere with the compliance.

The basic steps for validating COTS software are as follows:

Define the use of the system in company.

Define validation deliverables depend on system types. System risk, project scope and degree of the system modification.

Review already used vendor system and its validation documentation.

Determine strategy of validation as per system’s application.

Make applicable system requirements specification and design documentation.

Create requirements traceable validation protocol.

Determine system use, administration and maintenance procedure to ensure that system works as per intended used and remains in validated state.

If the requirements are different than those documented by vendor than company have to make a separate requirements document and then validation testing will cover all these requirements [5].

Commercial software is used in electronic recordkeeping system which have to be validated same like end user have to validate the written program. The bend user is responsible for the program suitability when it is used in the regulatory environment. End user’s validation approach is different from the developers because the source code and development documentation is not available with end user. End user validates that program to the macro level.

End user validate off the shelf software by doing following activities:

End User Requirements Specification: End user documented their requirements as per part 11 requirements and there requirement specifications are different from developer’s requirement specifications. They made a copy of the developer’s requirement specifications for the comparison.

Software Structural Integrity: End user’s don’t have source code so they do the software structural integrity by doing following activities:

They do the research on the program history. It includes program’s limitation, other end user’s experience, define software problems and their solutions.

End user’s evaluate supplier’s software development activities by defining its conformance to contemporary standards.

Functional Testing of Software: End user will perform functional testing which will covers all the functions of the program, which is used by end users. End user’s don’t have source code and development documentation and having limited experience with program, so they get warrant from the functional testing. Functional testing is not sufficient for defining software’s ability [7].

Principles of Validating COTS System:

There are some of the general principles those we have to consider during the validation of the commercial off the shelf software.

Requirements:

Software requirements specification is needed for the validation process. Documented requirements for COTS provides baseline for the validation and verification.

Defect prevention:

Quality assurance needs to focus on the prevention of the software defect that may do occur. Only testing for software is not sufficient for the conformation of the defect free software. Software testing is necessary but it is not sufficient to establish the confidence that it is feet for the intended purpose. We need to develop the mixture of method for the conformity of the error free software. This mixture of method is depending on the application, language, development environment, size of project and also including risk.

Time and effort:

For the complete validation of the COTS software takes time and much effort as we have to start the validation during development phase of the software. And final conclusion of the software is based on the evidence collected from the planned effort during software life cycle.

Software life cycle:

Software life cycle contains mainly about the software engineering task and documentation for the support of the validation process and also it describes the validation and verification task for the intended purpose of the software.

Plans:

Plan is necessary for the any type of validation process. COTS validation plan is quality control tool and it defines “what” we achieve through software validation effort. Plan specifies the scope and approach, schedules, tasks, types of activities etc.

Procedures:

Properly written procedure is necessary for the software validation. It describes “how” to conduct the validation process and it describes the specific actions and also sequence of the action to be taken during the COTS validation process.

COTS validation after a change:

Whenever we do change in the software, software re-validation is necessary for the conformance of the validated state. Validation analysis should be performed not only for individual changes but to determine the impact of the changes on the whole software. Design control and re testing gives assurance of the validation of the software even after changes made in it.

Validation coverage:

Validation coverage should be based on the risk and complexity associated with the use of the COTS not on the size of the software. For low risk COTS software, only base line validation task may perform for the validation process. While, for high risk devices, additional validation task should be added for the conformity of the software.

Independence of review:

Independent review is necessary for the higher risk application. Some firm gives contract to outside for the verification and validation of the COTS software for independent review. But this approach is not always feasible. Another way to independent review is to assaying a team for the internal staff for the review process who has sufficient knowledge and skill for the evaluation of the COTS software. Still, small firm needs to be creative in organizing and assigning validation task for the independence review.

Flexibility and Responsibility:

Devise manufacturer has the flexibility in choosing the application of validation principles. Software validation activities and tasks are dispersed as develop in different environment and conducted by different organization. Despite of this flexibility, Ultimate responsibility for the validated software is on the device manufacturer and specification developer [8].

Validation Process of COTS System:

By using the general life cycle model, organize the validation process of COTS. By this way software are subject to validation in all phases of its lifetime. This life cycle organize the validation schedule with necessary information for proper assessment. This gives basic knowledge of the tasks to be performed, method, criteria and input and output required for each tasks. Also it will outline the required documentation and the person who is responsible for the validation process.

This life cycle model includes the following phases.

Requirements and system acceptance test specification

Design and implementation process

Inspection and Testing

Precautions

Installation and System acceptance test

Performance, Servicing, Maintenance and Phase out

Requirements and system acceptance test specification:

The requirement describes the identity, analysis and documentation of information about the requirements and validation process. The requirements should state clearly about the intended use of the software. It is not possible to validate the COTS software without the predetermined and documented requirements.

Requirements specification:

The requirements should include everything about the use of software.

Information that identifies actual version identification

Software inputs

Software outputs

All function that the software will perform.

Traceability, limitation, hardware control

Required response time

Intended operating environment for software. For example, Hardware platform, operating system.

Safety related requirements, features and functions that will implemented in the software.

System acceptance test specification:

System acceptance test includes the criteria for the COTS system that how the COTS software will be tested to fulfil the acceptance criteria and satisfy the requirements and also it performs as required in the environment in which it will be used. This test is performs after the proper installation of the COTS software and it will approve the system for the use.

Design and implementation process:

In the design process, the requirements are translated into logical and physical representation of the software to be implemented. The design phase may be more complex and comprehensive because of the complexity of the software and number of persons involved and if there are special requirements etc. Design phase can be divided into several parts as each one is focusing on the development activities and tasks.

Design and development Planning:

Depending on the complexity and scheduled of the validation process, we have to make detailed development plan, prepare it and must approve it. It is a detailed plan that which part of the software should be evaluated and validated and which criteria should be used.

Design Input:

This phase establishes that the requirements can be implemented. This phase translates the design requirements into description concerning the implementation of the software. Resulting description should be documented and verified if needed.

Design output:

Design output includes the some activities such as, architectural design specification, detailed design specification, source code and user guides. Design output must meet the design input requirements as it identify the system characteristic that is beneficial for the safe and proper functioning of the software. Design output must be validated before release of the product for final testing.

Design verification:

Verify the design by the formal documented review before proceeding to the development process. The main purpose of this verification is to ensure that the validation process is going as planned.

Design Changes:

Design changes and implementation should be identified, documented, reviewed and approved before changes are made to the COTS software. Design changes request may arise at any time during software life cycle. Following task should be considered while dealing with software changes.

Documentation and justification of the changes.

Evaluation of the consequences of the changes on the software

Approve the changes

Implement and verify the changes

If we do minor changes in software and it has no impact on other modules then system do not require revalidation. And degree of revalidation is evaluated by the degree of changes and impact of changes on the system.

Inspection and Testing:

Inspection and testing of the COTS are well planned and also must be documented. The extent of the testing is depending on the complexity, risk, acceptance specification and intended use of the software.

The following elements are examined by the inspection.

Design output: Inspect the coding structure, documentation and compliance with the rules.

Documentation: Inspect of program documentation, user manuals, test results etc.

Software development environment: Inspect the file storage, source code protection. This inspection should include the installation kits.

Test plan explains about what to do, how to do testing and what to expect. Also it considers the past data that what was done and how was the result. Test plan should consider the tests objective, Scope of tests, level, type and sequence of the tests, Configuration and calculation of the tests, Acceptance criteria, and follow-up of tests and also result of testing. Test plan should be created during the development phase.

Precautions:

As COTS is a third party software, so it may creates some undesirable, inappropriate conditions. These conditions may impact on the software product cause malfunction; they must be identified, documented and if possible avoided. Also tests those done around these conditions must be verified and validated. Also precautions should be taken when description of the system that it supposed to work and also the way it actually dose is different. It is a best idea to maintain the logbook of registered undesirable conditions for the different operator and programmer to use.

Installation and system acceptance test:

This phase of validation process includes the tests perform on the COTS software and it is also documented. This documented evidence provides conformance that the operation and performance of this software is consistently meets the predetermined acceptance criteria. This testing generally includes various validation protocols such as, IQ, OQ and PQ. These protocols must be created during the inspection and testing phase. These protocols contain the tests to be performed, instruction for documenting test results and investing and resolving tests deviations.

System validation IQ/OQ testing:

For COTS system, generally vendor provides IQ and OQ test scripts and necessary documentation and will perform testing. If vendor will not provide the test scripts then system owner or designee have to establish and execute the process.

IQ testing provides assurance that the system properly installed in suitable environment. IQ protocol includes the acceptance criteria and generally executed by the COTS vendor’s engineer.

OQ testing provides assurance that the system functioning properly as per operational specification and in appropriate environment. OQ protocol also includes the acceptance criteria and it is also executed by the vendor’s engineer.

System Validation PQ testing:

PQ testing provides documented evidence that the integrated system performs as intended in the intended environment under actual condition of use. PQ testing should include the exercises the major system functions and full cycle of the system operation, exercising the system over full range of normal operational inputs.

Performance, Servicing, Maintenance and Phase out

In this phase, the software is in working condition and requirements for service, maintenance. This requirement is necessary when decision making related to changes, revalidation and phase out are made.

9.7 Validation Summary Report:

This report includes the summary of all validation activities. It includes summary of all tests results, analyses and conclusion. It also includes deviations those were found during inspection and also includes corresponding corrective and preventive actions those were taken.

Generally validation report includes the following items:

Hardware and software installation

Person who doing validation process and also date when tests were performed and also corresponding documents

Results of the IQ, OQ and PQ test

Summary of any deviation found during testing and also corrective actions

Finally report is clearly mentioned that the system is validated or not. And if not then what type of changes and corrective action needed to get validated state [9].

Case Study:

Here there is a one case study about “Applied Biosystems” that they are doing validation of Analyst software for the pharmaceutical company. Main purpose of publishing this article is to help the user in validating the Analyst software. They assist users in validation process and what point they have to consider during validation of analyst software. This article is mainly focusing on the need for validation of any software. It forces to software or business owner to validate their software as it is very lengthy and costly process but ultimately it is beneficial for the business. Also validation is a required activity for the reliable, secure and fit for the intended use of the software as per regulation.

It describes the validation process that validation is done as per software life cycle model.

First phase is requirements and system acceptance test specification that it ensures the technical control of hardware and software. It reduces the human effort through automation and reduces the incidence of human error. Also it controls the procedural control that ensures that each employee has unique identification. Also they check the SOPs of company as it is a part of procedural control. SOPs includes training procedures, change control procedures, documentation maintenance procedures etc. It is worthwhile for the company to document all the SOPs.

Second phase is design and implementation process. During validation they did change control procedures for the error found. This is very important that identify the new requirements, error found, or procedures revised and they change the validated system and also they documented it and approved by the Quality assurance and revalidate it for the compliance with regulation. Also they verify the user requirements for the design input and output.

Third phase is inspection and testing. They made validation plan that covered the strategic validation effort, individuals responsible for the validation activities, schedules, execution and testing and also approval. Also they specify the configuration such as equipment setup, security and data processing and document them for the assurance of the system that matches the actual configuration that intended.

Fourth phase is precautions. They did the precaution step and documented the undesirable and inappropriate conditions and validated them.

Fifth phase is installation and system acceptance test. They did all the four phase of qualification such as DQ, IQ, OQ and PQ. DQ was performed by the Applied Biosystems and also IQ and OQ was performed at on-site by a qualified Applied Biosystems’ field engineer. Then they did PQ test. They made test script for the task to be formed for the PQ test. This test includes both positive and negative test and also includes ranges of functionality. Also they documented the deviations what they found and also documented the corrective actions and verified them. They were not only document them but verified them that the software satisfies the each requirement.

Final and Sixth phase is Performance, Servicing, Maintenance and Phase out. They maintain the validation state of the Analyst software at the change control point. They followed the pre-defined plan for evaluating and approving changes to the system. By this way, they maintain the validation state of the software by using change control procedure.

Finally they made the validation report for the whole life cycle. These includes summary of all test results, analyses and conclusion.

For the validation of software they maintain the documents such as, user requirements specification, configuration specification, IQ, OQ and PQ test plans, deviation reports, Quality assurance review and recommendations, validation summary report and validation certification.

Conclusion:

Commercial off the shelf (COTS) software is ubiquitous and inevitable. As product gives significant savings in procurement and maintenance, it becomes mandatory in some government organization and business programs. Developers have to choose best suitable COTS components for the company which is highly qualified and also certified.

For the perfect fit for the company’s requirements, there are some evaluation techniques for the COTS software such as,

COTS based Integrated System Development (CISD) method

Off-The -Shelf-Option(OTSO)

Checklist Driven Software Evaluation Methodology(CDSEM)

Procurement- Oriented Requirements Engineering(PORE)

All system needs to be validated for the intended use of the software and also compliance with regulation. COTS also need to be validated. And two factors need to be considering while doing validation of COTS.

Systems actual compliance with applicable rules, standards and general guidelines

Formal validation

Also, there are some general principles that we have to keep in mind during the validation process those are, requirements, defect preventions, time and effort, software life cycle, plans, procedures, revalidation after a change in software, validation coverage, independence review, flexibility and responsibility. General life cycle model is very useful for the validation process of the COTS system as it includes six phase of validation process.

Requirements and system acceptance test specification

Design and implementation process

Inspection and Testing

Precautions

Installation and System acceptance test

Performance, Servicing, Maintenance and Phase out

Also, case study of validating Analyst software supports this model that they used the life cycle model. This is the best approach for the validation of the COTS. And they also got satisfactory result and qualify the Analyst software successfully.

As every coin has two sides, COTS have some demerits along with bindle of merits. Some of the cons are increased configurable problems, inability to meet all the requirements etc. But there are many pros such as, low cost, decreased development effort, faster procurement process etc. Because of many advantages of using COTS system, it is very vital solution for the company for computer system as COTS save time, money and effort.