Panel 4:

A Framework for Distributed Component Test Certification Facility - Conformity and Compliance Testing

[ About the Speaker s ] [ Abstract ]

About the Speakers:

Dr Sita Ramakrishnan
School CSSE, Monash University, Australia
sitar@csse.monash.edu.au

Sita received her Ph.D degree from the School CSSE, Monash University. Since joining Monash in 1989, her interests has been on O-O and SE in general, and in particular on O-O design, metrics and testing, components and reuse, distributed software systems architecture, development and testing, software engineering curriculum, web-engineering and web-based learning. Since 1993, she has published over 39 papers in international journals, conferences and others.

Associate Prof Christine Mingins
School CSSE, Monash University, Australia
cmingins@csse.monash.edu.au

Christine is an Assoc Prof and Assoc Head at the School CSSE. She has supervised a number of PhD students to successful completions. She hold various research grants and has been leading the TOOLS Pacific series for a number of years. More recently, she has been very closely involved with the .NET initiative and is an active member of the ECMA committee as part of the .NET initiative.

Professor. Jean-Marc Jezequel
IRISA, Rennes, France
jezequel@irisa.fr

Pr. Jean-Marc Jezequel received his Ph.D. in CS from the University of Rennes in 1989. He first worked in Telecom industry (at Transpac) before joining the CNRS. Currently, he is a professor at the University of Rennes, leading an INRIA research project. His interests include software engineering based on OO technologies for telecommunications and distributed systems. He is the author of the books "Object-Oriented Software Engineering with Eiffel" and "Design Patterns and Contracts" and of more than 60 publications in international journals and conferences.

Dr. Ing. Andreas Ulrich
Siemens AG, München / Germany
andreas.ulrich@mchp.siemens.de

Andreas received his Ph.D. in CS from Magdeburg university in 1998. He works at Siemens AG in the Corporate R&D division. He has extensive experiences in the area of software testing, especially of OO and distributed systems and provides consultancy services. His research interests include distributed systems design, formal specification techniques, and software testing of real-time systems. He is a PC member of the IFIP International Conference on Testing of Communicating Systems (TestCom).

Abstract:

A research and development facility is proposed for conformity and compliance testing and to bridge the gap between testing and analysis researchers and industry practitioners. This facility can provide advice in the form of individual consultancies and workshops on component design for compliance to interoperability standards, will directly benefit the competitiveness of such software developers in the international marketplace [13].

Software is at the centre of modern, global information and communication infrastructures. A high degree of trust in the underlying software is required to support the integrity of the infrastructure. Software trust is a quality concern. The suppliers and consumers of software applications must trust that the software meets their requirements, is reliable, robust and complies with specified conformance requirements. Software testing is critical to every aspect of information infrastructure protection [6,12]. Most software quality assessment techniques are qualitative and focus on process and personnel assessment. Techniques and standards such as ISO 9000, Carnegie Mellon's capability maturity models (CMM) and National Association Testing Authorities - NATA's ISO/IEC17025 cannot offer product assurance or certification, as they do not test the actual software. In the current era of component-based distributed software systems, the key software testing research that is needed is the integration testing of heterogeneous software systems. A focussed research initiative in software testing which deals with information integrity issues of reliability and fault-tolerance is proposed to address the development of quantitative product quality measures.

According to a study by the Gartner group, a research and consulting firm in the U.S., the market for pre-built components will grow from US$1.4 billion in 1997 to US$8 billion in 2002. Gartner predicts that at least 70% of new software applications will be assembled primarily from components by 2003[1]. The Component market is predicted to emerge as the dominant commodity software concept [3,4]. The Windows 2000 and Enterprise Java Beans (EJB) are mature platforms that support enterprise scale component-based applications. Over the next couple of years, most new applications will be delivered onto one of these platforms, and in both cases, use of components is mandatory from a technical perspective. The reality of distributed component platforms with multiple competing open and proprietary standards (.NET, Java, EJB, CORBA etc) is leading to industry awareness that components are commodities and market-leading applications are the differentiators [3, 4].

Commercial-off-the-shelf (COTS) consumers should be able to test components for robustness and conformance to standards. One of the verification and validation techniques for testing the robustness of a COTS component is software fault injection [7]. However, since the developer may not have access to the source code of COTS components, it is not usually possible to fix detected errors. By wrapping the COTS component and creating a barrier that disallows the component from incorrect and undesirable functionality is one way of ensuring trust in the component [8,9]. This technique of using software robustness wrappers handles exceptions thrown by a component by returning specified error codes that are known a priori to be handled robustly. We have recently implemented a prototype for a .NET component, which uses wrappers similar to [8,9].

The notion of Trusted Components is gaining currency in the software industry [1,12]. A facility that can facilitate trust in software components by testing for conformance to a standard or against a client-supplier component testing model is important for business clients.

Five Key Questioned to be addressed by the Panel

· How to increase the competitiveness of SMEs w.r..t.software testing

· What is meant by conformance to a standard?

· Why is a conformance and compliance testing program the way to go?

· Design for testability of distributed component-based architectural models

· Consider certification criteria for conformance and compliance properties of components

References

1. Williams J D, Raising Components, Application Development Trends, vol. 7, no.9, Sep 2000, pp.27--32.

2. Ramakrishnan Sita and Schmidt H, Model Engineering for Distributed O-OComponent Testing, Object-Oriented Information Systems, Springer -Verlag Pub., Dec 2000, pp. 407-413

3. Sprott D Open Market Components, The Forum for Component Based Development and Integration, Jan. 2000

4. Meyer, B. and Mingins, C., Component-Based Development: From Buzz to Spark Computer, IEEE, Inc., New Jersey USA, Vol 32 Number 7, pp 35 - 37, 1999

5. Meyer B., Mingins C. and Schmidt H., Providing Trusted Components to the Industry Computer: Innovative Technology for Computer Professionals, IEEE Inc., Piscataway USA, Vol 31 No 5, Column Article, pp 104 - 105, 1998

6. Voas J., Can Critical Information Infrastructure Protection be Achieved With Untested Software? IEEE Software, March 1999

7. Voas J.M. and McGraw G. Software Fault Injection: Inoculating Programs against Errors. John Wiley and Sons, NY 1998.

8. McGraw G. and Viega J. Why COTS Software Increases Security Risks, 1^st International ICSE Workshop on Testing Distributed Component-based Systems, May 1999

9. Ghosh A.K., Schmid M. and Hill F. Wrapping Windows NT Software for Robustness, Fault Tolerant Computing Symposium (FTCS-29), June 99.

10. Hecht H. Rare conditions An Important Cause of Failures. In Proc. of the Eighth Annual Conference on Computer Assurance, pp 81-85, National Institute of Standards, June 1993.

11. Bennett D. How can a COTS Vendor Trust a component? , Workshop on Trusted Components held in conjunction with the International Conference - TOOLS Pacific Nov1999, Melbourne

12. Ramakrishnan Sita Validating Interoperable Distributed Software and Systems (VISWAS), Ph.D thesis, School CSSE, Monash University, Australia, May 2001

13. Ramakrishnan Sita and Mingins Christine A Faciltity for Conformity and Compliance Testing, Commonwealth Govt Australia, Dept Communication IT and the Arts (DCITA) two year funded Test-IT project, June 2001



				Panel 4: A Framework for Distributed Component Test Certification Facility - Conformity and Compliance Testing [ About the Speaker s ] [ Abstract ] About the Speakers: Dr Sita Ramakrishnan School CSSE, Monash University, Australia sitar@csse.monash.edu.au Sita received her Ph.D degree from the School CSSE, Monash University. Since joining Monash in 1989, her interests has been on O-O and SE in general, and in particular on O-O design, metrics and testing, components and reuse, distributed software systems architecture, development and testing, software engineering curriculum, web-engineering and web-based learning. Since 1993, she has published over 39 papers in international journals, conferences and others. Associate Prof Christine Mingins School CSSE, Monash University, Australia cmingins@csse.monash.edu.au Christine is an Assoc Prof and Assoc Head at the School CSSE. She has supervised a number of PhD students to successful completions. She hold various research grants and has been leading the TOOLS Pacific series for a number of years. More recently, she has been very closely involved with the .NET initiative and is an active member of the ECMA committee as part of the .NET initiative. Professor. Jean-Marc Jezequel IRISA, Rennes, France jezequel@irisa.fr Pr. Jean-Marc Jezequel received his Ph.D. in CS from the University of Rennes in 1989. He first worked in Telecom industry (at Transpac) before joining the CNRS. Currently, he is a professor at the University of Rennes, leading an INRIA research project. His interests include software engineering based on OO technologies for telecommunications and distributed systems. He is the author of the books "Object-Oriented Software Engineering with Eiffel" and "Design Patterns and Contracts" and of more than 60 publications in international journals and conferences. Dr. Ing. Andreas Ulrich Siemens AG, München / Germany andreas.ulrich@mchp.siemens.de Andreas received his Ph.D. in CS from Magdeburg university in 1998. He works at Siemens AG in the Corporate R&D division. He has extensive experiences in the area of software testing, especially of OO and distributed systems and provides consultancy services. His research interests include distributed systems design, formal specification techniques, and software testing of real-time systems. He is a PC member of the IFIP International Conference on Testing of Communicating Systems (TestCom). Abstract: A research and development facility is proposed for conformity and compliance testing and to bridge the gap between testing and analysis researchers and industry practitioners. This facility can provide advice in the form of individual consultancies and workshops on component design for compliance to interoperability standards, will directly benefit the competitiveness of such software developers in the international marketplace [13]. Software is at the centre of modern, global information and communication infrastructures. A high degree of trust in the underlying software is required to support the integrity of the infrastructure. Software trust is a quality concern. The suppliers and consumers of software applications must trust that the software meets their requirements, is reliable, robust and complies with specified conformance requirements. Software testing is critical to every aspect of information infrastructure protection [6,12]. Most software quality assessment techniques are qualitative and focus on process and personnel assessment. Techniques and standards such as ISO 9000, Carnegie Mellon's capability maturity models (CMM) and National Association Testing Authorities - NATA's ISO/IEC17025 cannot offer product assurance or certification, as they do not test the actual software. In the current era of component-based distributed software systems, the key software testing research that is needed is the integration testing of heterogeneous software systems. A focussed research initiative in software testing which deals with information integrity issues of reliability and fault-tolerance is proposed to address the development of quantitative product quality measures. According to a study by the Gartner group, a research and consulting firm in the U.S., the market for pre-built components will grow from US$1.4 billion in 1997 to US$8 billion in 2002. Gartner predicts that at least 70% of new software applications will be assembled primarily from components by 2003[1]. The Component market is predicted to emerge as the dominant commodity software concept [3,4]. The Windows 2000 and Enterprise Java Beans (EJB) are mature platforms that support enterprise scale component-based applications. Over the next couple of years, most new applications will be delivered onto one of these platforms, and in both cases, use of components is mandatory from a technical perspective. The reality of distributed component platforms with multiple competing open and proprietary standards (.NET, Java, EJB, CORBA etc) is leading to industry awareness that components are commodities and market-leading applications are the differentiators [3, 4]. Commercial-off-the-shelf (COTS) consumers should be able to test components for robustness and conformance to standards. One of the verification and validation techniques for testing the robustness of a COTS component is software fault injection [7]. However, since the developer may not have access to the source code of COTS components, it is not usually possible to fix detected errors. By wrapping the COTS component and creating a barrier that disallows the component from incorrect and undesirable functionality is one way of ensuring trust in the component [8,9]. This technique of using software robustness wrappers handles exceptions thrown by a component by returning specified error codes that are known a priori to be handled robustly. We have recently implemented a prototype for a .NET component, which uses wrappers similar to [8,9]. The notion of Trusted Components is gaining currency in the software industry [1,12]. A facility that can facilitate trust in software components by testing for conformance to a standard or against a client-supplier component testing model is important for business clients. Five Key Questioned to be addressed by the Panel · How to increase the competitiveness of SMEs w.r..t.software testing · What is meant by conformance to a standard? · Why is a conformance and compliance testing program the way to go? · Design for testability of distributed component-based architectural models · Consider certification criteria for conformance and compliance properties of components References 1. Williams J D, Raising Components, Application Development Trends, vol. 7, no.9, Sep 2000, pp.27--32. 2. Ramakrishnan Sita and Schmidt H, Model Engineering for Distributed O-OComponent Testing, Object-Oriented Information Systems, Springer -Verlag Pub., Dec 2000, pp. 407-413 3. Sprott D Open Market Components, The Forum for Component Based Development and Integration, Jan. 2000 4. Meyer, B. and Mingins, C., Component-Based Development: From Buzz to Spark Computer, IEEE, Inc., New Jersey USA, Vol 32 Number 7, pp 35 - 37, 1999 5. Meyer B., Mingins C. and Schmidt H., Providing Trusted Components to the Industry Computer: Innovative Technology for Computer Professionals, IEEE Inc., Piscataway USA, Vol 31 No 5, Column Article, pp 104 - 105, 1998 6. Voas J., Can Critical Information Infrastructure Protection be Achieved With Untested Software? IEEE Software, March 1999 7. Voas J.M. and McGraw G. Software Fault Injection: Inoculating Programs against Errors. John Wiley and Sons, NY 1998. 8. McGraw G. and Viega J. Why COTS Software Increases Security Risks, 1^st International ICSE Workshop on Testing Distributed Component-based Systems, May 1999 9. Ghosh A.K., Schmid M. and Hill F. Wrapping Windows NT Software for Robustness, Fault Tolerant Computing Symposium (FTCS-29), June 99. 10. Hecht H. Rare conditions An Important Cause of Failures. In Proc. of the Eighth Annual Conference on Computer Assurance, pp 81-85, National Institute of Standards, June 1993. 11. Bennett D. How can a COTS Vendor Trust a component? , Workshop on Trusted Components held in conjunction with the International Conference - TOOLS Pacific Nov1999, Melbourne 12. Ramakrishnan Sita Validating Interoperable Distributed Software and Systems (VISWAS), Ph.D thesis, School CSSE, Monash University, Australia, May 2001 13. Ramakrishnan Sita and Mingins Christine A Faciltity for Conformity and Compliance Testing, Commonwealth Govt Australia, Dept Communication IT and the Arts (DCITA) two year funded Test-IT project, June 2001