When a requirement is expressed, the business expects the implementation to just work. But rarely in the history of business or software has there been agreement over whether the final product really works the way it was intended. From the engineering perspective, any issues related to performance, reliability, security, and so on should only be considered defects if they don’t match the requirements. From the business perspective, the presumption is that avoiding such issues is inherently a part of the expressed requirements.
The cause of this gap between business expectations and the evolution of the requirement through the development and testing phases goes deeper than semantics. It is a classic symptom associated with traditional organizational culture in which engineers do not feel that the business understands the complexity of the software development process. At the same time, people on the business side of the organization see engineers afflicted with Smart Developer Syndrome, a pathological disorder in which engineers do whatever they think is best because, hey, they’re smart.
Implementing Policies
So what’s the solution? How does an organization bridge the geek gap between business goals and development processes? Policy is the key to ensuring that engineers deliver software according to expectations and that quality practices are ingrained into the software development workflow. By implementing a policy-driven development approach, organizations reduce risk, increase productivity, and lower cost over the total software development lifecycle from build to support. Policy-driven development is based on three core activities:
1. Defining expectations in the form of policies that govern how engineers develop and test software
2. Training engineers on the business objectives driving those policies
3. Automatically monitoring policy adherence with the help of an unobtrusive infrastructure.
Clearly defining enforceable and measurable policies creates consistency and accuracy for ensuring that the strict quality processes you’ve defined on paper are put into practice. Furthermore, policy enforcement automation is necessary for efficiently achieving clarity and measurability.
Guidelines versus Policies
If you ask an organization about their development policies, many would be quick to point to their best practices and guidelines. But guideline and policy are not interchangeable terms. Guidelines describe suggested behavior, whereas policies prescribe expected behavior. All policies have the following three components:
Practices must be automatically enforceable: It is not feasible to manually check whether engineers are complying with policies. An automated mechanism must be in place to check for violations, notify Notifications for policy violations must be targeted: Only the engineers whose code is in violation of the policy should be notified. This way, engineers in compliance with policies may continue to work uninterrupted.
Automated workflows for remediating policy violations must be in place: The way in which engineers find and fix policy violations should be defined and automated.
The good news for embedded software organizations is that technology exists for automatically enforcing policy, which if done properly will lead to improved quality, increased productivity, lower development costs, and reduced risks.
Matching Policies with Procedures
“The Hoover Dam must not leak” is a sound policy. It establishes an organizational goal in plain, unequivocal language. Now consider the policy “Check for leaks once every day.” On the surface, this policy seems to move the organization toward its goals, but a closer examination exposes the types of shortcomings prevalent in many embedded development shops.
Where this policy falls flat is that it allows problems to occur in the first place. Instead of upholding the goals of the first policy, it only checks for symptoms associated with failing to meet the organizational goals established in “The Hoover Dam must not leak.” Unfortunately, many development testing practices follow this treat-the-symptoms-not-the-cause model.
A much better policy might be “Every day, strengthen the weakest point of the dam.” By matching policies and procedures to meet the intended business goals, you create a framework for ensuring and maintaining strength, security, and reliability of your embedded software.
Image may be NSFW.
Clik here to view.
Policy and Development Testing
Development testing is the continuous integration of software testing activities throughout the development lifecycle. This reduces technical debt and enables defect prevention, which is the key to increasing efficiency and reducing risks over the life of the application. A development testing platform is an infrastructure that automates the consistent application of development testing practices—static analysis, unit testing, peer review, coverage analysis, runtime error detection, etc.—as well as accurately and objectively measures productivity and application quality.
Each organization may find different development testing activities more valuable depending on specific circumstances. Policy-driven development testing ensures that those development testing activities are correctly applied in a value-focused, risk-aware, and auditable manner. A development testing platform accepts input in the form of policies and interprets them as a set of rules defining one or more processes. The platform automates the processes, tracks adherence to the policies, and validates outcomes against expectations. That is, the development testing platform gives business leaders more control over the development process and more visibility into the effects of process changes.
Conclusion
It’s worth noting that development testing is not a replacement for QA, which should serve as a process for confirming that the software functions as intended. Nor is it just a “shift-left” of the testing process. The emergence of development testing represents an alignment with the more iterative, agile-like development processes that organizations are engaging with today. Development testing is a model of software development behavior that, if executed thoughtfully and consistently, will create an environment in which software bugs cannot survive.
Source: http://www.edn.com/design/test-and-measurement/4417172/1/Test-to-avoid-embedded-software-bugs