Platforms, frameworks and librariescs2103/AY1617S2/files... · Web application frameworks ... (such as in scalability, security, performance, and robustness) than software meant for

Aug 2016 edition – for NUS students only

1

[L10P3]

Less work, more results: reusing existing artifacts

Platforms, frameworks and libraries

Frameworks

Reusability is a major theme in software engineering practices. By reusing tried and tested

“components”, the robustness of a new software system can be enhanced while reducing the

manpower and time requirement. Reusable components come in many forms; it can be reusing

a piece of code, a subsystem, or the whole software architecture. In this section, the software

framework as an example of reusable design is discussed.

The overall structure and execution flow of a specific category of software systems (e.g.,

compilers) can be very similar. A software framework is a general skeleton (system architecture

with partial implementation) for such a category of software. A framework facilitates the

adaptation and customization of some desired functionality. A software framework supplies an

abstraction of the architecture and behaviors to facilitate software development. For some

frameworks, a default behavior is already provided with the complete implementation, which

allows rapid deployment. Note that some frameworks covers the complete software (e.g.,

Drupal: a framework for create online content management systems) while others cover

specific components (e.g., Swing: a framework for creating Java GUIs).

Three examples of popular software frameworks are given below.

Web application frameworks Most web-applications have the following components:

Database: information to be stored or retrieved through a web based user interface. The data are commonly stored in a database.

Web UI: Web pages that present information to a user and accept user input. Request processor: Intermediary between the above two components.

The interaction between the components is also similar in most web based applications. For

example, a web page will request for data through the request processor, which retrieves the

required information from a data source. Upon receiving the requested data, the web page will

then present the data as needed. Instead of coding the components and managing the

interaction between them for every new web application, software developers can utilize a

software framework that supplies the general architecture and focus on the task of

specialization/adaptation so as to suit the requirement. Ruby on Rails is an example a software

framework that is designed for web based application development using the Ruby language.

Drupal is another PHP web application framework used specifically for CMS-type web

applications (CMS = Content Management System).

An automated testing framework A typical API testing framework requires the following steps:

Setup the component to be tested. Supply input to the component under test. Capture output and compare with expected answer. Report and collate statistics.

By extracting the common flow of the testing process, an automated testing framework is

obtained. Examples of the automated testing framework include JUnit for Java, and GUnit for

C++.


2

An IDE framework Eclipse is an IDE framework. Not only is it a ready-to-use IDE for Java, but it is also a framework

for developing IDEs for other languages, as well as adding more IDE features.

Frameworks VS Libraries

Although framework and library are both reuse mechanisms, they are not the same. A library is

a collection of modular code that is general and can be used in many independent programs.

Here are some characteristics of frameworks that differentiate them from libraries:

Libraries are used ‘as is’. In contrast, most frameworks are highly configurable to suit specific needs (using configuration files, or as part of the installation). As an example, Drupal can be configured during installation or by editing configuration files. Furthermore, frameworks are meant to be extended by writing plugins, sub-classing, etc. For example, writing plugins for Eclipse so that it can be used as an IDE for different languages (C++, PHP, etc.), adding modules and themes to Drupal, and adding test cases to JUnit .

Frameworks use a technique called inversion of control, also called the “Hollywood principle” (i.e. don’t call us, we’ll call you!). You write code that will be called by the framework, e.g. writing test methods that will be called by the JUnit framework. In the case of libraries, written code calls libraries.

Platforms

A platform provides a runtime environment for applications. While technically an operating

system can be called a platform, in practice a platform is bundled with various libraries, tools,

and technologies in addition to a runtime environment. For example, Windows PC is a platform

for desktop applications while iOS is a platform for mobile apps. .NET is considered a

framework to write enterprise applications on the Windows platform. Note that ‘enterprise

applications’ means software applications used at organizations level and therefore has to meet

much higher demands (such as in scalability, security, performance, and robustness) than

software meant for individual use. However, note that .NET is a general framework that can be

used to write any enterprise application whereas the framework examples given previously

apply to a much narrower scope.

JavaEE (Java Enterprise Edition, previously called J2EE) is both a framework and a platform for

writing Java enterprise applications. JavaEE comes with a set of libraries, tools, technologies,

techniques for developing any kind of enterprise applications.

Infrastructure services such as connection pooling, load balancing, remote code execution,

transaction management, authentication, security, messaging etc. are done similarly in most

enterprise applications. Both JavaEE and .NET provide these services to applications in a

customizable way without developers having to implement them from scratch every time.

For example, JavaEE

is built on top of JavaSE (Java Standard Edition). includes technologies like JSP, Servlets, JMS, EJB, RMI, etc. provides configurable infrastructure services such as EJB technology’s automatic object

persistence in a database.

Cloud computing

A topic that is loosely related to the reuse of existing artifacts is Cloud computing. Cloud

computing is the delivery of computing as a service over the network, rather than a product

running on a local machine. This means the actual hardware and software is located at a remote


3

location, typically, at a large server farm, while users access them over the network.

Maintenance of the hardware and software is managed by the cloud provider while users

typically pay for only the amount of services they use. This model is similar to the consumption

of electricity; the power company manages the power plant, while the consumers pay them only

for the electricity used. The cloud computing model optimizes hardware and software

utilization and reduces the cost to consumers. Furthermore, users can scale up/down their

utilization at will without having to upgrade their hardware and software. The traditional non-

cloud model of computing is similar to everyone buying their own generators to create

electricity for their own use.

As shown in the above diagram (source: Wikipedia), the cloud contains infrastructure, platform

elements, and applications. Accordingly, the cloud can deliver computing services at three

different levels:

i. Infrastructure as a service (IaaS): IaaS delivers computer infrastructure as a service. For example, a user can deploy virtual servers on the cloud instead of buying physical hardware and installing server software on them. Another example would be a customer using storage space on the cloud for off-site storage of data. Rackspace is an example of an IaaS cloud provider. Amazon Elastic Compute Cloud (Amazon EC2) is another one.

ii. Platform as a service (PaaS): PaaS provides a platform on which developers can build applications. Developers do not have to worry about infrastructure issues such as deploying servers or load balancing as is required when using IaaS. Those aspects are automatically taken care of by the platform. The price to pay is reduced flexibility; applications written on PaaS are limited to facilities provided by the platform. A PaaS example is the Google App Engine where developers can build applications using Java, Python, PHP, or Go whereas Amazon EC2 allows users to deploy application written in any language on their virtual servers.

iii. Software as a service (SaaS): This is when applications can be accessed over the network instead of installing them on a local machine. For example, Google Docs is an SaaS word processing software, while Microsoft Word is a traditional word processing software.


4

Worked examples

[Q1] One of your teammates is proposing to use a recently-released “cool” UI framework for your

class project. List the pros and cons of this idea.

[A1] Pros

The potential to create a much better product by reusing the framework.

Learning a new framework is good for the future job prospects.

Cons

Learning curve may be steep.

May not be stable (it was recently released).

May not allow us to do exactly what we want. While frameworks allow customization,

such customization can be limited.

Performance penalties.

Might interfere with learning objectives of the module.

Note that having more cons does not mean we should not use this framework. Further

investigation is required before we can make a final decision.

--- End of handout ---


1

TESTING

[ L1P3]

Stop Butterflies from Causing Tornadoes: Preventing Regressions

Testing IEEE5 defines testing as ‘operating a system or component under specified conditions, observing

or recording the results, and making an evaluation of some aspect of the system or component’.

Software Under Test

(SUT)

InputsActual output

Expected output

Test case

id: …description: objectives, etc.

1 2

3

Figure 13. Testing workflow

Testing entails the execution of a set of test cases. A test case specifies how to perform a test. At

a minimum, it specifies the input to the software under test (SUT) and the expected behavior. For

example, here is a minimal test case for testing a browser:

Input – Start the browser using a blank page (vertical scrollbar disabled). Then, load

‘longfile.html’ located in the ‘test data’ folder.

Expected behavior – The scrollbar should be automatically enabled upon loading

‘longfile.html’.

The above can be determined from the specification, reviewing similar existing systems, or

comparing to the past behavior of the SUT.

A more elaborate test case can have other details such as those given below.

A unique identifier e.g. TC0034-a

A descriptive name e.g. vertical scrollbar activation for long web pages

Objectives e.g. to check whether the vertical scrollbar is correctly activated when a long

web page is loaded to the browser

Classification information: e.g. priority - medium, category - UI features

Cleanup, if any e.g. empty the browser cache

5 Institute of Electrical and Electronics Engineers


2

A test case failure is a mismatch between the expected behavior and the actual behavior. In the

browser example, a test case failure is implied if the scrollbar remains disabled after loading

‘longfile.html’. A failure is caused by a defect (or a bug). The scrollbar problem could be due to

an uninitialized variable.

Scripted vs Exploratory testing Imagine you are required to test a new browser that your company has developed. One

approach you may take is to first write a set of test cases based on the browser’s system

specification, perform the test cases, and report any failures to the developers. Alternatively,

instead of using a predetermined set of test cases, you could devise test cases as you proceed

with testing, creating new test cases based on the results of the past test cases. The former

approach is called scripted testing while the latter is called exploratory testing.

Exploratory testing is ‘the simultaneous learning, test design, and test execution’ [1] whereby

the nature of the follow-up test case is decided based on the behavior of the previous test cases.

In other words, running the system and trying out various operations. It is called exploratory

testing because testing is driven by observations during testing. Exploratory testing is also

known as reactive testing, error guessing technique, attack-based testing, and bug hunting.

Exploratory testing usually starts with areas identified as error-prone, based on the tester’s past

experience with similar systems. One tends to conduct more tests for those operations where

more faults are found. For example,

“Hmm... looks like feature x is broken. This usually means feature n and k could be

broken too; we need to look at them soon. But before that, let us give a good test run to

feature y because users can still use the product if feature y works, even if x doesn’t

work. Now, if feature y doesn’t work 100%, we have a major problem and this has to be

made known to the development team sooner rather than later...”

The success of exploratory testing depends on the tester’s prior experience and intuition.

Exploratory testing should be done by experienced testers, using a clear

strategy/plan/framework. Ad-hoc exploratory testing by unskilled or inexperienced testers

without a clear strategy is not recommended for real-world non-trivial systems. While

exploratory testing may allow us to detect some problems in a relatively short time, it is not

prudent to use exploratory testing as the sole means of testing a critical system.

In scripted (or proactive) testing, we use a predetermined set of test cases. Scripted testing is

more systematic, and hence, likely to discover more bugs given sufficient time, while

exploratory testing would aid in quick error discovery, especially if the tester has a lot of

experience in testing similar systems.

So which approach is better – scripted or exploratory? To quote Bach [1],

In some contexts, you will achieve your testing mission better through a more scripted

approach; in other contexts, your mission will benefit more from the ability to create

and improve tests as you execute them. I find that most situations benefit from a mix of

scripted and exploratory approaches.

Regression testing When we modify a system that has been tested, the modification may result in some unintended

and undesirable effects on the system. Such an effect is called a regression. Unfortunately, with


3

software, even a tiny modification can result in a complete meltdown of the whole system6. To

detect and correct regressions, we need to retest all related components. Testing with this

intent is called regression testing. Regression testing is more effective when it is done frequently,

after each small change. However, doing so can be prohibitively expensive if testing is done

manually. Hence, regression testing is more practical when it is fully, if not partially, automated.

Test automation (Text UIs) An automated test case can be run programmatically and the result of the test case (pass or fail)

is determined programmatically. Test automation is especially important since manual

regression testing is tedious and impractical.

A simple way to test a text UI is using input/output re-direction. Let us assume we are testing a

program ‘RouteStore’ which has a text UI. First, we store the test input in the text file ‘input.txt’.

Similarly, we store the output we expect from the SUT to another text file ‘expected.txt’. Now, we

run the program as:

java RouteStore < input.txt > output.txt

or

RouteStore.exe < input.txt > output.txt

The above redirects the text in ‘input.txt’ as the input to ‘RouteStore’ and similarly, redirects the

output of RouteStore to a text file ‘output.txt’. Note that this does not require any code changes

to RouteStore.

Sidebar: The ‘>’ operator and the ‘<’ operator. A command line program written in the usual way (i.e., the way you are used to writing them during the first year programming modules) takes input from keyboard and outputs to the console. That is because those two are default input and output streams, respectively. But you can change that behavior using ‘<’ and ‘>’ operators. For example, if you run TextBuddy in the DOS prompt, the output will be shown in the console. But if you run it like this TextBuddy > output.txt (or java TextBuddy > output.txt) the Operating System then creates a file ‘output.txt’ and stores the output in that file instead of displaying it in the console. Program output using System.out.println (in Java) and cout (in C++) is directed to the console by default. When you add ‘> output.txt’, the OS redirects the output to ‘output.txt’ instead. No file I/O coding is required. Similarly, adding ‘< input.txt’ (or any other filename) makes the OS redirect the contents of the file as input to the program. The link below explains how Windows does it : http://technet.microsoft.com/en-us/library/bb490982.aspx

All we have to do now is to compare ‘output.txt’ with ‘expected.txt’. This can be done using a

utility such as Windows FC (i.e. File Compare) command or a GUI tool such as Winmerge. For

example, we can run the following two commands.

java RouteStore < input.txt > output.txt

FC output.txt expected.txt

Note that the above technique is only suitable when testing text UIs, and only if the exact output

is known. Automated testing of program components or GUIs (Graphical User Interfaces) will be

addressed in other handouts.

6 In nature, it is said that even something minor such as the flapping of a butterfly’s wing can trigger off a chain of events that can cause a tornado in another part of the world. Something similar can happen in software too. That’s the connection between this lecture content and its title.

http://technet.microsoft.com/en-us/library/bb490982.aspx


4

References

[1] Exploratory testing explained, an online article by James Bach (James Bach is an

industry thought leader in software testing). Softcopy available at

http://www.satisfice.com/articles/et-article.pdf

Worked examples

[Q1] A sample output from a text-based program ‘TriangleDetector’ is given below that determines

whether the three input numbers make up the three sides of a valid triangle. A sample output is

shown below. List test cases you would use to test ‘TriangleDetector’. Two sample test cases are

given below.

C:\> java TriangleDetector

Enter side 1: 34

Enter side 2: 34

Enter side 3: 32

Can this be a triangle? : Yes

Enter side 1:

Sample test cases,

34,34,34 : Yes

0,any valid, any valid : No

[A1] In addition to obvious test cases such as

sum of two sides == third,

sum of two sides < third, …

we may also devise some interesting test cases such as the ones depicted below. Note that their

applicability depends on the context in which the software is operating.

Non-integer number, negative numbers, 0, numbers formatted differently (e.g. 13F),

very large numbers (e.g. MAX_INT), numbers with many decimal places, empty string, …

Check many triangles one after the other (will the system run out of memory?)

Backspace, tab, CTRL+C , …

Introduce a long delay between entering data (will the program be affected by, say the

screensaver?), minimize and restore window during the operation, hibernate the system

in the middle of a calculation, start with invalid inputs (the system may perform error

handling differently for the very first test case), …

Test on different locale.

The main point to note is how difficult it is to test exhaustively, even on a trivial system.

http://www.satisfice.com/aboutjames.shtml

http://www.satisfice.com/articles/et-article.pdf


1

[ L4P2]

Never Too Early to Test: an Introduction to Early Developer Testing

Developer testing It seems logical to have the system built in its entirety before testing the system as a whole (also

known as system testing). But what if a test case fails during system testing? Firstly, locating the

cause of the failure is difficult due to a large search space; in a large system, the search space

could be millions of lines of code, written by hundreds of developers! The failure may also be

due to multiple inter-related bugs. Secondly, fixing the bug upon locating it could result in major

rework, especially if the bug originated during the design or during requirements specification

(i.e. a faulty design or faulty requirements). Furthermore, this process is costly and

unpredictable. The whole team needs to work together to locate and fix bugs. One bug might

'hide' other bugs, which could emerge

only after the first bug is fixed. Too many

bugs found during system testing can lead

to delivery delays. As illustrated by the

graph on the right, the earlier a bug is

found, the easier and cheaper to have it

fixed. That is why developers need to

start testing early, while the system is still

under development. Such early testing

done by developers is called developer

testing.

Automated testing Automated testing requires test drivers. A test driver is a module written specifically for ‘driving’

the SUT (Software Under Test) for the purpose of testing i.e. invoking the SUT with test inputs

and verifying the behavior is as expected. In the code example given below, PayrollTest is a test

driver for the Payroll class that ‘drives’ the PayRoll class by sending it test inputs and printing out

the output. However, it does not verify if the output is as expected.

public class PayrollTestDriver { public static void main(String[] args) { //test setup Payroll p = new Payroll(); //test case 1 p.setEmployees(new String[]{"E001", "E002"}); print("Test 1 output " + p.totalSalary()); //test case 2 p.setEmployees(new String[]{"E001"}); print("Test 2 output " + p.totalSalary()); //more tests System.out.println("Testing completed"); } ... }

The PayrollTest class below not only drives the SUT Payroll class using test inputs, it also

automatically verifies that output for each test input in as expected.

public class PayrollTestAtd {


2

public static void main(String[] args) throws Exception { //test setup Payroll p = new Payroll(); //test case 1 p.setEmployees(new String[]{"E001", "E002"}); // automatically verify the response if (p.totalSalary() != 6400) { throw new Error("case 1 failed "); } //test case 2 p.setEmployees(new String[]{"E001"}); if (p.totalSalary() != 2300) { throw new Error("case 2 failed "); } //more tests... System.out.println("All tests passed"); } }

JUnit is a tool for automated testing of Java programs. Similar tools are available for other

languages.

Below is the code of an automated test for Payroll class, written using JUnit libraries.

public class PayrollTestJUnit { @Test public void testTotalSalary(){ Payroll p = new Payroll(); //test case 1 p.setEmployees(new String[]{"E001", "E002"}); assertEquals(p.totalSalary(), 6400); //test case 2 p.setEmployees(new String[]{"E001"}); assertEquals(p.totalSalary(), 2300); //more tests... } }

Most modern IDEs come packaged with integrated support for testing tools. Figure 14 shows

the JUnit output when running some JUnit tests using the Eclipse IDE.

Testability

Testability is an indication of how easy it is to test an SUT. As testability depends a lot on the

design and implementation. We should try to increase the testability of our software when we

design and implement them.


3

Figure 14. JUnit output on the Eclipse IDE

Test-Driven Development (TDD) Typically, tests are written after writing the SUT. However, TDD advocates writing the tests

before writing the SUT. In other words, first define the precise behavior of the SUT using test

cases, and then write the SUT to match the specified behavior. While TDD has its fair share of

detractors, there are many who consider it a good way to reduce defects. Note that TDD does

not imply writing all the test cases first before writing functional code. Rather, proceed in small

steps:

i. Decide what behavior to implement. ii. Write test cases to test that behavior.

iii. Run those test cases and watch them fail. iv. Implement the behavior. v. Run the test case.

vi. Keep modifying the code and rerunning test cases until they all pass. vii. Refactor code to improve quality.

viii. Repeat the cycle for each small unit of behavior that needs to be implemented.

Worked examples

[Q1] Discuss advantages and disadvantages of developers testing their own code.

[A1] Advantages:

Can be done early (the earlier we find a bug, the cheaper it is to fix).

Can be done at lower levels, for examples, at operation and class level (testers usually

test the system at UI level).

It is possible to do more thorough testing since developers know the expected external

behavior as well as the internal structure of the component.

It forces developers to take responsibility for their own work (they cannot claim that

“testing is the job of the testers”).

Disadvantages:

Developer may unconsciously test only situations that he knows to work (i.e. test it too

“gently”).

Developer may be blind to his own mistakes (if he did not consider a certain

combination of input while writing code, he is likely to miss it again during testing).

Developer may have misunderstood what the SUT is supposed to do in the first place.

Developer may lack the testing expertise.

-- End of Handout --


4


1

[ L8P2]

Quality Assurance: Testing and Beyond

Quality assurance (QA) is the process of ensuring that the software being built has the required

levels of quality.

Quality Assurance = Validation + Verification

QA involves checking two aspects:

a) Building the right system, i.e. are the requirements correct? This is called validation. b) Building the system right, i.e. are the requirements implemented correctly? This is called

verification.

Whether something belongs under validation or verification is not that important. What is more

important is both are done, instead of limiting to verification (i.e., remember that the

requirements can be wrong too).

In this handout, the testing of the product as a whole is discussed. This is in contrast to

developer testing that is performed on a partial system.

Testing is the most common way of assuring quality, however there are other complementary

techniques such as formal verification. The second part of the handout gives a brief introduction

to such QA techniques.

Unit testing Unit testing is a form of early developer testing. Unit testing involves testing individual units

(methods, classes, subsystems, …) and finding out whether each piece works correctly in

isolation. A proper unit test require the unit we are testing to be isolated from other code. If the

unit depends on other code, we may have to use stubs to isolate the unit from its dependencies.

Stubs/Mocks

A stub or a mock7 is a dummy component that receives outgoing messages from the SUT. During

unit testing, stubs are used in place of collaborating objects in order to isolate the SUT from

these objects. Doing this prevents bugs within the collaborating objects from interfering with

the test. A stub has essentially the same interface as the collaborator it replaces, but its

implementation is meant to be so simple that it cannot have any bugs. A stub does not perform

any real computations or manipulate any real data. Typically, a stub could do the following

tasks:

Do nothing – A stub could simply receive method calls without doing anything at all. When a method is required to return something, it will return a default value.

Keep records – A stub could dutifully record information (e.g. by writing to a log file) about the messages it receives. This record can later be used to verify whether the SUT sent the correct messages to collaborating objects.

Return hard-coded responses – A stub could be written to mimic the responses of the collaborating object, but only for the inputs used for testing. That is, it does not know how to respond to any other inputs. These mimicked responses are hard-coded in the stub rather than computed or retrieved from elsewhere, e.g. from a database.

7 Although this handout uses stub and mock interchangeably, some define them slightly differently. However, such subtle differences are beyond the scope of this handout.


2

Dependency injection Dependency injection is the process of ‘injecting’ objects to replace current dependencies with a

different object. This is often used to inject stubs to isolate the SUT from other collaborating

objects so that it can be tested independently. In the example below, a Foo object normally

depends on a Bar object, but we have injected a BarStub object so that the Foo object no longer

depends on a Bar object. Now we can test the Foo object in isolation from the Bar object.

:Bar

:Foo

:BarStub

:Foo

Normal dependency After dependency injection

Given next is a sample testing scenario that tests the totalSalary of the Payroll class. The

production version of the totalSalary method collaborates with the SalaryManager object to

calculate the return value. During testing, the SalaryManager object is substituted with a

SalaryManagerStub object which responds with hard-coded return values.

public class PayrollTestDriver { public static void main(String[] args) { //test setup Payroll p = new Payroll(); p.setSalaryManager(new SalaryManagerStub()); //dependency injection //test case 1 p.setEmployees(new String[]{"E001", "E002"}); assertEquals(2500.0, p.totalSalary()); //test case 2 p.setEmployees(new String[]{"E001"}); assertEquals(1000.0, p.totalSalary()); //more tests System.out.println("Testing completed"); } }

//---------------------------- class Payroll{ private SalaryManager manager = new SalaryManager(); private String[] employees; void setEmployees(String[] employees) { this.employees = employees; } /*the operation below is used to substitute the actual SalaryManager with a stub used for testing */ void setSalaryManager(SalaryManager sm) { this. manager = sm; }


3

double totalSalary(){ double total = 0; for(int i=0;i<employees.length; i++){ total += manager.getSalaryForEmployee(employees[i]); } return total; } } //---------------------------- class SalaryManager{ double getSalaryForEmployee(String empID){ //code to access employee’s salary history //code to calculate total salary paid and return it } } //---------------------------- class SalaryManagerStub extends SalaryManager{ /* this method returns hard coded values used for testing */ double getSalaryForEmployee(String empID){ if(empID.equals("E001")) { return 1000.0; }else if(empID.equals("E002")){ return 1500.0; }else { throw new Error("unknown id"); } } }

Integration testing In Integration testing we test whether different parts of the software ‘work together’ (i.e.

integrates) as expected. Here, we assume the individual parts have been unit tested already.

Therefore, integration tests aim to discover bugs in the ‘glue code’ that are often the result of

misunderstanding of what the parts are supposed to do vs what the parts are actually doing. For

example let us assume a class Car users classes Engine and Wheel.

First, we should unit test Engine and Wheel.

Next, we should unit test Car in isolation of Engine and Wheel, using stubs for Engine and Wheel.

After that, we can do an integration test for Car using it together with the Engine and Wheel classes to ensure the Car integrates properly with the Engine and the Wheel.

In the example above, if the Car class assumed a Wheel can support 200 mph speed but the

Wheel can only support 150 mph, it is the integration test that is supposed to uncover this

discrepancy.

System testing Taking the whole system, instead of a part of the system, and testing it against the system

specification is called system testing. System testing is typically done by a testing team (also

called a QA team). System test cases are based exclusively on the specified external behavior of

the system. Sometimes, system tests go beyond the bounds defined in the specification. This is

useful when testing that the system fails ‘gracefully’ having pushed beyond its limits. Take the

example of an SUT (software under test) that is a browser capable of handling web pages

containing up to 5000 characters. A test case can involve loading a web page containing more


4

than 5000 characters. The expected ‘graceful’ behavior would be to ‘abort the loading of the

page and show a meaningful error message’. This test case would fail if the browser attempted

to load the large file anyway and crashed.

Note that system testing includes testing against non-functional requirements too. Here are

some examples.

Performance testing – to ensure the system responds quickly. Load testing (also called stress testing or scalability testing) – to ensure the system can

work under heavy load. Security testing – to test how secure the system is. Compatibility testing, interoperability testing – to check whether the system can work

with other systems. Usability testing – to test how easy it is to use the system. Portability testing – to test whether the system works on different platforms.

Acceptance testing Acceptance testing, also called User Acceptance Testing (UAT), is a type of validation test carried

out to show that the delivered system meets the requirements of the customer. Similar to

system testing, acceptance testing involves testing the whole system against the requirements

specification (rather than the system specification). Note the two specifications need not be the

same. For example, requirements specification could be limited to how the system behaves in

normal working conditions while the system specification can also include details on how it will

fail gracefully when pushed beyond limits, how to recover, additional APIs not available for

users (for the use of developers/testers), etc.

Acceptance testing comes after system testing. It is usually done by a team that represents the

customer, and it is usually done on the deployment site or on a close simulation of the

deployment site. UAT test cases are often defined at the beginning of the project, usually based

on the use case specification. Successful completion of UAT is often a prerequisite to the project

signoff.

Acceptance tests gives an assurance to the customer that the system does what it is intended to

do. Besides, acceptance testing is important because a system could work perfectly on the

development environment, but fail in the deployment environment due to subtle differences

between the two.

Alpha and Beta testing Alpha testing is performed by the users, under controlled conditions set by the software

development team. Beta testing is performed by a selected subset of target users of the system

in their natural work setting. An open beta release is the release of not-yet-production-quality-

but-almost-there software to the general population. For example, Google’s Gmail was in ‘beta’

for years before the label was finally removed.

GUI testing If a software product has a GUI component, all product-level testing (i.e. the types of testing

mentioned above) need to be done using the GUI. However, testing the GUI is much harder than

testing the CLI (command line interface) or API, for the following reasons:

Most GUIs contain a large number of different operations, many of which can be performed in any arbitrary order.


5

GUI operations are more difficult to automate than API testing. Reliably automating GUI operations and automatically verifying whether the GUI behaves as expected is harder than calling an operation and comparing its return value with an expected value. Therefore, automated regression testing of GUIs is rather difficult. However, there are testing tools that can automate GUI testing. For example, TestFx and support automated testing of JavaFX GUIs and Selenium (http://seleniumhq.org/) can be used to automate testing of Web application UIs. VisualStudio supports ‘record replay’ type of GUI test automation.

The appearance of a GUI (and sometimes even behavior) can be different across platforms and even environments. For example, a GUI can behave differently based on whether it is minimized or maximized, in focus or out of focus, and in a high resolution display or a low resolution display.

One approach to overcome the challenges of testing GUIs is to

minimize logic aspects in the GUI. Then, bypass the GUI to test

the rest of the system using automated API testing. While this

still requires the GUI to be tested manually, the number of such

manual test cases can be reduced as most of the system has been

tested using automated API testing.

Test coverage In the context of testing, coverage is a metric used to measure the extent to which testing

exercises the code. Here are some examples of different coverage criteria:

Function/method coverage measures the coverage in terms of functions executed e.g. testing executed 90 out of 100 functions.

Statement coverage measures coverage in terms of the number of line of code executed e.g. testing executed 23k out of 25k LOC.

Decision/branch coverage measures coverage in terms of decision points e.g. an if statement evaluated to both true and false with separate test cases during testing.

Condition coverage measures coverage in terms of boolean sub-expressions, each evaluated to both true and false with different test cases. Condition coverage is not the same as the decision coverage; e.g. if(x>2 && x<44) is considered one decision point but two conditions. For 100% branch or decision coverage, two test cases are required:

(x>2 && x<44) == true : [ e.g. x = 4]

(x>2 && x<44) == false : [ e.g. x = 100]

For 100% condition coverage, three test cases are required

(x>2) == true , (x<44) == true : [e.g. x = 4]

(x<44) == false : [ e.g. x = 100]

(x>2) == false : [ e.g. x = 0]

Path coverage measures coverage in terms of possible paths through a given part of the code executed. 100% path coverage means all possible paths have been executed. A commonly used notation for path analysis is called the Control Flow Graph (CFG). For an introduction to CFGs, refer to the side-note below.

Entry/exit coverage measures coverage in terms of possible calls to and exits from the operations in the SUT.

Measuring coverage is often done using coverage analysis tools. Coverage measurements are

used to improve testing E&E (Effectiveness and Efficiency). For example, if a set of test cases

does not achieve 100% branch coverage, more test cases are added to cover missed branches.

GU

I

Logic

Automated API tester

Manual testing

http://seleniumhq.org/


6

[Side-Note] Control Flow Graphs (CFG) This topic is not examinable CFG is a graphical representation of the execution paths of a code fragment. A CFG consists of: Nodes: Each node represents one or more sequential statements with no branches Directed Edges: Each edge represents a branch, a possible execution path Given below is the CFG notation :

A set of sequential statements

(without any branches) is

represented as a single node. E.g.

x=2; //node 1

y=3; //node 1

z=x+y; //node 1

print (z); //node 1

1

Conditional statements: E.g.

if (x < 10) then //node 1

z = x + y; //node 2

else z = x – y; //node 3

z = z + 2; //node 4

1

2

3

T

F

4

Loops: E.g.

x++; //node 0

while (x < 10) { //node 1

z = x+ y; //node 2

x++; //node 2

}

resetX(); //node 3

1 2T

F

30

Multi-way branching: E.g.

x++; //node 0

switch (x){ //node 1

case 0:

z = x; break; //node 2

case 1:

case 2:

z = y; break; //node 3

default:

z = x-y; //node 4

}

z = x+y ; //node 5

1

2

35

4

0

Note how the same edge represents

both case 1 and case 2.


7

The figure below shows the complete CFG for the min function given below.

void min(int[] A){

int min = A[0]; //node 1

int i = 1; //node 1

int n = A.length; //node 1

while (i < n){ //node 2

if (A[i] < min) //node 3

min = A[i]; //node 4

i++; //node 5

}

print(min); //node 6

}

1

2

3

TF

456

TF

It is recommended to have exactly one entry edge and exactly one exit edge for each CFG. Sometimes a logical node (i.e. a node that does not represent an actual program statement) is added to enforce the “exactly one exit edge” rule. Node 5 in the figure below is a logical node.

void foo(){

int min = A[0]; //node 1

if (A[i] < min) //node 2

min = A[i]; //node 3

else

i++; //node 4

}

}

1

5

2

43

FT

A path is a series of nodes that can be traversed from the entry edge to the exit edge in the direction of the edges that link them. For example, 1-2-4-5 in the above CFG is a path.

Other QA techniques There are many QA techniques that do not involve executing the SUT. Given next are a number

of such techniques that can complement the testing techniques discussed so far.

Inspections & reviews Inspections involve a group of people systematically examining a project artefact to discover

defects. Members of the inspection team play various roles during the process, such as the

author - the creator of the artefact, the moderator - the planner and executer of the inspection

meeting, the secretary - the recorder of the findings of the inspection, and the inspector/reviewer

- the one who inspects/reviews the artefact. All participants are expected to have adequate

prior knowledge of the artefact inspected. An inspection often requires more than one meeting.

For example, the first meeting is called to brief participants about the artefact to be inspected.

The second meeting is called once the participants have studied the artefact. This is when the

actual inspection is carried out. A third meeting could be called to re-inspect the artefact after

the defects discovered as an outcome of the inspection are fixed. An advantage of inspections is


8

that it can detect functionality defects as well as other problems such as coding standard

violations. Furthermore, inspections can verify non-code artefacts and incomplete code, and do

not require test drivers or stubs. A disadvantage is that an inspection is a manual process and

therefore, error prone.

Formal verification Formal verification uses mathematical techniques to prove the correctness of a program. An

advantage of this technique over testing is that it can prove the absence of errors. However, one

of the disadvantages is that it only proves the compliance with the specification, but not the

actual utility of the software. Another disadvantage is that it requires highly specialized

notations and knowledge which makes it an expensive technique to administer. Therefore,

formal verifications are more commonly used in safety-critical software such as flight control

systems.

Static analyzers These are tools that automatically analyze the code to find anomalies such as unused variables

and unhandled exceptions. Detection of anomalies helps in improving the code quality. Most

modern IDEs come with some inbuilt static analysis capabilities. For example, an IDE will

highlight unused variables as you type the code into the editor. Higher-end static analyzers can

check for more complex (and sometimes user-defined) anomalies, such as overwriting a

variable before its current value is used.


1

[ L8P3]

Heuristics for Better Test Case Design

Test case design approaches Exploratory testing vs scripted testing Previously, two alternative approaches to test case design were discussed: exploratory vs

scripted. That distinction was based on when the test cases are designed. In scripted approach,

they are designed in advance. In the exploratory approach, they are designed on the fly.

Given next are alternative approaches based on some other aspects of testing.

Black-box vs white-box This categorization is based on how much of SUT (software under test) internal details are

considered when designing test cases. In the black-box approach, also known as specification-

based or responsibility-based testing, test cases are designed exclusively based on the SUT’s

specified external behavior. In the white-box approach, also known as glass-box or structured or

implementation-based testing, test cases are designed based on what is known about the SUT’s

implementation, i.e. the code.

Knowing some important information about the implementation can help in black-box testing.

This kind of testing is sometimes called gray-box testing. For example, if the implementation of a

sort operation uses an algorithm to sort lists shorter than 1000 items and another to sort lists

longer than 1000 items, more meaningful test cases can then be added to verify the correctness

of both algorithms.

Test case design techniques

Testing all possible ways of using the SUT requires writing an infinite number of test cases. For

example, consider the test cases for adding a String object to a Collection:

Add an item to an empty collection.

Add an item when there is one item in the collection.

Add an item when there are 2, 3, .... n items in the collection.

Add an item that has an English, a French, a Spanish, … word.

Add an item that is the same as an existing item.

Add an item immediately after adding another item.

Add an item immediately after system startup.

...

Exhaustive testing of this operation can take many more test cases. As you can see from the

example above, except for trivial systems, exhaustive testing is not possible!

Program testing can be used to show the presence of bugs, but never to show their absence!

--Edsger Dijkstra

Every test case adds to the cost of testing. In some systems, a single test case can cost thousands

of dollars (e.g. on-field testing of flight-control software). Therefore, test cases have to be

designed to make the best use of testing resources. In particular:

Testing should be effective, i.e. it finds a high % of existing bugs. A set of test cases that finds 60 defects is more effective than a set that finds only 30 defects in the same system.


2

Testing should be efficient, i.e. it has a high rate of success (bugs found/test cases). A set of 20 test cases that finds 8 defects is more efficient than another set of 40 test cases that finds the same 8 defects.

Given below are various tools and techniques used to improve E&E (Effectiveness and

Efficiency) of testing.

Equivalence partitioning

Consider the testing of the following operation.

isValidMonth (int m): boolean

Description: checks if m is in the range [1..12]. returns true if m is in the range, false

otherwise.

It is inefficient and impractical to test this method for all integer values [-MIN_INT to MAX_INT].

Fortunately, there is no need to test all possible input values. For example, if the input value 233

failed to produce the correct result, the input 234 is likely to fail too; there is no need to test

both. In general, most SUTs do not treat each input in a unique way. Instead, they process all

possible inputs in a small number of distinct ways. That means a range of inputs is treated the

same way inside the SUT. Testing one of the inputs for a given range should be as good as

exhaustively testing all inputs in that range.

Equivalence partitioning (EP) is a technique that uses the above observation to improve the

efficiency and effectiveness of testing. By dividing possible inputs into groups which are likely to

be processed similarly by the SUT, testing every possible input in each group is avoided. Such

groups of input are called equivalence partitions (or equivalence classes). Equivalence

partitioning can minimize test cases that are unlikely to find new bugs.

Equivalence partitions are usually derived from the specifications of the SUT. Preconditions and

postconditions can also help in identifying partitions. For example, these could be equivalence

partitions for the isValidMonth example:

[MIN_INT ... 0] (below the range that produces ‘true’)

[1 … 12] (the range that produces ‘true’)

[13 … MAX_INT] (above the range that produces ‘true’)

Note that the equivalence partitioning technique does not suggest the number of test cases to

pick from each partition. It depends on how thorough the test is.

Here’s an example from an OO system. Consider the Minesweeper system that was explored

previously. What are the equivalence partitions for the newGame() operation of the Logic

component? In general, equivalence partitions of all data participants8 that take part in the

operation have to be considered. These include

the target object of the operation call, input parameters of the operation call, and other data/objects accessed by the operation such as global variables.

Since newGame() does not have any parameters, the only participant is the Logic object itself.

Note that if the glass-box or the grey-box approach is used, other associated objects that are

involved in the operation might also be included as participants. For example, Minefield object

8 We use the term “data participants” to mean both objects and primitive values. Note that this is not a standard term


3

can be considered as another participant of the newGame() operation. Here, the black-box

approach is assumed.

Next, identify equivalence partitions for each participant. Will the newGame operation behave

differently for different Logic objects? If yes, how will it differ? In this case, yes, it might behave

differently based on the game state. Therefore, the equivalence partitions are: PRE_GAME (i.e.

before the game starts, minefield does not exist yet), READY (i.e. a new minefield has been

created and waiting for player’s first move), IN_PLAY, WON, LOST.

Here’s another example. Consider the markCellAt(int x, int y) operation of the Logic component.

Applying the technique described above, the equivalence partitions for the markCellAt operation

can be obtained. The partitions in bold (green) represent valid inputs. Here, a Minefield of size

WxH is assumed :

Logic: PRE_GAME, READY, IN_PLAY, WON, LOST

x: [MIN_INT..-1] [0..(W-1)] [W..MAX_INT]

y: [MIN_INT..-1] [0..(H-1)] [H..MAX_INT]

Cell at (x,y): HIDDEN, MARKED, CLEARED

Here’s another example. Consider the push operation from a DataStack class.

Operation: push(Object o): boolean

Description: Throws MutabilityException if the global flag FREEZE==true

Else, throws InvalidValueException if o==null.

Else, returns false if the stack is full.

Else, puts o at the top of the DataStack and returns true.

Here are the equivalence partitions for the push operation.

DataStack object (ds): [full] [not full]

o: [null] [not null]

FREEZE: [true][false]

A test case for the push operation can be a combination of the equivalence partitions. Given

below is such a test case.

id: DataStack_Push_001

description: checks whether pushing onto a full stack works correctly

input: ds is full, o!=null, FREEZE==false

expected output: returns false, ds remains unchanged

How are equivalence partitions combined and how many test cases to create? This question is

addressed later in this handout. Moreover, the expected output should specify the return value

as well as the state of all data participants of the operation that may be changed during the

operation.

Knowledge of how the SUT behaves is used when deriving equivalence partitions for a given

data participant. The table below illustrates some examples. However, note that the EP

technique is merely a heuristic and not an exact science. The partitions derived depend on how

one ‘speculates’ the SUT to behave internally. Applying EP under a glass-box or gray-box

approach can yield more precise partitions.


4

Specification Equivalence partitions

isValidFlag(String s): boolean

Returns true if s is one of [“F”, “T”, “D”]. The

comparison is case-sensitive.

[“F”] [“T”] [“D”] [“f”, “t”, “d”] [any other

string][null]

squareRoot(String s): int

Pre-conditions: s represents a positive

integer

Returns the square root of s if the square

root is an integer; returns 0 otherwise.

[s is not a valid number] [s is a negative

integer] [s has an integer square root] [s does

not have an integer square root]

isPrimeNumber(int i): boolean

Returns true if i is a prime number

[prime numbers] [non-prime numbers]

* there are too many prime numbers to

consider each one as a separate equivalence

partition

When a data participant of an SUT is expected to be a subtype of a given type, each subtype that

has a bearing on the SUT’s behavior should be treated as a separate equivalence partition. For

example, consider the following operation.

Operation: compare(Expression first, Expression second): boolean

Description: returns true if both expressions evaluate to the same value

If the Expression is an abstract class which has two sub-classes Sum and Product, then the

operation has to be tested for both parameter types Sum and Product.

Boundary Value Analysis

Boundary value analysis is another heuristic that can enhance the E&E of test cases designed

using equivalence partitioning. It is based on the observation that bugs often result from

incorrect handling of boundaries of equivalence partitions. This is not surprising, as the end

points of the boundary are often used in branching instructions etc. where the programmer can

make mistakes.

E.g. markCellAt(int x, int y) operation could contain code such as

if (x > 0 && x <= (W-1)) which involves boundaries of x’s equivalence partitions.

When doing boundary value analysis, values around the boundary of an equivalence partition

are tested. Typically, three values are chosen: one value from the boundary, one value just

below the boundary, and one value just above the boundary. The table below gives some

examples of boundary values.

Equivalence partition Some possible boundary values

[1-12] 0,1,2, 11,12,13

[MIN_INT, 0]

*MIN_INT is the minimum possible

integer value allowed by the

MIN_INT, MIN_INT+1, -1, 0 , 1


5

environment.

[any non-null String] Empty String, a String of maximum possible length

[prime numbers],

[“F”]

[“A”, “B”, “C”]

No specific boundary



[non-empty Stack] *we assume a fixed

size stack

Stack with: one element, two elements, no empty

spaces, only one empty space

Combining multiple inputs

Often, an SUT can take multiple data participants. Having selected test values for each data

participant (using equivalence partitioning, boundary value analysis, or some other technique),

how are they combined to create test cases? Consider the following scenario.

Operation to test:

calculateGrade(participation, projectScore, isAbsent, examScore)

Values to test (invalid values are underlined)

participation: 0, 1, 19, 20, 21, 22

projectScore: A, B, C, D, F

isAbsent: true, false

examScore: 0, 1, 69, 70, 71, 72

Given next are some techniques that can be used.

All combinations - This technique has a higher chance of discovering bugs. However, the

number of combinations can be too high to test. In the above example, there are 6x5x2x6=360

cases to be tested.

At least once – This technique, illustrated in the table below, aims to include every value at

least once.

Table 1. Test cases for calculateGrade (V1.0)

Case No participation projectScore isAbsent examScore Expected

1 0 A true 0 …

2 1 B false 1 …

3 19 C AVV 69 …

4 20 D AVV 70 …

5 21 F AVV 71 Err Msg

6 22 AVV AVV 72 Err Msg

AVV = Any Valid Value, Err Msg = Error Message


6

This technique uses one test case to verify multiple input values. For example, test case 1

verifies SUT for participation==0, projectScore==A, isAbsent==true, and examScore==0. However,

the expected result for test case 5 could be an error message, because of the invalid input data.

This means that it remains unknown whether the SUT works correctly for the projectScore==F

input as it is not being used by the other four test cases that do not produce an error message.

Furthermore, if the error message was due to participation==21 then it does not guarantee that

examScore==71 will also return the correct error message. This is why invalid input values

should be tested one at a time, and not combined with the testing of valid input values. Doing

this will result in nine test cases, as shown in the table below.

Table 2. Test cases for calculateGrade (V2.0)

Case No participation projectScore isAbsent examScore expected

1 0 A true 0 …

2 1 B false 1 …

3 19 C AVV 69 …

4 20 D AVV 70 …

5 AVV F AVV AVV …

6 21 AVV AVV AVV Err Msg

7 22 AVV AVV AVV Err Msg

8 AVV AVV AVV 71 Err Msg

9 AVV AVV AVV 72 Err Msg

Other links between parameters can increase the number of test cases further. For example,

assuming that an absent student can only have examScore==0, a link between isAbsent and

examScore is established. To cater for the hidden invalid case arising from this, a 10th test case is

added for which isAbsent==true and examScore!=0. In addition, test cases 3-9 should have

isAbsent==false so that the input remains valid.


7

Table 0.3. Test cases for calculateGrade (V3.0)

Case No participation projectScore isAbsent examScore Expected

1 0 A true 0 …

2 1 B false 1 …

3 19 C false 69 …

4 20 D false 70 …

5 AVV F false AVV …

6 21 AVV false AVV Err Msg

7 22 AVV false AVV Err Msg

8 AVV AVV false 71 Err Msg

9 AVV AVV false 72 Err Msg

10 AVV AVV true !=0 Err Msg

All-pairs – This technique creates test cases so that for any given pair of parameters, all

combinations between them are tested. It is based on the observations that a bug is rarely the

result of more than two interacting factors. The resulting number of test cases is lower than the

“all combinations” approach, but higher than the “at least once” approach. The technique for

creating such a set of test cases is beyond the scope of this handout.

Testing use cases

Use case testing is straightforward in principle: test cases are simply based on the use cases. It is

used for system testing (i.e. testing the system as a whole). For example, the main success

scenario can be one test case while each variation (due to extensions) can form another test

case. However, note that use cases do not specify the exact data entered into the system.

Instead, it might say something like “user enters his personal data into the system”. Therefore,

the tester has to choose data by considering equivalence partitions and boundary values. The

combinations of these could result in one use case producing many test cases. To increase E&E

of testing, high-priority use cases are given more attention. For example, a scripted approach

can be used to test high priority test cases, while an exploratory approach is used to test other

areas of concern that could emerge during testing.

Worked examples

[Q1] a) Explain the concept of exploratory testing using Minesweeper as an example.

b) Explain why exhaustive testing is not possible using the newGame operation (from Logic

class in the Minesweeper case study) as an example.

[A1] (a) When we test the Minesweeper by simply playing it in various ways, especially trying out

those that are likely to be buggy, that would be exploratory testing.

b) Consider this sequence of test cases:


8

Test case 1. Start Minesweeper. Activate newGame() and see if it works.

Test case 2. Start Minesweeper. Activate newGame(). Activate newGame() again and see

if it works.

Test case 3. Start Minesweeper. Activate newGame() three times consecutively and see if

it works.

…

Test case 267. Start Minesweeper. Activate newGame() 267 times consecutively and see

if it works.

Well, you get the idea. Exhaustive testing of newGame() is not possible.

[Q2] Assume students are given matriculation number according to the following format:

[Faculty Alphabet] [Gender Alphabet] [Serial Number] [Check Alphabet]

E.g. CF1234X

The valid value(s) for each part of the matriculation number is given below:

Faculty Alphabet:

• Single capital alphabet

• Only 'C' to 'G' are valid

Gender Alphabet:


• Either 'F' or 'M' only

Serial Number

• 4-digits number

• From 1000 to 9999 only

Check Alphabet


• Only 'K', 'H', 'S', 'X' and 'Z' are valid

Assume you are testing the operation isValidMatric(String matric):boolen. Identify equivalence

partitions and boundary values for the matriculation number.

[A2] String length: (less than 7 characters), (7 characters), (more than 7 characters)

For those with 7 characters,

[Faculty Alphabet]: (‘C’, ‘G’), (‘c’, ‘g’), (any other character)

[Gender Alphabet]: (‘F’, ‘M’), (‘f’, ‘m’), (any other character)

[Serial Number]: (1000-9999), (0000-0999), (any other 4- characters string)

[Check Alphabet]: ('K', 'H', 'S', ‘X’, 'Z'), ('k', 'h', ’s’, ‘x’, 'z'), (any other character)


9

[Q3] Given below is the overview of the method dispatch(Resource, Task), from an emergency management system (e.g. a system used by those who handle emergency calls from the public about incidents such as fires, possible burglaries, domestic disturbances, etc.). A task might need multiple resources of multiple types. For example, the task ‘fire at Clementi MRT’ might need two fire engines and one ambulance.

dispatch(Resource r, Task t):void

Overview: This method dispatches the Resource r to the Task t. Since this can dispatch

only one resource, it needs to be used multiple times should the task need multiple

resources.

Imagine you are designing test cases to test the method dispatch(Resource,Task). Taking into

account equivalence partitions and boundary values, which different inputs will you combine to

test the method?

[A3] Test input for r Test input for t

A resource required by the task

A resource not required by the task

A resource already dispatched for

another task

null

A fully dispatched task

A task requiring one more resource

A task with no resource dispatched

Considering the resource types required

A task requiring only one type of

resources

A task requiring multiple types of

resource

null

[Q4] Given below is an operation description taken from a restaurant booking system. Use

equivalence partitions and boundary value analysis technique to design a set of test cases for it.

boolean transferTable (Booking b, Table t)

Description: Used to transfer a Booking b to Table t, if t has enough room.

Preconditions: t has room for b , b.getTable() != t

Postconditions: b.getTable() == t

[A4] Equivalence partitions

Booking:

Invalid: null, not null and b.getTable==t

Valid: not null and b.getTable != t

Table:

Invalid: null, not vacant, vacant but doesn’t have enough room,


10

Valid: vacant and has enough room.

Boundary values:

Booking:

Invalid: null, not null and b.getTable==t

Valid:not null and b.getTable != t

Table:

Invalid: null, not vacant, (booking size == table size + 1)

Valid: (booking size == table size), (booking size == table size-1)

Test cases:

Test case Booking Table

1 null Any valid

2 not null and b.getTable==t Any valid

3 Any valid null

4 Any valid not vacant

5 Any valid (booking size == table size + 1)

7 Any valid (booking size == table size)

8 Any valid (booking size == table size-1)

Note: We can use Bookings of different sizes for different test cases so that we increase the

chance of finding bugs. If there is a minimum and maximum booking size, we should include

them in those test cases.

[Q5] Assume you are testing the add(Item) method specified below.

add(Item):voidcontains(Item):booleancount():int

ItemList

Assume i to be the Item being added.

Preconditions:

i != null [throws InvalidItemException if i == null ]

contains(i) == false [throws DuplicateItemException if contains(i) == true]

count() < 10 [throws ListFullException if count() == 10]

Postconditions:

contains(i) == true;

new count() == old count()+1


11

Invariants: (an “invariant” is true before and after the method invocation).

0 <= count() <= 10

(a) What are the equivalence partitions relevant to testing the add(Item) method?

(b) What are the boundary and non-boundary values you will use to test the add(Item) method?

(c) Design a set of test cases to test the add(Item) method by considering the equivalence

partitions and boundary values from your answers to (a) and (b) above.

[A5] (a)

i: i != null, i == null

list: contains(i)==true, contains(i)==false, count() < 10, count() == 10

list == null should NOT be considered.

(b) list: count()==0, count()==9, count()==10; count()== [1|2|3|4|5|6|7|8] (1 preferred)

(c)

count == 0

count == 1

count == 9

!contains

count == 10

contains

i==null

i!=null

[Q6] Use equivalence partitions and boundary values to choose test inputs for testing the setWife

operation in the Man class.

Man Woman0..1 0..1

husband wife

setWife(Woman):voidgetWife():Woman

setHusband(Man):voidgetHusband():Man


12

[A6]

Man[3 partitions]

married

single

To the same woman

To a different woman

Woman[4 partitions]

married

single

To the same man

To a different man

null

Partitioning ‘married’ as ‘to same woman’ and ‘to different woman’ seems redundant at first.

Arguments for having it:

The behavior (e.g. the error message shown) may be different in those two situations.

The ‘to same woman’ partition has a risk of misunderstanding between developer and

user. For example, the developer might think it is OK to ignore the request while the

users might expect to see an error message.

[Q7]

b) Identify a set of equivalence partitions for testing isValidDate(String date) method. The method

checks if the parameter date is a day that falls in the period 1880/01/01 to 2030/12/31 (both

inclusive). The date is given in the format yyyy/mm/dd.

[A7] Initial partitions: [null] [10 characters long][shorter than 10 characters][longer than 10

characters]

For 10-character strings:

c1-c4: [not an integer] [less than 1880] [1880-2030 excluding leap years][ leap years

within 1880-2030 period][2030-9999]

c5: [‘/’][not ‘/’]

c6-c7: [not an integer][less than 1][2][31-day months: 1,3,5, 7,8, 10,12][30-day

months: 4,6,9,11] [13-99]

c8: [‘/’][ not ‘/’]

c9-c10: [not an integer][less than 1][1-28][29][30][31][more than 31]

In practice, we often use ‘trusted’ library functions (e.g. those that come with the Java JDK or

.NET framework) to convert strings into dates. In such cases, our testing need not be as

thorough as those suggested by the above analysis. However, this kind of thorough testing is

required if you are the person implementing such a trusted component.


13

[Q8] Consider the following operation that activates an alarm when the landing gear (i.e. wheels

structure) of an airplane should be deployed but has not been deployed.

isAlarmToBeSounded(timeSinceTakeOff,

timeToLand,

altitude,

visibility,

isGearAlreadyDeployed):boolean

Here is the logic for the operation. Landing gear must be deployed whenever the plane is within

2 minutes before landing or after takeoff, or within 2000 feet from the ground. If visibility is less

than 1000 feet, then the landing gear must be deployed whenever the plane is within 3 minutes

from landing or lower than 2500 feet. The operation should return true if the landing gear is not

deployed (i.e. isGearAlreadyDeployed ==false) on meeting a deployment condition. Assume that

the smallest unit of measurement for time is 1s and for distance it is 1 feet. Also, assume that

invalid input such as negative values will not be produced by the sensors. takeoff_max,

landing_max, altitude_max, visibility_max are the maximum values allowed by the instruments.

(a) List the equivalence partitions of the conditions that can lead to a decision about whether

the alarm should be sounded.

(b) List the boundary and non-boundary values you will test, using no more than one non-

boundary value per equivalence partition.

(c) Typically, for a critical system such as the “landing gear alarm system”, all combinations of

equivalence partitions must be checked. Ignoring that requirement for a moment, design a

minimal set of test cases that includes each equivalence partition at least once. Use only

boundary values.

(d) Now, you have been told that the correct functioning of the alarm is critically important if

the plane is within 2 minutes of landing no matter what the other conditions are. How would

you modify the test cases? Use only boundary values.

[A8] (a) Equivalence partitions:

timeSinceTakeOff: [ 0s .. 2m] [2m1s .. takeoff_max]

timeToLand : [land_max.. 3m1s][3m .. 2m1s] [2m .. 0s]

altitude: [0 .. 2000], [2001 .. 2500], [2501 .. altitude_max]

visibility: [0 .. 1000], [1001 .. visibility_max]

isGearAlreadyDeployed: [true] [false]

(b) take all values mentioned in (a), and one non-boundary value for each partition. For

example,

timeSinceTakeOff: 0s, 1m, 2m, 2m1s, 3m, takeoff_max

(c) To test every partition at least once, we need only three test cases.


14

timeSinceTakeOff timeToLand altitude visibility isGearAlreadyDeployed

0s land_max 0 0 [true]

2m1s 3m 2001 1001 [false]

-any value- 2m 2501 -any value- -any value-

[Not required by the question] To test every boundary value at least once, we need six test

cases.

timeSinceTakeOff timeToLand altitude visibility isGearAlreadyDeployed

0s land_max 0 0 [true]

2m0s 3m1s 2000 1000 [false]

2m1s 3m 2001 1001 -any value-

takeoff_max 2m1s 2500 visibility_max -any value-

-any value- 2m 2501 -any value- -any value-

-any value- 0s altitude_max -any value- -any value-

(d) Values 2m and 0s for the timeToLand should be combined with every possible combination

of the other four inputs.

For TimeToLand ==2m -> 4x6x4x2 -> 192 test cases

For TimeToLand == 0s -> 4x6x4x2 -> 192 test cases

For TimeToLand == land_max, 3m1s, 3m, 2m1s -> 4 test cases

In total we need 192+192+4=388 test cases.

We can also test a non-boundary value from the critical equivalence partition (e.g. timeToLand

==1m), in which case, we will have an additional 192 test cases.


1

PROJECT MANAGEMENT

[ L2P3]

Your Own Private Time Machine: Introduction to Revision Control

Revision control (individual) Revision control system (RCS) is an indispensable tool in the software engineer’s tool box.

The following introduction to revision control was adapted (with minor modifications) from the

excellent online book Mercurial: The Definitive Guide by Bryan O'Sullivan

Why revision control?

Revision control is the process of managing multiple versions of a piece of information. In

its simplest form, this is something that many people do by hand: every time you modify a

file, save it under a new name that contains a number, each one higher than the number of

the preceding version.

Manually managing multiple versions of even a single file is an error-prone task, though, so

software tools to help automate this process have long been available. The earliest

automated revision control tools were intended to help a single user to manage revisions of a

single file. Over the past few decades, the scope of revision control tools has expanded

greatly; they now manage multiple files, and help multiple people to work together. The best

modern revision control tools have no problem coping with thousands of people working

together on projects that consist of hundreds of thousands of files.

Why use revision control?

There are a number of reasons why you or your team might want to use an automated

revision control tool for a project. It will track the history and evolution of your project, so

you don't have to. For every change, you'll have a log of who made it; why they made it; when

they made it; and what the change was.

When you're working with other people, revision control software makes it easier for you

to collaborate. For example, when people more or less simultaneously make potentially

incompatible changes, the software will help you to identify and resolve those conflicts.

It can help you to recover from mistakes. If you make a change that later turns out to be

an error, you can revert to an earlier version of one or more files. In fact, a really good

revision control tool will even help you to efficiently figure out exactly when a problem was

introduced.

It will help you to work simultaneously on, and manage the drift between, multiple

versions of your project. Most of these reasons are equally valid, at least in theory, whether

you're working on a project by yourself, or with a hundred other people.

The many names of revision control

Revision control is a diverse field, so much so that it is referred to by many names and

acronyms. Here are a few of the more common variations you'll encounter:

Revision control system (RCS)

http://hgbook.red-bean.com/


2

Software configuration management (SCM), or configuration management Source code management Source code control, or source control Version control system (VCS)

Some people claim that these terms actually have different meanings, but in practice they

overlap so much that there's no agreed or even useful way to tease them apart.

[End of extract]

It follows from the explanation above that using an RCS frees us from having to keep copies of

all the past versions on our hard disk. It reduces problems such as ‘updating the wrong version’,

‘overwriting latest changes with an older copy’, ‘multiple developers overwriting each others’

changes’ etc. If the RCS spans multiple machines, it reduces the risk of ‘losing everything’ in case

of a hard disk crash.

Glossary of Basic RCS (Git specific) concepts (individual use)

Repository (Repo): The database where the files and historical data are stored, including the author of the changes and the summary of each change. Commonly called repo for short.

Init: Initialize a repository in a directory so that the RCS can start version tracking the files in that directory.

Working directory: The local directory of your files. Ignore: Requesting the RCS not to track the versions of a file. This is usually used for

binary files such as executables and generates files such as logs. Stage: Take a copy of a file in preparation to save its current version in the version

history. Add: Request the RCS to start tracking the versions of a file. Commit (verb) : Storing the staged files in the permanent version history. Commit (noun)/ Revision/ Changeset : The object containing the copies of the staged

files at the point of committing, and other meta data. Diff: The differences between two specific versions. Remove: Asking the RCS to stop tracking a file. Tag (noun): A label added to a commit for easy reference. Tag (verb): Adding a tag to a commit Checkout commit: Update the working directory to match a commit Stash: Store not-yet-ready-to-be-committed changes in a temporary location, to be

retrieved later.

Revision control (team) When using an RCS in a team setting, in addition to the local repository on one’s machine, there

is also a remote repository (‘remote repo’ for short) that other team members can access

remotely.

Some concepts related to remote repos:

Clone: Creates a copy of a repo. For example, a local repo can be created by cloning a remote repo.

Push: Uploads the revision history to another repo. Typically, only the commits previously not pushed will be uploaded.

Pull: Downloads the revision history from another repo. Typically, only the commits previously not pulled will be downloaded.


3

There are two models that can be followed when using an RCS in a team setting: the centralized

model and the distributed model.

1. Centralized RCS (CRCS for short): In this model, there is a central remote repo shared by the team. Team members download (‘pull’) and upload (‘push’) changes between their own local repositories and the central repository. Older RCS tools such as CVS and SVN support only this model. Note that these older RCS do not support the notion of a local repo. Instead, they force users to do all the versioning with the remote repo. This situation is illustrated in Figure 15.

Team’s Central Repo

Team Leader

Member C

Member A

Member B

Figure 15. The centralized RCS approach without any local repos (e.g., CVS, SVN)

Member B’s Local

Repo

Member A’s Local

Repo

Member A’s

remote Repo

Member B’s

remote Repo

Member B

Member A

Figure 16. The decentralized RCS approach

2. Distributed RCS (DRCS for short, also known as Decentralized RCS): In this model, there can be multiple remote repos and pulling and pushing can be done among them in arbitrary ways. The workflow can vary differently from team to team. For example, every team member can have his/her own remote repository in addition to their own local repository, as shown in Figure 16. Mercurial, Git, and Bazaar are some prominent RCS tools that support distributed RCS.

Branching and merging are two terms that are often used in team-based RCS usage.

Conceptually, branching is the process of evolving multiple versions of the software in parallel.

For example, one team member can create a new branch and add an experimental feature to it

while the rest of the team keeps working on the master. ‘Master’ refers to the main line of

development while ‘branches’ are other variants of the software being developed in parallel. A


4

branch usually contains multiple commits. Once the experimental feature is stable, that branch

can be merged to the master. Branching and merging is illustrated in Figure 17.

Master

Branch A

Branch B

merge

merge

Figure 17. Branching and merging

When using the DRCS approach, there are multiple repositories (local and remote), each

containing different parallel versions.

For simplicity, it is also possible to use DRCS in centralized fashion, as shown below, before

moving to a more decentralized workflow.

Ravi’s Local Repo

Adam’s Local Repo

Central remote

Repo

Jean’s Local Repo

Adam

Ravi

Jean

Here is an example sequence of actions when using a DRCS in centralized fashion.

1. Adam writes the first bit of code for the project. Initializes a local repo, and commits the code.

2. Adam creates an account in a remote repo service (e.g. GitHub) and pushes his code to the remote repo. Adam’s code is now in the remote repo.

3. Ravi clones Adam’s remote repo to his computer. As a result, Ravi now has a local repo with Adam’s code in it.

4. Jean too clones the repo and gets Adams code. 5. Ravi writes some more code. Commits to the local repo. Pushes it to the remote repo.


5

6. Adam pulls from the remote repo. As a result, Ravi’s changes are downloaded to Adam’s local repo.

7. Adam runs the update command. As a result, his working copy gets updated with Ravis additions.

[Steps 5,6,7] can be repeated to share one person’s code with the rest of the team.

The above approach (i.e. using DRCS in a centralized way) is recommended for beginners

because it is simple. Having attained some familiarity using remote repos, one can then switch

to a more flexible workflow. There are many DRCS workflows around. Given below is one

example. This workflow can be characterized by ‘one branch per component’.

Master

Branch UI

Branch Logic

merge master to component branch merge component branch to trunk

1

2

3

4

5

6

7

This workflow comprises several branches, one for each major component (it can also be ‘one

for each developer’). The master branch contains the stable code. The tip (i.e. the most recent

commit) of the master represents the latest working version of the product. The component

branches evolve in parallel. Their tip need not be stable. A component branch can be merged

from the master whenever there is new code in the master. A component branch can merge to

the master whenever it has stable code it wants to contribute to the master.

Here is an example sequence of actions in this workflow. Let us assume Adam is in charge of the

Logic branch. Item numbers correspond to the numbers shown in the revision tree diagram

above.

1. Adam adds a new method to the Logic component. He commits to the Logic branch several times during this work.

2. Adam thinks he is ready to share this new method with others. He pulls the master from the remote repo to see if there are new commits in the master. Adam sees some new commits in the master he just pulled. He merges the master to his branch.

3. He runs the test cases and finds the code that came from the master has broken some tests in his branch. He fixes them and does another commit. At this point, his branch contains the latest code from the master and the new method he wrote. The code is stable.

4. He merges his branch with master and pushes the latest master to the remote repo. 5. Adam continues the branch. This time, he fixes a bug in the logic component that was

reported recently by Jean. He commits the fix. 6. Before he could merge the fix to the master, Jean (who is working on the UI branch)

contributes some code to the master. Adam merges the master to his branch to get the new code.

7. Adam realizes his fix no longer works because of the code pushed by Jean. He continues to fix the bug with further commits.


1

[ L8P1]

Herding Cats: Making Teams Work

Work Breakdown Structure (WBS) Before starting project work, it is useful to divide the total work into smaller, well-defined units.

Relatively complex tasks can be further split into subtasks. Commonly, the high-level tasks that

represent the overall software development stages are first defined, followed by a detailed

version containing information about subtasks. A structure that depicts the above information

about tasks and their details in terms of subtasks is known as a work breakdown structure. In

complex projects a WBS can also include prerequisite tasks and effort estimates for each task.

For the Minesweeper example, the high level tasks for a single iteration could look like the

following:

Task Id Task Estimated Effort Prerequisite

Task

A Analysis 1 man day -

B Design 2 man day A

C Implementation 4.5 man day B

D Testing 1 man day C

E Planning for next version 1 man day D

The effort is traditionally measured in man hour/day/month i.e. work that can be done by one

person in one hour/day/month. The Task Id is a label for easy reference to a task. Simple

labeling is suitable for a small project, while a more informative labeling system can be adopted

for bigger projects. The high level tasks structure can be further refined into subtasks. For

example, the above table can be further refined to:

Task Id Task Estimated Effort Prerequisite

Task

A High level design 1 man day -

B

Detail design

1. User Interface

2. Game Logic

3. Persistency Support

2 man day

0.5 man day

1 man day

0.5 man day

A

C

Implementation

1. User Interface

2. Game Logic

3. Persistency Support

4.5 man day

1.5 man day

2 man day

1 man day

B.1

B.2

B.3

D System Testing 1 man day C

E Planning for next version 1 man day D


2

All tasks should be well-defined. In particular, it should be clear as to when the task will be

considered “done”. E.g.

not good: more coding | better: implement component X

not good: do research on UI testing | better: find a suitable tool for testing the UI

Scheduling and tracking Milestones

In projects, a milestone is the end of a stage which indicates a significant progress. For example,

each intermediate product release is a milestone. Take into account dependencies and priorities

when deciding on the features to be delivered at a certain milestone.

In some projects, it is not practical to have a very detailed plan for the whole project due to the

uncertainty and unavailability of required information. In such cases, consider a high-level plan

for the whole project and a detailed plan for the next few weeks.

Here is an example: [Note that the scope of the task is defined by the product version that is to

be released at the next milestone. i.e. TextUi means “Text UI for V0.1” ]

Project schedule (contd)

Day 1 Day 2-5 Day 6 Day 7

Jean Write MSLogic skeleton MSLogic

Test

ing

bef

ore

rel

ease

Fin

e-tu

nin

g an

d/o

r p

lan

nin

g

the

nex

t ve

rsio

n

James Specify test case format

for automated tester

Automated tester

Kumar Circulate meeting minutes TextUI

Chunling Update project manual (in

point form)

test cases

390

[Ch.8]

Buffers

It is important to include buffers (i.e. time set aside to absorb any unforeseen delays). Do not

inflate tasks to create hidden buffers. Have them explicitly set aside.

Task X Task Y

Task X Task YBuffer Buffer

Estimates

Schedule (using inflated estimates)

Schedule (using buffers)

Task X Task Y

Bug/Issue trackers

Issue trackers (sometimes called bug trackers) are commonly used to track task assignment and

progress. E.g. Bugzilla. Most online project management software such as GoogleCode, GitHub,

SourceForge and BitBucket come with an integrated issue tracker. The following is a screenshot

from the Jira Issue tracker software.


3

Issue trackers can also be used as task trackers, i.e. to define tasks to be done, to track who is

doing what, and the status of each task.

More elaborate tools for scheduling and tracking projects include PERT chars and Gantt charts.

PERT (Program Evaluation Review Technique) charts

PERT chart uses a graphical technique to show the order/sequence of tasks. It is based on a

simple idea of drawing a directed graph in which:

Node or vertex captures the effort estimation of a task, and Arrow depicts the precedence between tasks

Here is a PERT chart for the Minesweeper example:

With the above chart, it is much easier to determine the order of a particular task. For example,

observe that the Final Testing phase cannot begin until all coding of individual subsystems have

A. High

level

design

1 md

B.1 UI Design

0.5 md

E.

Planning

1md

B.2 Game

Logic Design

1 md

B.3

Persistency

Support

Design

0.5 md

C.1 UI Coding

1.5 md

C.2 Game

Logic Coding

2 md

C.3

Persistency

Support

Design

1 md

D. Final

Testing

1 md


4

been completed. Also, is it easy to determine the concurrent tasks that can proceed upon

completion of the prerequisite task(s). For example, the various subsystem designs can start

independently once the High level design is completed. The depiction of parallel tasks facilitates

the optimal utilization of manpower.

A closely related issue is estimating the shortest possible completion time. From the above PERT

chart, there is a path (indicated by the shaded boxes) from start to end that determines the

longest possible completion time. This path is given the name Critical Path to emphasize that it

is critical to ensure tasks on this path are completed on time. Any delay would directly affect the

delivery time for the project.

Gantt charts

Despite the usefulness of PERT charts, there is a shortfall. As each node only shows the elapsed

time of a task, it is hard to see the actual start and end time of the node. A Gantt chart is a

graphical tool designed specifically for this purpose. The basic idea is to have a 2-D bar-chart,

drawn as time vs tasks to be performed (represented by horizontal bars). Using minesweeper

as an example:

In the chart, a solid bar represents the main task, which is generally composed of a number of

subtasks, shown as grey bars. The diamond shape indicates an important deadline or

deliverable or a milestone.

Here are some of points to note about scheduling and tracking:

Manpower allocation is not just arithmetic: Only a simplistic view has been presented so far, which might lead one to think that adding manpower to a task would reduce the time needed by a proportional amount. However, Brook’s law, by Frederick Brooks, states that: “Adding more manpower to an already late project makes it even later”. It may sound counter-intuitive, but the reasons are simple: a newly assigned team member requires time to understand the existing system before he/she can be productive. Secondly, adding manpower also increases the communication overhead. Do not forget that a typical task


5

might not be easily split into totally independent subunits. As such, frequent synchronization between team members can take a significant amount of time. Take this into consideration when assigning more than one person to a task, and adjust the estimated time accordingly.

Progress chart is not progress itself: Having a complex chart does not equate to accomplishing a lot of tasks. The various charts are just tools to monitor project progress, and real work must still be done on the tasks to push the project forward. In the same vein, do not spend too much time on chart drawing.

Use reasonable estimates: Assigning the minimum possible time to each of the tasks may make the project seem well under control, but it will do the team more harm than good if the schedule has to be adjusted frequently due to overly optimistic estimations.

Team structures

Given below are three commonly used team structures in software development. Irrespective of

the team structure, it is a good practice to assign roles and responsibilities to different team

members so that someone is clearly in charge of each aspect of the project. In comparison, the

‘everybody is responsible for everything’ approach can result in more chaos and hence slower

progress.

385

Team structures

The team structure is the basis for communication (e.g., who to ask when you need more info), responsibility (e.g., who is responsible for what aspect), authority (e.g., who reports to whom) etc. The team structure can take many forms. Here are some examples.

egoless team chief-programmer strict-hierarchy

[Ch.8]

Egoless team

In this structure, every team member is equal in terms of responsibility and accountability.

When any decision is required, consensus must be reached. This team structure is also known

as a democratic team structure. This team structure usually finds a good solution to a relatively

hard problem as all team members contribute ideas.

However, the democratic nature of the team structure bears a higher risk of falling apart due to

the absence of an authority figure to manage the team and resolve conflicts.

Chief programmer team

Frederick Brooks proposed that software engineers learn from the medical surgical team in an

operating room. In such a team, there is always a chief surgeon, assisted by experts in other

areas. Similarly, in a chief programmer team structure, there is a single authoritative figure, the

chief programmer. Major decisions, e.g. system architecture, are made solely by him/her and

obeyed by all other team members. The chief programmer directs and coordinates the effort of

other team members. When necessary, the chief will be assisted by domain specialists e.g.

business specialists, database expert, network technology expert, etc. This allows individual

group members to concentrate solely on the areas where they have sound knowledge and

expertise.

The success of such a team structure relies heavily on the chief programmer. Not only must he

be a superb technical hand, he also needs good managerial skills. Under a suitably qualified

leader, such a team structure is known to produce successful work. .


6

Strict hierarchy team In the opposite extreme of an egoless team, a strict hierarchy team has a strictly defined

organization among the team members, reminiscent of the military or bureaucratic government.

Each team member only works on his assigned tasks and reports to a single “boss”.

In a large, resource-intensive, complex project, this could be a good team structure to reduce

communication overhead.


1

[ L7P3]

How to Avoid a Big Bang: Integrating Software Components

Integration

Timing and frequency: ‘Late and one time’ vs ‘early and frequent’

Integrating parts written by different team members is inevitable in multi-person projects. It is

also one of the most troublesome tasks and it rarely goes smoothly.

In terms of timing and frequency, there are two general approaches to integration:

1. Late and one-time: In an extreme case of this approach, developers wait till all

components are completed and integrate all finished components just before the

release. This approach is not recommended because integration often causes many

component incompatibilities (due to previous miscommunications and

misunderstandings) to surface which can lead to delivery delays: Late integration

incompatibilities found major rework required cannot meet the delivery date.

2. Early and frequent: The other approach is to integrate early and evolve in parallel in

small steps, re-integrating frequently. For example, a working skeleton9 can be written

first (i.e. it compiles and runs but does not produce any useful output). This can be done

by one developer, possibly the one in charge of integration. After that, all developers can

flesh out the skeleton in parallel, adding one feature at a time. After each feature is done,

simply integrate the new code to the main system.

Whether using frequent integration or one-time late integration, there is still a need to decide

the order in which components are to be integrated. There are several approaches to doing this,

as explained next.

The order of integration: Big bang vs incremental

Big-bang integration In the big-bang integration approach, all components are integrated at the same time. This

approach is not recommended since it will uncover too many problems at the same time which

could make debugging and bug-fixing more complex than when problems are uncovered more

gradually.

Incremental integration

For non-trivial integration efforts, the following three incremental integration approaches are

more suitable.

Top-down integration: In top-down integration, higher-level components are integrated before bringing in the lower-level components. One advantage of this approach is that higher-level problems can be discovered early. One disadvantage is that this requires the use of dummy or skeletal components (i.e. stubs) in place of lower level components until the real lower-level components are integrated to the system. Otherwise, higher-level components cannot function as they depend on lower level ones.

9 Some call it a ‘walking skeleton’


2

Bottom-up integration: This is the reverse of top-down integration. Advantages and disadvantages are simply the reverse of those of the top-down approach.

Sandwich integration: This is a mix of the top-down and the bottom-up approaches. The idea is to do both top-down and bottom-up so as to “meet up” in the middle.

Build automation In a non-trivial project, building a product from source code can be a complex multi-step

process. For example, it can include steps such as to pull code from the revision control system,

compile, link, run automated tests, automatically update release documents (e.g. build number),

package into a distributable, push to repo, deploy to a server, delete temporary files created

during building/testing, email developers of the new build, and so on. Furthermore, this build

process can be done ‘on demand’, it can be scheduled (e.g. every day at midnight) or it can be

triggered by various events (e.g. triggered by a code push to the revision control system).

Some of these build steps such as to compile, link and package are already automated in most

modern IDEs. For example, several steps happen automatically when the ‘build’ button of the

IDE is clicked. Some IDEs even allow customization to this build process to some extent.

However, most big projects use specialized build tools to automate complex build processes.

GNU Make (http://www.gnu.org/software/make/) and Apache Ant (http://ant.apache.org/)

are two build tools that used to be very popular about a decade ago and still being used. Two

popular build tools at the moment are Maven (http://maven.apache.org/) and Gradle

(https://gradle.org/)

Dependency Management Modern software projects often depend on third party libraries that evolve constantly. That

means developers need to download the correct version of the required libraries and update

them regularly. Dependency Management tools can automate that aspects of a project. Maven

and Gradle, in addition to managing the build process, are dependency management tools too.

Continuous Integration An extreme application of build automation is called continuous integration (CI) in which

integration, building, and testing happens automatically after each code change. Travis

(https://travis-ci.org/) and Jenkins (http://jenkins-ci.org) are examples of popular CI tools.

Worked examples

[Q1] Consider the architecture given below. Describe the order in which components will be integrated with one another if the following integration strategies were adopted.

(a) big-bang (b) top-down (c) bottom-up (d) sandwich

Note that dashed arrows show dependencies (e.g. A depend on B, C, D and therefore, higher-level than B, C and D).

http://www.gnu.org/software/make/

http://ant.apache.org/

http://maven.apache.org/

https://gradle.org/

https://travis-ci.org/

http://jenkins-ci.org/


3

[CS2103]

A

B C D

HFE IG

K L MJ

[A1]

(a) Big-bang approach: integrate A-M in one shot. (b) Top-down approach and (c) bottom-up approach [side by side comparison]

27

A

Integrate

A,B,C,D

Integrate

A..I

Integrate

A..M

26

C

G

Integrate

A..M

Integrate

B, E, F, J

Integrate

C,G,K,L

Integrate

D,H,M,I

Integrate

H, M

FIntegrate

E, JIIntegrate

G, K, L

K L MJ

(d) Sandwich approach

28

A

Integrate

A,B,C,D

Integrate

H, M

FIntegrate

E, J

IIntegrate

G, K, L

K L MJ

Integrate

A..M

top-down

bottom-up


4

[Q2] Give two arguments in support and two arguments against the following statement.

Because there is no external client, it is OK to use big bang integration for the CS2103 module

project.

[A2] Arguments for:

It is relatively simple; even big-bang can succeed.

Project duration is short; there is not enough time to integrate in steps.

The system is non-critical, non-production (demo only); the cost of integration issues is

relatively small.

Arguments against:

Inexperienced developers; big-bang more likely to fail

Too many problems may be discovered too late. Submission deadline (fixed) can be

missed.

Team members have not worked together before; increases the probability of

integration problems.

[Q3] Suggest an integration strategy for the system represented by following diagram. You need not follow a strict top-down, bottom-up, sandwich, or big bang approach. Dashed arrows represent dependencies between classes. Also take into account the following facts in your test strategy.

HospitalUI will be developed early, so as to get customer feedback early. HospitalFacade shields the UI from complexities of the application layer. It simply

redirects the method calls received to the appropriate classes below IO_Helper is to be reused from an earlier project, with minor modifications Development of OutPatient component has been outsourced, and the delivery is not

expected until the 2nd half of the project.

Hospital UI

HospitalFacade

PatientMgr MedicineMgr RecordMgr

IO_Helper<<interface>>

PatientInterfaceTypeA TypeB

TypeCOutPatient

[A3] There can be many acceptable answers to this question. But any good strategy should consider

at least some of the below.

Since HospitalUI will be developed early, it’s OK to integrate it early, using stubs, rather

than wait for the rest of the system to finish. (i.e. a top-down integration is suitable for

HospitalUI)


5

Because HospitalFacade is unlikely to have a lot of business logic, it may not be worth

to write stubs to test it (i.e. a bottom-up integration is better for HospitalFacade).

Since IO_Helper is to be reused from an earlier project, we can finish it early. This is

especially suitable since there are many classes that use it. Therefore IO_Helper can be

integrated with the dependent classes in bottom-up fashion.

Since OutPatient class may be delayed, we may have to integrate PatientMgr using a

stub.

TypeA, TypeB, and TypeC seem to be tightly coupled. It may be a good idea to test them

together.

Given below is one possible integration test strategy. Relative positioning also indicates a rough

timeline.

Hospital_UI

All except

Hospital_UI

PatientMgr

IO_Helper

MedicineMgr, IO_Helper

TypeA, TypeB, TypeC

RecordMgr

TypeA, TypeB, TypeC

IO_Helper

IO_Helper

TypeA

TypeB

TypeC

OutPatient

All

time

-- End of handout --


1

[L9P1]

The M in RTFM: When Code Is Not Enough

Writing developer documentation Developer-to-developer documentation can be in one of two forms:

1. Documentation for developer-as-user: Software components are written by developers and used by other developers. Therefore, there is a need to document as to how the components are to be used. API documents form the bulk of this category.

2. Documentation for developer-as-maintainer: There is a need to document how a system or a component is designed, implemented and tested so that other developers can maintain and evolve the code.

Writing documentation of the first kind is easier because a component with a well-defined API

exposes functionality in small-sized, independent and easy-to-use chunks. Examples of such

documentation can be found in the Java API (http://download.oracle.com/javase/8/docs/api/).

Writing documentation of the second type is harder because of the need to explain complex

internal details of a non-trivial system. Given below are some points to note when writing the

second type of documentation.

Go top-down, not bottom-up

When writing project documents, a top-down breadth-first explanation is easier to understand

than a bottom-up one. To explain a system called SystemFoo with two sub-systems, front-end

and back-end, start by describing the system at the highest level of abstraction, and

progressively drill down to lower level details. An outline for such a description is given below.

[First, explain what the system is, in a black-box fashion (no internal details, only the external

view).]

SystemFoo is a ....

[Next, explain the high-level architecture of SystemFoo, referring to its major components only.]

SystemFoo consists of two major components: front-end and back-end.

The job of front-end is to ... while the job of back-end is to ...

And this is how front-end and back-end work together ...

[Now we can drill down to front-end's details.]

front-end consists of three major components: A, B, C

A's job is to ... B's job is to... C's job is to...

And this is how the three components work together ...

[At this point, further drill down the internal workings of each component. A reader who is not

interested in knowing nitty-gritty details can skip ahead to the section on back-end.]

...

[At this point drill down details of the back-end.]

...

The main advantage of this approach is that the document is structured like an upside down

tree (root at the top) and the reader can travel down a path he is interested in until he reaches

the component he has to work in, without having to read the entire document or understand the

whole system.

http://download.oracle.com/javase/8/docs/api/


2

'Accurate and complete' is not enough

Documentation that is “accurate and complete” is not going to be very effective. In addition to

accuracy and completeness, a document should be easy and pleasant to read. That is to say, it is

not enough to be comprehensive, it should also be comprehensible. The following are some tips on

writing effective documentation.

Use plenty of diagrams: It is not enough to explain something in words; complement it with visual illustrations (e.g. a UML diagram).

Use plenty of examples: When explaining algorithms, show a running example to illustrate each step of the algorithm, in parallel to worded explanations.

Use simple and direct explanations: Convoluted explanations and fancy words will annoy readers. Avoid long sentences.

Get rid of statements that do not add value. For example, 'We made sure our system works perfectly' (who didn't?), 'Component X has its own responsibilities' (of course it has!).

Do not duplicate text chunks

When describing several similar algorithms/designs/APIs, etc., do not simply duplicate large

chunks of text. Instead, describe the similarity in one place and emphasize only the differences

in other places. It is very annoying to see pages and pages of similar text without any indication

as to how they differ from each other.

Make it as short as possible

Aim for 'just enough' documentation. The readers are developers who will eventually read the

code. The documentation complements the code and provides just enough guidance to get

started. Anything that is already clear in the code need not be described in words. Instead, focus

on providing higher level information that is not readily visible in the code or comments.

Explain things, not list diagrams

It is not a good idea to have separate sections for each type of artifact, such as 'use cases',

'sequence diagrams', 'activity diagrams', etc. Such a structure, coupled with the blatant inclusion

of diagrams without justifying their need, indicates a failure to understand the purpose of

documentation. Include diagrams when they are needed to explain the system. If it is a must to

provide a comprehensive collection of diagrams, include them in the appendix as a reference.


1

[L9P3]

Gems of wisdom: software development principles

Principles This handout covers some general Software engineering ‘principles’.

Separation of concerns

In most introductory programming courses, students learn about the idea of modularity, which

is to split a program into smaller, more manageable modules. This is actually the first step

towards a more general principle known as separation of concerns. The term separation of

concerns was coined by Edsger W. Dijkstra. A “concern” can be taken as a single feature or a

single functionality. This principle basically states that a program should be separated into

modules, each tackling distinct concerns. This reduces functional overlaps and also limits the

ripple effect when changes are introduced to a specific part of the system.

This principle can be applied at the class level, as well as on higher levels. For example, the n-

tier architecture utilizes this principle. Each layer in the architecture has a well-defined

functionality that has no functional overlap with each other. Clearly, a correct application of this

principle should lead to higher cohesion and lower coupling.

Law of Demeter

Another famous principle goes by the name Law of Demeter (two other names: Principle of Least

Knowledge; Don’t talk to strangers). This principle aims to lower coupling by restricting the

interaction between objects in the following ways:

An object should have limited knowledge of another object. An object should only interact with objects that are closely related to it.

In other words, ‘only talk to your immediate friends and don’t talk to strangers’. More concretely,

a method m of an object O should invoke only the methods of the following kinds of objects:

The object O itself

Objects passed as parameters of m

Objects created/instantiated in m

Objects from the direct association of O

For example, the following code fragment violates LoD due to the reason: while b is a ‘friend’ of

foo (because it receives it as a parameter), g is a ‘friend of a friend’ (which should be considered

a ‘stranger’), and g.doSomething() is analogous to ‘talking to a stranger’.

void foo(Bar b){ Goo g = b.getGoo(); g.doSomething(); }

Limiting the interaction to a closely related group of classes aims to reduce coupling. In the

example able, foo is already coupled to Bar. Upholding LoD avoids foo being coupled to Goo.

An analogy for LoD can be drawn from Facebook. If Facebook followed LoD, you would not be

allowed to see posts of friends of friends, unless they are your friends as well. If Jake is your

friend and Adam is Jake’s friend, you should not be allowed to see Adam’s posts unless Adam is

a friend of yours as well.


2

SOLID principles

Five OOP principles covered in other handouts are popularly known as SOLID principles:

Single Responsibility Principle(SRP) - Every class should have a single responsibility, and that responsibility should be entirely encapsulated by the class. In other words, a class should be highly cohesive. This was proposed by Robert C. Martin. This is a more concrete application of the Separation of Concerns Principle.

Open-Closed Principle (OCP) – A software module should be open for extension but should be closed for modification. This was proposed by Bertrand Meyer.

Liskov Substitution Principle – if a program module is using a super class, then the reference to the super class can be replaced with a sub class without affecting the functionality of the program module. This was proposed by Barbara Liskov.

Interface Segregation Principle – No client should be forced to depend on methods it does not use.

Dependency Inversion Principle (DIP). High-level modules should not depend on low-level modules. Both should depend on abstractions.

Other principles

Some other principles:

Brooks’ law – Adding people to a late project will make it later. This is because the additional communication overhead will outweigh the benefit of adding extra manpower. This principle was proposed by Fred Brooks (author of The Mythical Man-Month)

The later you find a bug, the more costly it is to fix. Covered in a previous handout. Good, cheap, fast – select any two. For example, good software developed within a very

short time will not come cheap.

As you can see, different principles have varying degrees of formality – rules, opinions, rules of

thumb, observations, and axioms. Hence, their applicability is wider with correspondingly

greater overlap as compared to patterns.

Worked examples

[Q1] Explain the Law of Demeter using code examples. You are to make up your own code examples.

Take Minesweeper as the basis for your code examples.

[A1] Let us take the Logic class as an example. Assume that it has the following operation.

setMinefield(Minefiled mf):void

Consider the following that can happen inside this operation.

mf.init(); //this does not violate LoD since LoD allows calling operations of parameters

received.

mf.getCell(1,3).clear(); //this violates LoD since Logic is handling Cell objects deep inside

Minefield. Instead, it should be mf.clearCellAt(1,3);

timer.start(); //this does not violate LoD since timer appears to be an internal component (i.e.

a variable) of Logic itself.

Cell c = new Cell(); c.init(); // this does not violate LoD since c was created inside the

operation.


3

[Q2] Do these principles apply to your project? If yes, briefly explain how.

i. Brooks’ Law.

ii. Good, cheap, fast –select any two.

iii. The later you find a bug, the more costly it is to fix.

[A2] One of the important learning points here is that it is easy to dismiss a principle as ‘not

applicable’. However, a second look often reveals a way we can relate them to situations that

look ‘unrelated’ at the beginning.

i. Brooks’ Law:

Yes. Adding a new student to a project team can result in a slow-down of the project for

a short period. This is because the new member needs time to learn the project and

existing members will have to spend time helping the new guy get up to speed. If the

project is already behind schedule and near a deadline, this could delay the delivery

even further.

ii. Good, cheap, fast –select any two:

Yes. While our project does not involve payments, there is still a hidden price to pay. For

example, assume we need the product to be good but finish the project within one week.

During that week, the time invested in the project will be very heavy and your other

modules will suffer as a result (i.e. you will pay a heavy price in terms of falling behind in

other modules). If there is no such cost, everyone can do the project in the last week and

still have a good project, right?

iii. The later you find a bug, the more costly it is to fix:

Yes. Imagine you found a bug just before the submission deadline. Not only you have to

find the bug and fix it, it could mean redoing the video, modifying the user guide, re-

uploading files, not forgetting the additional stress incurred on all other members and

the possibility of penalties for overrunning a deadline. The matter could have cost much

less if you were able to find the bug and have it fixed before the code was integrated.

[Q3] Find out, and explain in your own words (in 2-3 sentences), what is meant by the ‘second

system effect’ (attributed to Prof Fred Brooks). Do you think it is applicable to your project?

[A3] The second system effect refers to the tendency to follow a relatively small, elegant, and

successful system with an over-engineered, feature-laden, unwieldy successor (source:

Wikipedia).

We should keep the above in mind when we do V0.2. If we are not careful, our relative

success in V0.1 could mislead us to create an unwieldy feature-laden product at V0.2. Taking

this notion further, your success in the CS2103 project can even mislead you in coming up

with a failed product in your next project module.

--- End of document ---


1

[L10P3]

One Destination, Many Paths: Software Process Models

SDLC process models Software development goes through different stages such as requirements, analysis, design,

implementation and testing. These stages are collectively known as the software development

life cycle (SDLC). There are several approaches, known as software development life cycle models

(also called software process models) that describe different ways to go through the SDLC. Each

process model prescribes a “roadmap” for the software developers to manage the development

effort. The roadmap describes the aims of the development stage(s), the artifacts or outcome of

each stage as well as the workflow i.e. the relationship between stages.

This handout covers,

(a) Two basic process models that are used as the building blocks for more sophisticated process models:

i) sequential model ii) iterative model

(b) Three popular process models, each a combination of the two basic process models mentioned in (a).

iii) Unified process iv) Extreme programming v) Scrum

(c) A process improvement approach called CMMI

i) Sequential model

The sequential model, also called the waterfall model, is probably the earliest process

model employed. It models software development as a linear process, in which the

project is seen as progressing steadily in one direction through the development stages.

The name waterfall stems from how the model is drawn to look like a waterfall (see

below).

Specify Requirements

Design product

Implement

Test

Deploy

requirements

design

code

Bug fixes

Final product

When one stage of the process is completed, it should produce some artifacts to be used

in the next stage. For example, upon completion of the requirement stage a

comprehensive list of requirements is produced that will see no further modifications. A

strict application of the sequential model would require each stage to be completed

before starting the next.

This could be a useful model when the problem statement that is well-understood and

stable. In such cases, using the sequential model should result in a timely and systematic

development effort, provided that all goes well. As each stage has a well-defined

outcome, the progress of the project can be tracked with a relative ease.


2

The major problem with this model is that requirements of a real-world project are

rarely well-understood at the beginning and keep changing over time. One reason for

this is that users are generally not aware of how a software application can be used

without prior experience in using a similar application.

ii) Iterative model

The iterative model (sometimes called iterative and incremental10) advocates having

several iterations of SDLC. Each of the iterations could potentially go through all the

development stages, from requirement gathering to testing & deployment. Roughly, it

appears to be similar to several cycles of the sequential model.

Iterative (contd)

nth intermediate version

nth Iteration•Evolve existing functionality•Add new subset of functionality

feedback

n=n+1

V0.1 V0.2 V1.0V0.3

feedback feedback

In this model, each of the iterations produces a new version of the product. Feedback on

the version can then be fed to the next iteration. Taking the Minesweeper game as an

example, the iterative model will deliver a fully playable version from the early

iterations. However, the first iteration will have primitive functionality, for example, a

clumsy text based UI, fixed board size, limited randomization etc. These functionalities

will then be improved in later releases.

The iterative model can take a breadth-first or a depth-first approach to iteration

planning.

breadth-first: an iteration evolves all major components in parallel. depth-first: an iteration focuses on fleshing out only some components.

The breadth-first approach is considered purely ‘iterative’, while depth-first is

‘incremental’. As mentioned before, most project use a mixture of breadth-first and

depth-first iterations. Hence, the common phrase ‘an iterative and incremental process’.

In summary,

The sequential model organizes the project based on activity. The iterative and incremental model organizes the project based on functionality.

Most process models used today, including the ones described below, are hybrids of the

sequential and iterative models.

10 While some authors define iterative and incremental as two distinct models, the definitions of each vary across authors. In this handout, we define them together as they are most often used together.


3

iii) Unified process

The unified process is developed by the Three Amigos - Ivar Jacobson, Grady Booch and

James Rumbaugh (the creators of UML).

The unified process consists of four phases: inception, elaboration, construction and

transition. The main purpose of each phase can be summarized as follows:

Phase Activities Typical Artifacts

Ince

pti

on Understand the problem and requirements

Communicate with customer

Plan the development effort

Basic use case model

Rough project plan

Project vision and scope

Ela

bo

rati

on Refines and expands requirements

Determine a high-level design e.g. system

architecture

System architecture

Various design models

Prototype

Co

nst

ruct

ion

Major implementation effort to support the use

cases identified

Design models are refined and fleshed out

Testing of all levels are carried out

Multiple releases of the system

Test cases of all levels System release

Tra

nsi

tio

n Ready the system for actual production use

Familiarize end users with the system

Final system release

Instruction manual

Given above is a visualization of a project done using the Unified process (source:

Wikipedia). As the diagram shows, a phase can consist of several iterations. Each vertical

column (labeled “I1” “E1”, “E2”, “C1”, etc.) represents a single iteration. Each of the

iterations consists of a set of ‘workflows’ such as ‘Business modeling’, ‘Requirements’,

‘Analysis & Design’ etc. The shaded region indicates the amount of resource and effort

spent on a particular workflow in a particular iteration.

Note that the unified process is a flexible and customizable process model framework

rather than a single fixed process. For example, the number of iterations in each phase,

definition of workflows, and the intensity of a given workflow in a given iteration can be

adjusted according to the nature of the project. Take the Construction Phase, to develope

a simple system, one or two iterations would be sufficient. For a more complicated

system, multiple iterations will be more helpful. Therefore, the diagram above simply


4

records a particular application of the UP rather than prescribe how the UP is to be

applied. However, this record can be refined and reused for similar future projects.

In 2001, a group of prominent software engineering practitioners met and brainstormed for an

alternative to documentation-driven, heavyweight software development processes that were

used in most large projects at the time. This resulted in something called the agile manifesto (a

vision statement of what they were looking to do). The agile manifesto (taken from

http://agilemanifesto.org/) is given below.

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Subsequently, some of the signatories of the manifesto went on to create process models that

try to follow it. These processes are collectively called agile processes. Some of the key features

of agile approaches are:

Requirements are prioritized based on the needs of the user, are clarified regularly (at times almost on a daily basis) with the entire project team, and are factored into the development schedule as appropriate.

Instead of doing a very elaborate and detailed design and a project plan for the whole project, the team works based on a rough project plan and a high level design that evolves as the project goes on.

Strong emphasis on complete transparency and responsibility sharing among the team members. The team is responsible together for the delivery of the product. Team members are accountable, and regularly and openly share progress with each other and with the user.

There are a number of agile processes in the development world today. eXtreme Programming

(XP) and Scrum are two of the well-known ones.

iv) eXtreme programming (XP)

The following description was adapted from http://www.extremeprogramming.org

The first Extreme Programming project was started March 6, 1996. Extreme

Programming is one of several popular Agile Processes. Extreme Programming (XP)

stresses customer satisfaction. Instead of delivering everything you could possibly want

on some date far in the future, this process delivers the software you need as you need

it. XP aims to empower developers to confidently respond to changing customer

requirements, even late in the life cycle.

XP emphasizes teamwork. Managers, customers, and developers are all equal partners

in a collaborative team. XP implements a simple, yet effective environment enabling

teams to become highly productive. The team self-organizes around the problem to

solve it as efficiently as possible.


5

XP aims to improve a software project in five essential ways: communication, simplicity,

feedback, respect, and courage. Extreme Programmers constantly communicate with

their customers and fellow programmers. They keep their design simple and clean. They

get feedback by testing their software starting on day one. Every small success deepens

their respect for the unique contributions of each and every team member. With this

foundation, Extreme Programmers are able to courageously respond to changing

requirements and technology.

XP has a set of simple rules. XP is a lot like a jig saw puzzle with many small pieces.

Individually the pieces make no sense, but when combined together a complete picture

can be seen. This flow chart shows how Extreme Programming's rules work together.

Pair programming, CRC cards, project velocity, and standup meetings are some

interesting topics related to XP. Refer to extremeprogramming.org to find out more

about XP.

v) Scrum

This description of Scrum was adapted from Wikipedia [retrieved on 18/10/2011]:

Scrum is a process skeleton that contains sets of practices and predefined roles. The

main roles in Scrum are:

The ScrumMaster, who maintains the processes (typically in lieu of a project manager)

The Product Owner, who represents the stakeholders and the business The Team, a cross-functional group who do the actual analysis, design,

implementation, testing, etc.

A Scrum project is divided into iterations called Sprints. A sprint is the basic unit of

development in Scrum. Sprints tend to last between one week and one month, and are a

timeboxed (i.e. restricted to a specific duration) effort of a constant length.

Each sprint is preceded by a planning meeting, where the tasks for the sprint are

identified and an estimated commitment for the sprint goal is made, and followed by a

review or retrospective meeting, where the progress is reviewed and lessons for the

next sprint are identified.

During each sprint, the team creates a potentially deliverable product increment (for

example, working and tested software). The set of features that go into a sprint come

from the product backlog, which is a prioritized set of high level requirements of work

to be done. Which backlog items go into the sprint is determined during the sprint


6

planning meeting. During this meeting, the Product Owner informs the team of the items

in the product backlog that he or she wants completed. The team then determines how

much of this they can commit to complete during the next sprint, and records this in the

sprint backlog. During a sprint, no one is allowed to change the sprint backlog, which

means that the requirements are frozen for that sprint. Development is timeboxed such

that the sprint must end on time; if requirements are not completed for any reason they

are left out and returned to the product backlog. After a sprint is completed, the team

demonstrates the use of the software.

Scrum enables the creation of self-organizing teams by encouraging co-location of all

team members, and verbal communication between all team members and disciplines in

the project.

A key principle of Scrum is its recognition that during a project the customers can

change their minds about what they want and need (often called requirements churn),

and that unpredicted challenges cannot be easily addressed in a traditional predictive or

planned manner. As such, Scrum adopts an empirical approach—accepting that the

problem cannot be fully understood or defined, focusing instead on maximizing the

team’s ability to deliver quickly and respond to emerging requirements.

Like other agile development methodologies, Scrum can be implemented through a wide

range of tools. Many companies use universal software tools, such as spreadsheets to

build and maintain artifacts such as the sprint backlog. There are also open-source and

proprietary software packages dedicated to management of products under the Scrum

process. Other organizations implement Scrum without the use of any software tools,

and maintain their artifacts in hard-copy forms such as paper, whiteboards, and sticky

notes. The diagram below illustrates the essence of the Scrum process.

CMMI (Capability Maturity Model Integration)

The following description was adapted from http://www.sei.cmu.edu/cmmi/

CMMI (Capability Maturity Model Integration) is a process improvement approach

defined by Software Engineering Institute at Carnegie Melon University. CMMI provides

organizations with the essential elements of effective processes, which will improve

their performance. CMMI-based process improvement includes identifying an

organization’s process strengths and weaknesses and making process changes to turn

weaknesses into strengths.


7

CMMI defines five maturity levels for a process and provides criteria to determine if the process

of an organization is at a certain maturity level. The diagram below [taken from Wikipedia]

gives an overview of the five levels.

Worked examples

[Q1] Explain the process you followed in your project using the unified process framework. You may

also include smaller iterations that you followed within the two iterations we specified.

[A1] Sample only. Will be different from project to project. We assume that you are building a

Minesweeper game as your project.

Inception Elaboration Construction

Reqts

Analysis

Design

Impl.

Testing

Iter. 1 Iter. 2 Iter. 3 Iter. 4 Iter. 5 Iter. 6 Iter. 7

Proposal V0.1 V0.2

Iter 1: Project kick-off. We decide to do Minesweeper


8

Iter2: Produce the proposal

Iter 3: Modify proposal on feedback, construct a minimal V0.0

Iter 4: Construct version V0.1

V0.2 is produced in three short iterations (5,6,7).

[Q2] Discuss how sequential approach and the iterative approach can affect the following aspects of a

project.

a) Quality of the final product.

b) Risk of overshooting the deadline.

c) Total project cost.

d) Customer satisfaction.

e) Monitoring the project progress.

f) Suitability for CS2103 project

[A2] a) Quality of the final product:

Iterative: Frequent reworking can deteriorate the design. Frequent refactoring should be used to prevent this. Frequent customer feedback can help to improve the quality (i.e. quality as seen by the customer).

Sequential: Final quality depends on the quality of each phase. Any quality problem in any phase could result in a low quality product.

b) Risk of overshooting the deadline.

Iterative: Less risk. If the last iteration got delayed, we can always deliver the previous version. However, this does not guarantee that all features promised at the beginning will be delivered on the deadline.

Sequential: High risk. Any delay in any phase can result in overshooting the deadline with nothing to deliver.

c) Total project cost.

Iterative: We can always stop before the project budget is exceeded. However, this does not guarantee that all features promised at the beginning will be delivered under the estimated cost. (The sequential model requires us to carry on even if the budget is exceeded because there is no intermediate version to fall back on). Iterative reworking of existing artifacts could add to the cost. However, this is “cheaper” than finding at the end that we built the wrong product.

d) Customer satisfaction

Iterative: Customer gets many opportunities to guide the product in the direction he wants. Customer gets to change requirements even in the middle of the product. Both these can increase the probability of customer satisfaction.

Sequential: Customer satisfaction is guaranteed only if the product was delivered as promised and if the initial requirements proved to be accurate. However, the customer is not required to do the extra work of giving frequent feedback during the project.

e) Monitoring project progress

Iterative: Hard to measure progress against a plan, as the plan itself keeps changing.


9

Sequential: Easier to measure progress against the plan, although this does not ensure eventual success.

f) Suitability for CS2103 project:

Reasons to use iterative:

Requirements are not fixed.

Overshooting the deadline is not an option.

Gives a chance to learn lessons from one iteration and apply them in the next.

Sequential:

Can save time because we minimize rework.

[Q3] Find out more about the following three topics and give at least three arguments for and three

arguments against each.

(a) Agile processes, (b) Pair programming, (c) Test-driven development

[A3]

(a) Arguments in favor of agile processes:

More focus on customer satisfaction.

Less chance of building the wrong product (because of frequent customer feedback).

Less resource wasted on bureaucracy, over-documenting, contract negotiations.

Arguments against agile processes (not necessarily true):

It is ‘just hacking’. Not very systematic. No discipline.

It is hard to know in advance the exact final product.

It does not give enough attention to documentation.

Lack of management control (gives too much freedom to developers)

(b) Arguments in favor of pair programming:

It could produce better quality code.

It is good to have more than one person know about any piece of code.

It is a way to learn from each other.

It can be used to train new programmers.

Better discipline and better time management (e.g. less likely to play Farmville while

working).

Better morale due to more interactions with co-workers.

Arguments against pair programming:

Increase in total man hours required

Personality clashes between pair-members

Workspaces need to be adapted to suit two developers working at one computer.

If pairs are rotated, one needs to know more parts of the system than in solo

programming

(c) Arguments in favor of TDD:

Testing will not be neglected due to time pressure (because it is done first).


10

Forces the developer to think about what the component should be before jumping into

implementing it.

Optimizes programmer effort (i.e. if all tests pass, there is no need to add any more

functionality).

Forces us to automate all tests.

Arguments against TDD (not necessarily true):

Since tests can be seen as ‘executable specifications’, programmers tend to neglect

others forms of documentation.

Promotes ‘trial-and-error’ coding instead of making programmers think through their

algorithms (i.e. ‘just keep hacking until all tests pass’).

Gives a false sense of security. (what if you forgot to test certain scenarios?)

Not intuitive. Some programmer might resist adopting TDD.

--- End of handout ---


11

Index

Abstract classes/operations, 5

abstraction, 6

control, 6

data, 6

acceptance testing, 4

activity diagrams, 1

activity, 1

branch nodes, 2

forks, 2

guards, 2

joins, 2

merge nodes, 2

rake, 3

aggregation, 2

alpha testing, 4

API, 5

Application programming interface, 5

architecture, 1

broker, 4

client-server, 3

peer-to-peer, 4

pipes-and-filters, 4

service-oriented, 3

transaction-processing, 3

architecture:, 2

association

compulsory, 2

label, 2

multiplicity, 3

beta testing, 4

bottom-up, 2

boundary value analysis, 4

brainstorming, 2

brown-field project, 1

CFG, 5

CMMI, 6

cohesion, 1

Communication diagrams, 9

component diagram, 7

composite structure diagram, 9

control flow graphs, 5

coupling, 1

coverage, 5

data participants, 2

debugging, 1

Deployment diagrams, 7

design

agile, 6

full-design-upfront, 6

patterns, 1

design pattern, 1

Abstraction occurrence, 1

anti-pattern, 1

Command, 5

context, 1

Façade, 4

Model-View-Controller, 6

Observer, 8

Singleton, 4

State, 19

design-by-contract, 5

dynamic binding, 5

enumeration, 5

equivalence partitioning, 2

eXtreme programming, 4

feature list, 4

focus groups, 2

formal verification, 8

Gantt charts, 4

inheritance, 10

multiple, 6

inspections, 7

integrated development environments, 1

integration

big-bang, 1

continuous, 2

sandwich, 2

top-down, 1

Interaction overview, 8

interface, 12

interviews, 2

Iterative model, 2

magic numbers, 3

milestone, 4

observation, 2

operation contracts

postconditions, 6

preconditions, 6

overloading, 4

overriding, 4

package diagram, 8

PERT, 3

polymorphism, 2

principle

law of Demeter, 1

open-closed, 2

separation of concerns, 1

Program Evaluation Review Technique, 3

prototype, 2

prototyping, 2

refactoring, 1

referential integrity, 4

regression, 2

regression testing, 3

requirements

analysis, 1

categorizing, 11

elicitation, 1

functional, 1

gathering, 1

non-functional, 1

prioritizing, 11

supplementary, 10

revision control, 1

Scrum, 5

SDLC, 1

sequence diagram, 2

activation bar, 3


12

lifeline, 3

Sequential model, 1

Software Development Life Cycle, 1

agile approaches, 4

waterfall model, 1

software framework, 1

state machine diagram

action, 13

activity, 13

activity states, 14

determinism, 12

do activity, 14

effect, 13

entry activity, 13

exit activity, 13

final pseudostate, 12

guards, 12

initial pseudostate, 12

internal activity, 13

state, 12

transition, 12

trigger, 12

state tables, 19

static analyzers, 8

stub, 1

SUT, 1

system testing, 3

TDD, 3

team

chief programmer, 5

egoless, 5

strict hierarchy, 6

test

automation, 3

case, 1

driver, 1

testability, 2

test-driven development, 3

testing

black-box, 1

exploratory, 2

gray-box, 1

scripted, 2

Timing diagrams, 6

UML, 5

Unified process, 3

use cases, 5

actor, 6

diagrams, 9

extensions, 8

guarantees, 9

inclusions, 8

main success scenario, 7

MSS, 7

preconditions, 9

scenario description, 6

user stories, 4

user surveys, 2

Documents

Platforms, frameworks and librariescs2103/AY1617S2/files... · Web application frameworks ... (such as in scalability, security, performance, and robustness) than software meant for