For this assignment, you will create a set of functional tests for a command-line utility that reads in a file and transforms or computes with it to produce output. You will choose one of the command-line utility specifications found in the Posix specification for compatible Unix systems.

The Open Group Base Specifications Issue 7, IEEE Std 1003.1, 2013 Edition

The utility must perform some nontrivial transformation or computation based upon the provided file command line options. For instance, the cat command would is insufficient.

This is a group assignment, where n is your group size. A size of 2 or 3 is preferred. The assignment requirements are based on group size as follows.

• You will apply the method of combinatorial testing with input space partitioning and design a test set that achieves at least t-way combinatorial coverage, where t >= n and n is your group size.

• You will apply your test set to at least m distinct implementations of the utility, where m = max(2n-t, 2). Note that open-source BSD and GNU implementations of most utilities exist.

This assignment incorporates the following components.

1. Specification of the Program Under Test
2. Input Space Partitioning
3. Combinatorial Test Generation
4. Test Report

Find bugs! The goal of testing is to find bugs.

Specification of the Program Under Test

Provide a concise specification of the input/output requirements of the program or portion of a program that you are testing, with references to appropriate documents.

Input Space Partitioning

Recall the approach presented in class for the category-partition method.

1. Clearly identify the component you are testing along with all of its inputs, whether from parameters, user input, the environment, etc.
2. Identify the categories for partitioning the input space.
3. Clearly present your partitioned input domain model along with any constraints that are necessary for making it consistent.

You should clearly identify the blocks of choices resulting from your categories and also explain why you added any constraints as a part of your report.

Test Case Generation

Automate the test-frame and test-case generation as much as possible. You may choose to use a tool such as ACTS provided by the National Institute of Standards and Technology (NIST) for assisting in combinatorial testing, or you may choose to write your own test generation program such as that of the UTF-8 to UTF-16 study.

Once you have generated a plan for which tests to run, create an actual test for each planned test frame. You should run these tests on your selected program.

Test Report and Submission

Using the generated test suite, evaluate the programs under test and report the results. How many tests did you generate? Did you achieve all-pairs or other t-way coverage? How many of these tests were successful/passing?

Your assignment submission should be well-organized with the following sections.

1. Introduction and identification of the Program Under Test and its specification documents. (You need a well-documented specification from which to derive your input space model).
2. Your Input Space Model
   1. Parameter Environment Categories
   2. Choices or Values in each Category.
   3. Constraints on choice combinations.
3. Method of generating test frames and summary of the results.
4. Method of generating test cases and summary of the results.
5. Results of testing. Bugs found. Discrepancies between different implementations.
6. Observations and Conclusions

The assignment is due Tuesday October 18 2016, 13:00.