# Testing Tutorial Part I: `[@Starter]` Bundle Tests

Right up there with `use strict`, `use warnings` and "eat your vegetables" is
the admonition to "test your code!" We'll spare nagging you with all the
excellent reasons why you should write tests for your code. But if you've been
avoiding testing, this is a good opportunity to start doing it.

The testing tutorial is split into three parts. The first is a basic overview of
testing and provides details about `[@Starter]` bundle test plugins. The second
is more hands-on and demonstrates how to add basic tests and run them with
`Dist::Zilla`. The third introduces useful test plugins.

If you aren't familiar with tests, don't worry. This tutorial is designed for
beginners. Just remember that a test file is nothing but a perl script that runs
functions to determine if your module works the way you intend. Each function in
the script returns a simple pass or fail value. After these tutorials, you'll
hopefully have the confidence to write simple tests for your code and/or get
motivated to learn more about testing from other very fine learning resources.

If you're already comfortable writing tests, you'll learn about the `dzil test`
command, how `Dist::Zilla` integrates tests with a distribution, and maybe some
new testing tools and strategies you're not using presently.

## Tests Supplied with the `[@Starter]` Bundle

The `[@Starter]` bundle includes plugins that generate two different types of
tests for us: **standard tests** and **author tests.**

### Standard Tests

Files containing standard tests are placed in the distribution's `t` ("t" for
"tests") directory. During module development, developers frequently run
standard tests to ensure new features work as expected and old code still works.
Standard tests should also be executed immediately before shipping the
distribution to end users.

End user's should run standard tests before installing modules to ensure that
the module still works properly despite having different hardware and software
than the developer. As much as possible, these tests are automated and require
no end user intervention. However, end users can skip them if they want to.

### Author Tests

Author tests are not run by end users but by developers, often in conjunction
with standard tests. The `[@Starter]` bundle plugins adds author tests to the
`xt/author` ("xt" for "extra tests") directory. Like standard tests, author
tests should be run just before releasing your code to the world.

### Running Tests Manually with the `[@Starter]` Bundle

Developers can manually initiate tests with:

`dzil test`

Normally, `dzil test` runs only standard tests. The `[@Starter]` bundle's
`[RunExtraTests]` plugin modifies `dzil test` to run author tests as well.
We'll cover other kinds of tests and how to run them in the next tutorial.

Behind the scenes, the `test` subcommand triggers the construction of a
temporary build of your distribution. Tests are run on this build using the
`Makefile.PL` build script generated by the build. If all tests pass, the build
files are removed. Otherwise, a copy of them are left behind in a hidden
`.build` directory for your inspection, though, in practice, you will rarely
need to.

## Examining Our Test Output

If you executed the previous tutorial tasks faithfully, `dzil test` will reward
you with a clean test and an `All's well` message at the very end:

```

[DZ] building distribution under .build/VHTxZpnZvW for installation
[DZ] beginning to build App-sayhi
[DZ] guessing dist's main_module is lib/App/sayhi.pm
[DZ] writing App-sayhi in .build/VHTxZpnZvW
[ReadmeAnyFromPod] overriding README.md in root
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for App::sayhi
Writing MYMETA.yml and MYMETA.json
cp lib/App/sayhi.pm blib/lib/App/sayhi.pm
cp bin/sayhi blib/script/sayhi
"/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/sayhi
PERL_DL_NONLAZY=1 PERL_USE_UNSAFE_INC=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-report-prereqs.t .. #
# Versions for all modules listed in MYMETA.json (including optional ones):
#
# === Configure Requires ===
#
#     Module              Want    Have
#     ------------------- ---- -------
#     ExtUtils::MakeMaker  any 7.10_02
#
# === Build Requires ===
#
#     Module              Want    Have
#     ------------------- ---- -------
#     ExtUtils::MakeMaker  any 7.10_02
#
# === Test Requires ===
#
#     Module              Want     Have
#     ------------------- ---- --------
#     ExtUtils::MakeMaker  any  7.10_02
#     File::Spec           any  3.63_01
#     Test::More           any 1.302140
#
# === Test Recommends ===
#
#     Module         Want     Have
#     ---------- -------- --------
#     CPAN::Meta 2.120900 2.150005
#
# === Develop Requires ===
#
#     Module     Want     Have
#     ---------- ---- --------
#     File::Spec  any  3.63_01
#     IO::Handle  any     1.36
#     IPC::Open3  any     1.20
#     Test::More  any 1.302140
#     Test::Pod  1.41     1.52
#
t/00-report-prereqs.t .. ok
All tests successful.
Files=1, Tests=1,  1 wallclock secs ( 0.01 usr  0.00 sys +  0.10 cusr  0.01 csys =  0.12 CPU)
Result: PASS
xt/author/00-compile.t .. ok
xt/author/pod-syntax.t .. ok
All tests successful.
Files=2, Tests=5,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.16 cusr  0.01 csys =  0.20 CPU)
Result: PASS
[DZ] all's well; removing .build/VHTxZpnZvW

```

If you see errors, review this tutorial's previous instructions to make sure
your code is correct and that the `Greetings` module is installed. If that looks
OK, check that you ran `dzil test` from the top level of the the work area
(source tree) and not from inside the distribution (build tree). Having a
misconfigured Perlbrew installation or some other complication regarding how
modules are installed on your machine might also result in errors. Whatever the
cause, try to resolve the issues before proceeding.

The output generated by `dzil test` can be confusing and takes practice
deciphering. It's a mix of information from a chain of tools that work together
to run the tests located in the `t` and `xt/author` directories in the
disribution:

1) Our old friend `Dist::Zilla` which...
2) builds the distribution end executes its `Makefile.PL` build script which...
3) loads the tests into the `Test::Harness` module which...
4) executes the module tests which may generate their own output.

You'll want to get good at finding the output from `Test::Harness` which tells
you which tests have passed and which have failed. In the output above, look for
the 'ok' messages and the three lines near the end, beginning with the `All
tests successful`. If you don't see that success message, you'll want to review
all error messages that will jump out at you as you gain more experience parsing
test output.

### The Prerequisites Standard Test Output

Most of the test output–the stuff in the middle in a tabular layout-is a report
generated by `[Test::ReportPrereqs]`, listing the software requirements for
installing, testing, configuring and running your module. Together, these are
known as **prerequisites.** The report can be helpful to CPAN testers,
developers, and end users who know what they are looking at. Again, the most
important part is the small bit from `Test::Harness`:

`t/00-report-prereqs.t .. ok`

If you look at `sayhi-0.001/t/00-report-prereqs.t` you'll see a lot of hairy,
scary code but don't let that worry you. Most tests are much, much simpler than
this. Accompanying the `00-report-prereqs.t` file in the `t` directory is
`00-report-prereqs.dd` which is not a test but a helper file to the test and you
don't have to concern yourself with how that works either.

### The `[@Starter`] Bundle Author Test Output

The `[@Starter]` bundle generates two author test files, both found in the
`xt/author` distribution directory. `Test::Harness` output reports the tests in
these file passed:

```

xt/author/00-compile.t .. ok
xt/author/pod-syntax.t .. ok
Files=2, Tests=5,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.15 cusr  0.02 csys =  0.20 CPU)
Result: PASS

```

The first of these test files is generated by the `[Test::Compile]` plugin and
the second by the `[PodSyntaxTests]`.

You may recall a while back we mentioned that bundles can modify the behavior of
plugins. The `[Test::Compile]` plugin is an example of that with the
`[@Starter]` bundle passing an `xt_mode = 1` parameter to the plugin, telling
`Dist::Zilla` to treat it like an author test instad of a standard test.

Let's look at what these tests do to make your job as a developer easier.

#### The Compile Test

The tests in `00-compile.t` ensure that our perl code won't throw any errors
when compiled by the perl interpreter. In other words, it acts as a syntax
checker and will report code that fails to compile.

#### The POD Syntax Test

The tests performed by `pod-syntax.t` will inspect your pod documenation and
report any syntax issues with it. We'll see an example of a failed pod syntax
test in the next tutorial.

## Automated Testing with the `[TestRelease]` Plugin

The `[TestRelease]` plugin, supplied by `[@Starter]`, runs all the standard,
author, and release tests when you issue a `dzil release` command to try to stop
you from introducing crappy code into the world. If a test doesn't pass, it will
alert you and stop the release process. More on the `release` subcommand soon.

This completes our look at the tests and test plugins the `[@Starter]` bundle
provides. Now it's time to don a pair of coveralls and mess with some tests.
