From 5e12d220d39fdbe721700af23682fddd7e3dd09b Mon Sep 17 00:00:00 2001 From: ph0zzy Date: Feb 27 2019 15:47:25 +0000 Subject: add quick start guide page --- diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index e65e045..26efe1a 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,3 +1,4 @@ +* xref:quick_start_guide.adoc[Quick Start Guide] * xref:standard-test-interface.adoc[Standard Test Interface] * xref:standard-test-roles.adoc[Standard Test Roles] * xref:faq.adoc[FAQ] diff --git a/modules/ROOT/pages/quick_start_guide.adoc b/modules/ROOT/pages/quick_start_guide.adoc new file mode 100644 index 0000000..9dfae61 --- /dev/null +++ b/modules/ROOT/pages/quick_start_guide.adoc @@ -0,0 +1,175 @@ +== Introduction == + +Are you eager to try out how the Fedora CI tests work? +Do you want to get a quick hands-on experience without having to read too much documentation? +This quick introduction for the impatient will show you a minimal set of steps to execute existing tests as well as provide useful links to resources where you can learn more. + +== First Steps == + +Install the following essential packages on your system (consider using a virtual machine for safe experimenting): + + sudo dnf install fedpkg standard-test-roles + +Use `fedpkg` to clone the package git repository. +See the https://fedoraproject.org/wiki/Package_maintenance_guide[Package Maintenance Guide] for more info about the tool. + + fedpkg clone -a bash + +Tests are defined according to the xref:standard-test-interface.adoc[Standard Test Interface] in the `tests` directory: + + cd bash/tests/ + +Test coverage to be executed together with the basic set of metadata is described in the https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml[tests.yml] playbook. +Use `ansible-playbook` to run all available tests for the classic environment on the local host (needs to be run as root): + + ansible-playbook --tags=classic tests.yml + +From the ansible output you can directly see an overall summary of the testing. +If you see `failed=0` at the end of the log then all tests passed: + + localhost: ok=29 changed=11 unreachable=0 failed=0 + +For more detailed test results check the `test.log` and other files in the `artifacts` directory: + + vim artifacts/test.log + +That's it! You just executed test coverage for the Bash package :) + +== Test Subjects == + +To execute tests against different test subjects we need to prepare the environment. +Let's store the detailed test results in `/tmp/artifacts`, use dynamic inventory as defined by the xref:standard-test-roles.adoc[Standard Test Roles] and download the latest Atomic Host image. + + export TEST_ARTIFACTS=/tmp/artifacts + export ANSIBLE_INVENTORY=/usr/share/ansible/inventory + curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest + +Now let's try to run tests against all supported test subjects. + +=== Classic === + +Run tests against classic rpms installed on the system: + + export TEST_SUBJECTS='' + ansible-playbook --tags=classic tests.yml + +See xref:standard-test-roles.adoc#_classic[Classic] for detailed docs. + +=== Container === + +For testing containers there is an additional dependency needed: + + sudo dnf install standard-test-roles-inventory-docker + +Run tests in a docker container: + + export TEST_SUBJECTS=docker:docker.io/library/fedora:latest + ansible-playbook --tags=container tests.yml + +See xref:standard-test-roles.adoc#_container[Container] for detailed docs. + +=== Atomic === + +Run tests against the Atomic Host: + + export TEST_SUBJECTS=/tmp/atomic.qcow2 + ansible-playbook --tags=atomic tests.yml + +See xref:standard-test-roles.adoc#_atomic[Atomic] for detailed docs. + +== Hints == + +=== Debug === + +Would you like to investigate why a test failed? Enable debugging to easilly connect to running Atomic or Container to investigate: + + export TEST_DEBUG=1 + ansible-playbook --tags=atomic tests.yml + +See xref:standard-test-roles.adoc#_debug[Debug] for details about debugging. + +=== Ignore === + +Use `.gitignore` to specify files that Git should ignore. +Such files are created during tests run. +Create a `tests/.gitignore` file with the following contents: + +[source,gitignore] +---- +# Ignore tests runs/artefacts. +artifacts/** +**/*.retry +---- + +== Contribute == + +Are you interested in contributing a new test coverage? +You are most welcome! +As you have seen xref:tests.adoc#_executing[Executing] a test is quite easy. +xref:tests.adoc#_writing[Writing] a new test or xref:tests.adoc#_wrapping[Wrapping] an existing one is quite simple as well. +Here's a few recommendations for creating a new pull request. + +=== Fork === + +Unless you are maintainer of the package, who has direct commit access, create a fork of the package git repository using the Fork button in https://src.fedoraproject.org/rpms/bash[Pagure] web interface and add your private fork as a new remote. +Create a branch for your new tests. +For example: + + git remote add fork ssh://psss@pkgs.fedoraproject.org/forks/psss/rpms/bash.git + git checkout -b tests + +If you are not a Fedora packager, use fedpkg command to clone you fork and set up the git repo config so that you are able to push to it. +See xref:pull_requests.adoc[Pull Requests] for more detailed info. + + fedpkg clone -a forks/psss/rpms/bash + git checkout -b tests + +=== Add === + +Create new test coverage under the `tests` directory, update the `tests.yml` file accorgingly or create a new one. +Run tests and verify they are stable and working fine in all supported environments. +Add files to git, commit and push: + + git add tests.yml test1 test2 test3 + git commit -m "Add CI tests using the Standard Test Interface" + git push fork tests:tests + +It is a good idea to include more details and links in the commit message to make the pull request easier for review: + +[source,message] +---- +Enable CI tests using the Standard Test Interface + +Adding initial set of basic functionality tests for bash +according to the Standard Test Interface [1]. See Quick Start +Guide [2] for brief introduction about how to run these tests +and the Fedora CI portal [3] for more detailed info and links. + +[1] https://fedoraproject.org/wiki/CI/Standard_Test_Interface +[2] https://fedoraproject.org/wiki/CI/Quick_Start_Guide +[3] https://fedoraproject.org/wiki/CI +---- + +Create a new pull request from your `tests` branch against the master branch in the https://src.fedoraproject.org/fork/psss/rpms/bash[Pagure] web interface. +You might want to include an additional info about the tests such as: + +[source,message] +---- +There are three tests available: smoke and func have been tested +across all environments (classic, container, atomic), login is +relevant for classic only (because of a missing dependency). +Please, merge the tests into all currently supported branches. +---- + +=== Results === + +Once the pull request is created CI Pipeline will detect it and execute tests. +Once the test execution is finished you will see results of the testing on the pull request page. +See the xref:pipeline.adoc[Pipeline] page for the list of active pipelines and result examples. + +=== Gating === + +Currently gating the package on test results is an opt-in feature. +In order to enable gating for you component create a `gating.yaml` file in the root of your component dist git repository. +See xref:gating.adoc[Gating] for more details. +