Automated Testing in Jenkins

# CHAPTER 10

Automated Testing in Jenkins #

1. Introduction #

2. Learning Objectives #

• Understand the role of Unit Testing in a CI pipeline.

• Configure Jenkins to execute a test suite (e.g., PHPUnit or Jest).

• Understand how Jenkins uses exit codes to determine test success or failure.

• Configure Jenkins to collect and display test reports (JUnit XML).

• Prevent pipeline progression upon test failure.

3. Beginner-Friendly Explanation #

• The Developer: Writes a new chapter of a book and submits it.

• The Code: Contains a massive typo: "The quick brown fxo."

• Jenkins (Without Testing): Automatically prints 10,000 copies of the book with the typo and ships them to stores.

• Jenkins (With Testing): Runs an automated spell-check script on the chapter. The script finds "fxo", throws an error, and stops the printing press. Jenkins emails the author: "Fix your typo." The bad book never reaches the customer.

4. The Anatomy of a Unit Test #

How Jenkins runs tests: Jenkins doesn't know how to test PHP or JavaScript inherently. Jenkins simply runs a terminal command (like phpunit tests/) and watches the Exit Code.

• If the test suite finishes and returns an exit code of 0 -> Jenkins says "Tests Passed."

• If the test suite returns an exit code of 1 -> Jenkins says "Tests Failed" and halts the pipeline immediately.

5. Collecting Test Reports #

6. Mini Project: Run Automated PHP Tests in Jenkins #

Step-by-Step Pipeline Concept:

pipeline {
    agent any
    stages {
        stage('Install') {
            steps {
                sh 'composer install'
            }
        }
        stage('Unit Testing') {
            steps {
                echo 'Running PHPUnit tests...'
                // Run PHPUnit, and output the results to an XML file.
                // If any test fails, this 'sh' step returns exit code 1, stopping the pipeline.
                sh './vendor/bin/phpunit --log-junit test-results.xml tests/'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying to Staging...'
            }
        }
    }
    // The post block runs even if the Unit Testing stage failed
    post {
        always {
            // Tell Jenkins to read the XML file and create a dashboard graph
            junit 'test-results.xml'
        }
    }
}

7. Real-World Scenarios #

8. Best Practices #

• Never Ignore Failing Tests: Sometimes developers get frustrated with a failing test and comment it out just to force Jenkins to turn green and deploy the code. This defeats the entire purpose of CI/CD. Broken tests must be fixed or properly evaluated, never silenced.

• Test Coverage: Use tools (like Xdebug for PHP) to generate Code Coverage reports. These tell you what percentage of your application code is actually covered by unit tests. A good goal is 80% coverage.

9. Security Recommendations #

• SAST as Testing: Testing isn't just for functionality; it is for security. You should run Static Application Security Testing (SAST) tools (like phpstan or SonarQube) in your testing stage. If the SAST tool detects a SQL injection vulnerability in the new code, it will return exit code 1, and Jenkins will halt the deployment.

10. Troubleshooting Tips #

• XML Path Errors: If your pipeline fails on the junit 'test-results.xml' step, verify that PHPUnit is actually saving the file in the exact root directory of the workspace, and that the filename perfectly matches.

11. Exercises #

1. 1. How does Jenkins know if a testing framework (like PHPUnit) succeeded or failed?

1. 2. Explain the purpose of the junit step in the post { always { ... } } block. Why put it in always instead of success?

12. FAQs #

13. Interview Questions #

• Q: Describe the integration point between a testing framework (like Jest or PHPUnit) and Jenkins. How do you extract structured test data to generate trend graphs on the Jenkins dashboard?

• Q: A pipeline executes unit tests, but if the tests fail, the pipeline immediately stops, meaning the test report graph is never updated. How do you architect the Jenkinsfile to ensure test reports are generated regardless of the test outcome?

14. Summary #

15. Next Chapter Recommendation #