ForgeTesting

Overview

ForgeTesting provides a complete testing framework for Forge Kernel applications. Built with a tiny footprint (CLI-only), it offers powerful features for testing your application, engine, and modules. No web dependencies, minimal overhead, maximum productivity.

What ForgeTesting Provides

• Tiny Footprint: CLI-only module, no web dependencies, minimal overhead
• Flexible Scoping: Test app, engine, modules separately or together
• Powerful Grouping: Filter tests by groups for focused testing
• Comprehensive Assertions: 30+ assertion methods for all testing needs
• HTTP Testing: Built-in HTTP client with CSRF support
• Database Testing: Database assertions and migration support
• Performance Testing: Benchmark methods and execution time tracking
• Test Caching: Directory hash-based caching for faster test discovery

Architecture & Design Philosophy

ForgeTesting is built with performance, simplicity, and developer experience in mind. Its tiny footprint and powerful features make it ideal for both small projects and enterprise applications.

Tiny Footprint Design

ForgeTesting is designed as a CLI-only module:

• CLI-Only: Module marked with isCli: true — only loads in CLI context
• No Web Dependencies: Zero overhead for production web applications
• Minimal Dependencies: Uses only core Forge Kernel features
• Fast Execution: Optimized for speed with test caching

Attribute-Based Test Definition

Tests are defined using PHP 8 attributes, not method naming conventions:

• No requirement for method names to start with "test"
• Use #[Test] attribute to mark test methods
• Optional test descriptions in the attribute
• More flexible and explicit than naming conventions

Test Scoping Philosophy

Tests are organized by scope for better organization:

• App Tests: Application-specific tests in app/tests/
• Engine Tests: Core engine tests in engine/tests/
• Module Tests: Module-specific tests in modules/{ModuleName}/src/tests/
• Run tests by scope for focused testing

Test Caching System

Directory hash-based caching for performance:

• Cache file: storage/framework/cache/test_cache.php
• Cache TTL: 3600 seconds (1 hour)
• Automatic invalidation when test files change
• Faster test discovery on subsequent runs

Installation

Install ForgeTesting using ForgePackageManager. The module is CLI-only and has no web dependencies.

Using ForgePackageManager

# Install with wizard (interactive)
php forge.php package:install-module

# Install directly (skip wizard)
php forge.php package:install-module --module=ForgeTesting

# Install specific version
php forge.php package:install-module --module=ForgeTesting@0.1.4

CLI-Only Module

ForgeTesting is marked as CLI-only:

#[Module(
    name: "ForgeTesting",
    version: "0.1.4",
    description: "A Test Suite Module By Forge",
    order: 9999,
    isCli: true,  // Only loads in CLI context
)]

This means:

• Module only loads when running CLI commands
• Zero overhead for web requests
• No web dependencies or middleware
• Perfect for development and CI/CD

Test Scoping

ForgeTesting supports three test scopes: app, engine, and module. Run tests by scope for focused testing.

App Tests

Application-specific tests located in app/tests/:

# Run all app tests
php forge.php test

# Explicitly run app tests
php forge.php test --type=app

Example: app/tests/HomeTest.php

Engine Tests

Core engine tests located in engine/tests/:

# Run all engine tests
php forge.php test --type=engine

Examples: engine/tests/ValidatorTest.php, engine/tests/Engine/ContainerEngineTest.php

Module Tests

Module-specific tests located in modules/{ModuleName}/src/tests/:

# Run all module tests
php forge.php test --type=module

# Run tests for specific module
php forge.php test --type=module --module=ForgeAuth

Example: modules/ForgeAuth/src/tests/AuthenticationTest.php

Test Grouping

Group tests using the #[Group] attribute and filter by group when running tests.

Class-Level Grouping

Apply groups to entire test classes:

#[Group("http")]
final class HomeTest extends TestCase
{
    // All tests in this class belong to "http" group
}

Method-Level Grouping

Apply groups to individual test methods:

#[Group("auth")]
final class AuthenticationTest extends TestCase
{
    #[Test]
    #[Group("Database")]
    public function user_exists(): void
    {
        // This test belongs to both "auth" and "Database" groups
    }
}

Filtering by Group

Run only tests in a specific group:

# Run only Database group tests
php forge.php test --group=Database

# Run only http group tests
php forge.php test --group=http

# Run only validator group tests
php forge.php test --group=validator

Common Group Names

Examples from the codebase:

• http — HTTP/route tests
• auth — Authentication tests
• Database — Database tests
• validator — Validation tests
• container — Container/Dependency Injection tests
• helpers — Helper function tests

Writing Tests

All tests extend TestCase and use the #[Test] attribute to mark test methods.

Basic Test Structure

use App\Modules\ForgeTesting\Attributes\Test;
use App\Modules\ForgeTesting\TestCase;

final class HomeTest extends TestCase
{
    #[Test("Home / route is working")]
    public function home_route_is_ok(): void
    {
        $response = $this->get("/");
        $this->assertHttpStatus(200, $response);
    }
}

Test Method Naming

Test methods can have any name — they're identified by the #[Test] attribute:

// ✅ Valid — has #[Test] attribute
#[Test]
public function my_custom_test_name(): void { }

// ✅ Valid — descriptive name
#[Test("User can login")]
public function user_can_login(): void { }

// ❌ Not a test — no #[Test] attribute
public function helper_method(): void { }

Test Descriptions

Provide optional descriptions in the #[Test] attribute:

#[Test("Home / route is working")]
public function home_route_is_ok(): void { }

// Description appears in test results
// Test: Home / route is working
// Class: App\Tests\HomeTest
// Method: home_route_is_ok

Attributes System

ForgeTesting uses PHP 8 attributes to define test behavior. All attributes are optional except #[Test] for marking test methods.

#[Test] - Mark Test Methods

Required attribute to mark a method as a test:

#[Test]
public function my_test(): void { }

#[Test("Optional test description")]
public function my_test_with_description(): void { }

#[Group] - Group Tests

Group tests for filtering (class or method level):

#[Group("http")]
final class HomeTest extends TestCase { }

#[Group("Database")]
public function user_exists(): void { }

#[Skip] - Skip Tests

Skip tests with a reason:

#[Test]
#[Skip("Waiting on implementation")]
public function login_works(): void { }

#[Incomplete] - Mark Incomplete Tests

Mark tests as incomplete with a reason:

#[Test]
#[Incomplete("Needs to check save performance in the model")]
public function create_user(): void { }

#[DataProvider] - Provide Test Data

Provide multiple test data sets:

#[DataProvider("userProvider")]
#[Test]
public function multiple_users(array $users): void
{
    $this->assertArrayHasKey("email", $users);
}

public function userProvider(): array
{
    return [
        [["email" => "user1@example.com"]],
        [["email" => "user2@example.com"]],
    ];
}

#[Depends] - Test Dependencies

Run tests in order based on dependencies:

#[Test]
public function create_user(): void { }

#[Test]
#[Depends("create_user")]
public function update_user(): void
{
    // create_user() runs first
}

#[BeforeEach] - Setup Methods

Run before each test method:

#[BeforeEach]
public function setup(): void
{
    // Runs before each test
}

#[AfterEach] - Teardown Methods

Run after each test method:

#[AfterEach]
public function tearDown(): void
{
    // Runs after each test
}

Assertions

ForgeTesting provides a comprehensive assertions library with 30+ assertion methods covering all testing needs.

Basic Assertions

$this->assertTrue($value);
$this->assertFalse($value);
$this->assertEquals($expected, $actual);
$this->assertNotEquals($expected, $actual);
$this->assertSame($expected, $actual);      // ===
$this->assertNotSame($expected, $actual);    // !==

Null and Empty Assertions

$this->assertNull($value);
$this->assertNotNull($value);
$this->assertEmpty($value);
$this->assertNotEmpty($value);

Type Assertions

$this->assertInstanceOf(ExpectedClass::class, $object);
$this->assertNotInstanceOf(ExpectedClass::class, $object);

Array Assertions

$this->assertArrayHasKey("key", $array);
$this->assertArrayNotHasKey("key", $array);
$this->assertCount(5, $array);
$this->assertContains($needle, $haystack);
$this->assertNotContains($needle, $haystack);

String Assertions

$this->assertStringContainsString("needle", $haystack);
$this->assertStringNotContainsString("needle", $haystack);
$this->assertMatchesRegularExpression("/pattern/", $string);
$this->assertDoesNotMatchRegularExpression("/pattern/", $string);

Comparison Assertions

$this->assertGreaterThan(10, $value);
$this->assertLessThan(10, $value);
$this->assertGreaterThanOrEqual(10, $value);
$this->assertLessThanOrEqual(10, $value);

File Assertions

$this->assertFileExists($filename);
$this->assertFileDoesNotExist($filename);

JSON Assertions

$this->assertJsonStringEqualsJsonString($expectedJson, $actualJson);

HTTP Assertions

$this->assertHttpStatus(200, $response);
$this->assertHttpStatus(404, $response);

Exception Assertions

Expect exceptions to be thrown:

// Expect any exception
$this->shouldFail(function () {
    throw new \Exception("Error");
});

// Expect specific exception type
$this->shouldFail(
    function () {
        throw new ValidationException("Error");
    },
    ValidationException::class
);

HTTP Testing

ForgeTesting provides built-in HTTP testing capabilities with CSRF token support.

HTTP Methods

// GET request
$response = $this->get("/");

// POST request
$response = $this->post("/", ["key" => "value"]);

// PATCH request
$response = $this->patch("/1", ["key" => "value"]);

// With headers
$response = $this->get("/", [], ["X-Custom-Header" => "value"]);

CSRF Token Handling

ForgeTesting automatically handles CSRF tokens:

// Add CSRF token to POST data
$response = $this->post("/", $this->withCsrf([
    "email" => "test@example.com",
    "password" => "password123",
]));

// Add CSRF token to headers
$response = $this->patch(
    "/1",
    ["identifier" => "newuser"],
    $this->csrfHeaders()
);

Complete Example

#[Test("Home / route is working")]
public function home_route_is_ok(): void
{
    $response = $this->get("/");
    $this->assertHttpStatus(200, $response);
}

#[Test("POST / with invalid data redirects back")]
public function register_route_returns_redirect_on_validation_error(): void
{
    $response = $this->post(
        "/",
        $this->withCsrf([
            "email" => "invalid-email",
            "password" => "123",
        ]),
    );

    $this->assertHttpStatus(302, $response);
}

Database Testing

ForgeTesting provides database assertions and migration support for testing database operations.

Database Assertions

// Assert record exists
$this->assertDatabaseHas("users", [
    "identifier" => "example",
    "email" => "test@example.com",
]);

// Assert record does not exist
$this->assertDatabaseMissing("users", [
    "email" => "deleted@example.com",
]);

// Assert exact count
$this->assertDatabaseCount("users", 5, [
    "status" => "active",
]);

Database Refresh

Refresh database before tests:

#[BeforeEach]
public function setup(): void
{
    $this->refreshDatabase(); // Runs migrations
}

Seeding

Seed database with test data:

$this->seed(UserSeeder::class);

Complete Example

#[Test("Check a record exists in the Database by identifier")]
#[Group("Database")]
public function user_exists(): void
{
    $this->assertDatabaseHas("users", [
        "identifier" => $this->exampleUser["identifier"],
    ]);
}

Performance Testing

ForgeTesting provides performance benchmarking and execution time tracking.

Benchmarking

Measure performance with the benchmark() method:

#[Test("Benchmark user lookup")]
#[Group("Database")]
public function benchmark_user_lookup(): array
{
    $results = $this->benchmark(function () {
        $this->assertDatabaseHas("users", [
            "email" => $this->exampleUser["email"],
        ]);
    }, 1); // 1 iteration

    return $results; // Returns: ['avg', 'min', 'max', 'total']
}

Benchmark results are displayed in test output:

Benchmark Results:

App\Modules\ForgeAuth\Tests\AuthenticationTest::benchmark_user_lookup:
| Avg Time/Iter | Min Time/Iter | Max Time/Iter | Total Time |
+---------------+---------------+---------------+------------+
| 0.388 ms      | 0.388 ms      | 0.388 ms      | 0.388 ms   |

Execution Time Limits

Assert maximum execution time:

$this->assertMaxExecutionTime(0.5, function () {
    // Code that should complete in 0.5 seconds
    $this->doSomething();
});

Cache Testing

ForgeTesting provides utilities for testing cache behavior.

Flush Cache

$this->flushCache(); // Clears framework cache

Clear Logs

$this->clearLogs(); // Clears log files

Test Lifecycle

ForgeTesting provides lifecycle hooks for test setup and teardown.

Setup Method

The setup() method runs before each test:

#[BeforeEach]
public function setup(): void
{
    if (self::$kernel === null) {
        $bootstrap = Bootstrap::getInstance();
        self::$kernel = $bootstrap->getKernel();
    }
}

Teardown Method

The tearDown() method runs after each test:

#[AfterEach]
public function tearDown(): void
{
    // Cleanup after each test
}

Kernel Initialization

The kernel is initialized once and shared across all tests for performance:

protected static ?Kernel $kernel = null;

#[BeforeEach]
public function setup(): void
{
    if (self::$kernel === null) {
        $bootstrap = Bootstrap::getInstance();
        self::$kernel = $bootstrap->getKernel();
    }
}

Data Providers

Use data providers to run the same test with multiple data sets.

Using Data Providers

#[DataProvider("userProvider")]
#[Test]
#[Group("Database")]
public function multiple_users(array $users): void
{
    $this->assertArrayHasKey("email", $users);
}

public function userProvider(): array
{
    $users = $this->queryBuilder
        ->reset()
        ->setTable("users")
        ->select("*")
        ->limit(10)
        ->get(null);

    $dataProvider = [];
    foreach ($users as $user) {
        $dataProvider[] = [["email" => $user["email"]]];
    }

    return $dataProvider;
}

The test runs once for each data set returned by the provider method.

Test Dependencies

Use #[Depends] to ensure tests run in order.

Defining Dependencies

#[Test]
public function create_user(): void
{
    // Create user
}

#[Test]
#[Depends("create_user")]
public function update_user(): void
{
    // create_user() runs first
    // Then update_user() runs
}

Skipping and Incomplete Tests

Mark tests as skipped or incomplete with reasons. These appear in test results.

Skipping Tests

#[Test("User login functionality")]
#[Skip("Waiting on implementation")]
public function login_works(): void
{
    $this->assertTrue(true);
}

#[Test]
#[Group("smtp")]
#[Skip("Waiting on SMTP implementation")]
public function password_reset_email(): void
{
    // Test implementation
}

Skipped tests appear in test results:

Skipped Tests:
App\Modules\ForgeAuth\Tests\AuthenticationTest::login_works (0 ms)
Reason: Waiting on implementation

Incomplete Tests

#[Test("Insert a new record in the Database")]
#[Group("Database")]
#[Incomplete("Needs to check save performance in the model")]
public function create_user(): void
{
    $user = new User();
    $user->identifier = $this->exampleUser["identifier"];
    $user->save();
    $this->assertNotNull($user->id);
}

Incomplete tests appear in test results:

Incomplete Tests:
App\Modules\ForgeAuth\Tests\AuthenticationTest::create_user
Reason: Needs to check save performance in the model

CLI Commands

Run tests using the test command with various options.

Basic Commands

# Run all app tests
php forge.php test

# Run app tests explicitly
php forge.php test --type=app

# Run engine tests
php forge.php test --type=engine

# Run all module tests
php forge.php test --type=module

# Run specific module tests
php forge.php test --type=module --module=ForgeAuth

Group Filtering

# Run only Database group tests
php forge.php test --group=Database

# Run only http group tests
php forge.php test --group=http

# Run only validator group tests
php forge.php test --group=validator

Wizard Support

The test command supports interactive wizard mode for selecting test types and modules.

Test Caching

ForgeTesting uses directory hash-based caching to speed up test discovery.

How It Works

• Cache file: storage/framework/cache/test_cache.php
• Cache TTL: 3600 seconds (1 hour)
• Directory hashes are calculated from all test files
• Cache is invalidated when file hashes change
• Test classes are cached to avoid repeated file scanning

Cache Structure

return [
    'meta' => [
        'hashes' => [
            '/path/to/app/tests/' => 'abc123...',
            '/path/to/engine/tests/' => 'def456...',
        ],
        'timestamp' => 1234567890,
    ],
    'classes' => [
        'App\Tests\HomeTest',
        'Forge\tests\ValidatorTest',
        // ... more test classes
    ],
];

Performance Benefit

Test caching significantly speeds up test discovery on subsequent runs, especially with large test suites.

Test Results

ForgeTesting provides comprehensive test result reporting with detailed metrics.

Summary Table

Test Results:
| Total | Passed | Failed | Skipped | Incomplete |
+-------+--------+--------+---------+------------+
| 4     | 4      | 0      | 0       | 0          |

Slowest Tests

Top 5 slowest tests are displayed:

Slowest Tests:
| Test                                                                            | Duration |
+---------------------------------------------------------------------------------+----------+
| App\Tests\HomeTest::home_route_is_ok                                            | 14.40 ms | 
| App\Tests\HomeTest::register_route_returns_redirect_on_validation_error         | 1.00 ms  |

Benchmark Results

Benchmark Results:

App\Modules\ForgeAuth\Tests\AuthenticationTest::benchmark_user_lookup:
| Avg Time/Iter | Min Time/Iter | Max Time/Iter | Total Time |
+---------------+---------------+---------------+------------+
| 0.388 ms      | 0.388 ms      | 0.388 ms      | 0.388 ms   |

Passed Tests

Passed Tests:
| Test                                              | Class              | Method                                                      | Duration |
+---------------------------------------------------+--------------------+-------------------------------------------------------------+----------+
| Home / route is working                           | App\Tests\HomeTest | home_route_is_ok                                            | 14.40 ms |

Failure Details

Failed tests show detailed error information including file and line number.

Skipped and Incomplete Tests

Skipped and incomplete tests are listed with their reasons.

Execution Time

Tests completed in 0.02s

Usage Examples

Complete examples from the Forge Kernel codebase.

Simple HTTP Test

#[Group("http")]
final class HomeTest extends TestCase
{
    #[Test("Home / route is working")]
    public function home_route_is_ok(): void
    {
        $response = $this->get("/");
        $this->assertHttpStatus(200, $response);
    }

    #[Test("POST / with invalid data redirects back")]
    public function register_route_returns_redirect_on_validation_error(): void
    {
        $response = $this->post(
            "/",
            $this->withCsrf([
                "email" => "invalid-email",
                "password" => "123",
            ]),
        );

        $this->assertHttpStatus(302, $response);
    }
}

Database Test with Performance

#[Group("auth")]
final class AuthenticationTest extends TestCase
{
    #[Test("Check a record exists in the Database by identifier")]
    #[Group("Database")]
    public function user_exists(): void
    {
        $this->assertDatabaseHas("users", [
            "identifier" => $this->exampleUser["identifier"],
        ]);
    }

    #[Test("Benchmark user lookup")]
    #[Group("Database")]
    public function benchmark_user_lookup(): array
    {
        $results = $this->benchmark(function () {
            $this->assertDatabaseHas("users", [
                "email" => $this->exampleUser["email"],
            ]);
        }, 1);

        return $results;
    }
}

Engine Test

#[Group('validator')]
final class ValidatorTest extends TestCase
{
    #[Test('Validate required rule')]
    public function validate_required_rule(): void
    {
        $isValid = false;
        try {
            $data = ["identifier" => "juan"];
            $rules = ['identifier' => ["required"]];
            $this->validate($data, $rules);
            $isValid = true;
        } catch (ValidationException) {
            $isValid = false;
        }
        $this->assertTrue($isValid);
    }

    #[Test('Validate validation failing')]
    public function validate_should_fail(): void
    {
        $this->shouldFail(function () {
            $data = ["identifier" => ""];
            $rules = ['identifier' => ["required"]];
            $this->validate($data, $rules);
        });
    }
}

Best Practices

Recommendations for writing effective tests with ForgeTesting.

Test Organization

When to Use Skip vs Incomplete