Test Run

A Test Run is the process of executing your defined tests. It involves running a selected Test Plan, which includes a collection of configured Test Cases, on a designated Bench and against the target. This action puts your test strategy into motion to verify the DUT’s behavior.

Executing a Test Run

1. Sign-in to vsec.blockharbor.io

2. Navigate to Test on the sidebar menu to enter the VSEC Test application and then select the Test Runs submenu.

3. Click the Create Test Run button to open the prompt for creating a New Test Run.

4. Fill out the required details to Create a Test Run.

   Field	Requirement	Description
   Bench *	Required	Select the Bench to execute the testing through. This can be a physical Bench or a Simulation Bench.
   Test plan *	Required	Select the Test Plan to execute.
   Test name *	Required	A unique, short descriptor (e.g., MY26 CAN Protocol Evaluation).
   Description	Optional	Internal notes or context regarding the purpose of this Test Run.

5. Click the Create button to execute / queue the Test Run.

Running Tests on a Simulation Bench

Test Runs can be executed against a Simulation Bench instead of a physical Bench. Once selected, the Test Run executes the same way as on a physical Bench.

1. Click the Create Test Run button to open the prompt for creating a New Test Run.
2. In the Bench select box, choose the desired Simulation (e.g., UDS and XCP Starter Challenge).
3. Select the Test plan to execute.
4. Specify a Test name.
5. Click the Create button to execute / queue the Test Run.

Exporting Test Run Results

1. Navigate to the Test Runs submenu.
2. Select the desired Test Run by clicking the view icon ( ) on the corresponding row.
3. Details and Test Case results will be displayed with a passed / failed / error status.
4. Scroll to the bottom of the Test Run results and click one of these buttons to export:
   • Download log - A zip file showing details relating to any errors on the Test Cases (e.g., vehicle-my-2026_run.zip).
   • Download report - A report-style PDF with Pass / Fail stats and descriptions (e.g., vehicle-my-2026_report_3.pdf).

Viewing Test Case Parameters

In a Test Run’s details, the Test Cases table includes a Parameters column listing the parameter name/value pairs recorded for that execution (or No parameters recorded if none were captured).

Comparing Test Runs

1. Open a Test Run’s details as described above.
2. Click the Compare button next to the download buttons.
3. Choose the Test Run to compare against from the Compare against… dropdown.
4. The Test Cases table switches to a comparison view with columns for Case, Interface, Status, Execution time, Results, and Parameter changes. Rows where the status differs between the two runs are highlighted, and any changed value is shown alongside the other run’s value. A Test Case that only ran in one of the two runs is flagged as only present in that run.
5. Click Exit comparison (shown as Cancel comparison before a target run is chosen) to return to the normal results view.

Deleting Test Run Results

1. Navigate to the Test Runs submenu.
2. Click the trash can icon ( ) to delete the Test Run.
3. A confirmation window will pop-up to confirm deletion. Click the Delete button to delete the Test Run.