Initial prototype of test refactor w/ runner, helper and test layers

.github/workflows/build-and-test.yml

@@ -32,6 +32,9 @@ jobs:
         repository: wolfssl/wolfssl
         path: wolfssl
 
+    - name: Build and test refactor
+      run: cd test-refactor/posix && make clean && make -j WOLFSSL_DIR=../../wolfssl && make run
+
     # Build and test standard build
     - name: Build and test
       run: cd test && make clean && make -j WOLFSSL_DIR=../wolfssl && make run

test-refactor/README.md

@@ -0,0 +1,136 @@
+# test-refactor
+
+Prototype of the refactored wolfHSM test infrastructure.
+
+## Key differences from test/
+
+- **Runner** (`wh_test_runner.h/c`): generic suite executor.
+  Each suite is a name + NULL-terminated test array, run
+  either via `whTestRunner_Run` (suite owns its setup/cleanup)
+  or `whTestRunner_RunWithCtx` (caller provides the live
+  context). Group functions use the latter.
+- **App-owned init**: the port's main() brings up the server
+  and client once at startup (mirroring real firmware boot)
+  and hands the live contexts to the group functions. Suites
+  no longer stand up their own fixtures.
+- **Port helpers** (`wh_test_helpers_server_<port>.h/c`,
+  `wh_test_helpers_client_<port>.h/c`): per-port files that
+  stand in for what a real target does at boot -- configure
+  flash, init NVM/crypto, wire up a transport, bring up the
+  server or client context.
+- **Groups** (`wh_test_groups.h/c`): three portable entry
+  points (Misc/Server/Client) that main invokes. Each runs
+  its gated suites and calls the caller-implemented reset
+  hook between them.
+- **Threaded driver**: the POSIX port's main runs the server
+  and client on separate threads. The server thread runs the
+  server-only group first, then enters a `HandleRequestMessage`
+  loop; the client thread runs the client-only group against
+  the live server. Ports that already split server and client
+  onto different cores/tasks do the same thing natively.
+- **Platform split**: platform-specific code is isolated in
+  `wh_test_helpers_server_<port>.c`,
+  `wh_test_helpers_client_<port>.c`, and
+  `wh_test_main_<port>.c`. Test modules and groups are
+  identical on all platforms.
+
+## Suites implemented so far
+
+| Suite | Group | Description |
+|-------|-------|-------------|
+| Flash RamSim | misc | Write-lock, erase, program, verify, blank-check |
+| NVM Flash | misc | Flash unit ops, NVM add/overwrite/destroy/reclaim |
+| Cert | server | Server-side cert add/verify/chain/erase |
+| ClientServer | client-server | Echo round-trip, server info query |
+| ThreadSafe Stress | client-server | Phased multi-thread contention (unchanged internals) |
+
+## Remaining tests to port
+
+| Suite | Group | Description |
+|-------|-------|-------------|
+| Comm | client-server | Transport layer (mem, TCP, SHM) |
+| Crypto | client-server | AES, RSA, ECC, CMAC, curve25519, ed25519, etc. |
+| Crypto Affinity | client-server | Device ID operation routing |
+| SHE | client-server | Secure Hardware Extension key load, crypto, secure boot |
+| Keywrap | client-server | Key wrap/unwrap operations |
+| Log | misc | Logging frontend, ringbuf, POSIX file backends |
+| Lock | misc | Lock primitives with POSIX backend |
+| DMA | misc | DMA address translation and allow-list |
+| Server Img Mgr | server | Image manager verify/install/erase |
+| Timeout | client-server | POSIX timeout enforcement |
+| wolfCrypt Test | client-server | wolfCrypt test suite via wolfHSM transport |
+| MultiClient | client-server | 2 CS pairs, shared NVM, global/local key isolation |
+
+## Platforms requiring update
+
+Each platform with test infrastructure needs its own
+`wh_test_helpers_server_<port>.c`,
+`wh_test_helpers_client_<port>.c`, and
+`wh_test_main_<port>.c` (see "Porting" below).
+
+| Platform | Vendor | Test files |
+|----------|--------|------------|
+| POSIX | wolfSSL | `test-refactor/posix/` (done) |
+| Bernina | STMicro | `bernina-server/src/bh_test.c` |
+| SR6 | STMicro | (no test files found) |
+| TC3xx | Infineon | `port/client/wolfhsm_tests.c`, `port/server/ccb_tests.c` |
+| RH850 F1KM | Renesas | `rh850_test2_1/`, `rh850_test2_2/` |
+| PIC32CZ | Microchip | `czhsm-client/tests/`, `czhsm-server/` |
+| TDA4VH | TI | (no test files found) |
+| New Eagle | Customer | (no test files found) |
+
+## File layout
+
+```
+Portable (ships in wolfHSM):
+  wh_test_runner.h/c         - suite runner
+  wh_test_groups.h/c         - Misc/Server/Client entry points
+  server/wh_test_*.c/h       - server-only test modules
+  client-server/wh_test_*.c/h - client-server test modules
+  misc/wh_test_*.c/h         - standalone test modules
+
+Platform-specific (one directory per platform, e.g. posix/):
+  <port>/wh_test_helpers_misc_<port>.h/c   - misc fixtures
+  <port>/wh_test_helpers_server_<port>.h/c - server bringup
+  <port>/wh_test_helpers_client_<port>.h/c - client bringup
+  <port>/wh_test_main_<port>.c             - init, group
+                                             dispatch, reset
+                                             hooks
+  <port>/Makefile                          - build rules
+```
+
+## Porting to other platforms
+
+1. Implement the init helpers for the side(s) the target
+   needs. These stand in for what your firmware's normal
+   boot flow already does -- if it's simpler to call your
+   existing init code directly from main, that works too:
+   - `whTestHelperPosix_Server_Init/Cleanup` (reference):
+     bring up flash/NVM/crypto/transport/server.
+   - `whTestHelperPosix_Client_Init/Cleanup` (reference):
+     bring up client comm + handshake. On single-process
+     targets, the server runs in its own thread and pumps
+     `HandleRequestMessage` itself.
+2. Provide a `main()` that:
+   - Calls `whTestGroup_Misc()` for standalone tests.
+   - Brings up the server/client contexts once.
+   - Calls `whTestGroup_Server(&server)` and/or
+     `whTestGroup_Client(&client)` with the live handles.
+   - Tears the contexts down.
+   - Implements `whTestGroup_ResetServer` and/or
+     `whTestGroup_ResetClient` -- called between suites to
+     scrub persistent state.
+3. Add the portable `.c` files and your port files to your
+   build system.
+
+See `wh_test_main_posix.c` and the two `*_posix.c` helpers as
+a reference implementation.
+
+## Build and run (POSIX)
+
+```
+cd posix
+make run
+make run DEBUG=1
+make run THREADSAFE=1    # enables stress test gate
+```

test-refactor/client-server/wh_test_crypto.c

@@ -0,0 +1,178 @@
+/*
+ * Copyright (C) 2026 wolfSSL Inc.
+ *
+ * This file is part of wolfHSM.
+ *
+ * wolfHSM is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * wolfHSM is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with wolfHSM.  If not, see <http://www.gnu.org/licenses/>.
+ */
+/*
+ * test-refactor/wh_test_crypto.c
+ *
+ * Basic crypto test suite. Minimal SHA256 and AES-CBC
+ * round-trips routed through the server via WH_DEV_ID.
+ */
+
+#include "wolfhsm/wh_settings.h"
+
+#if !defined(WOLFHSM_CFG_NO_CRYPTO)
+
+#include <stdint.h>
+#include <string.h>
+
+#include "wolfssl/wolfcrypt/settings.h"
+#include "wolfssl/wolfcrypt/types.h"
+#include "wolfssl/wolfcrypt/aes.h"
+#include "wolfssl/wolfcrypt/sha256.h"
+
+#include "wolfhsm/wh_error.h"
+#include "wolfhsm/wh_client.h"
+
+#include "wh_test_common.h"
+#include "wh_test_runner.h"
+#include "wh_test_crypto.h"
+
+#ifndef NO_SHA256
+static int whTest_CryptoSha256(void* c)
+{
+    whClientContext* ctx   = (whClientContext*)c;
+    int              devId = WH_DEV_ID;
+    int              ret   = WH_ERROR_OK;
+    wc_Sha256        sha256[1];
+    uint8_t          out[WC_SHA256_DIGEST_SIZE];
+    /* Vector exactly one block size in length */
+    const char inOne[] =
+        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
+    const uint8_t expectedOutOne[WC_SHA256_DIGEST_SIZE] = {
+        0xff, 0xe0, 0x54, 0xfe, 0x7a, 0xe0, 0xcb, 0x6d, 0xc6, 0x5c, 0x3a,
+        0xf9, 0xb6, 0x1d, 0x52, 0x09, 0xf4, 0x39, 0x85, 0x1d, 0xb4, 0x3d,
+        0x0b, 0xa5, 0x99, 0x73, 0x37, 0xdf, 0x15, 0x46, 0x68, 0xeb};
+
+    (void)ctx;
+
+    /* Initialize SHA256 structure */
+    ret = wc_InitSha256_ex(sha256, NULL, devId);
+    if (ret != 0) {
+        WH_ERROR_PRINT("Failed to wc_InitSha256 on devId 0x%X: %d\n", devId,
+                ret);
+    } else {
+        /* Single-block update should trigger a server transaction */
+        ret = wc_Sha256Update(sha256,
+                (const byte*)inOne,
+                WC_SHA256_BLOCK_SIZE);
+        if (ret != 0) {
+            WH_ERROR_PRINT("Failed to wc_Sha256Update %d\n", ret);
+        } else {
+            /* Finalize should trigger a server transaction with empty buffer */
+            ret = wc_Sha256Final(sha256, out);
+            if (ret != 0) {
+                WH_ERROR_PRINT("Failed to wc_Sha256Final %d\n", ret);
+            } else {
+                /* Compare the computed hash with the expected output */
+                if (memcmp(out, expectedOutOne, WC_SHA256_DIGEST_SIZE) != 0) {
+                    WH_ERROR_PRINT("SHA256 hash does not match expected.\n");
+                    ret = -1;
+                }
+            }
+        }
+        (void)wc_Sha256Free(sha256);
+    }
+    if (ret == 0) {
+        WH_TEST_PRINT("SHA256 DEVID=0x%X SUCCESS\n", devId);
+    }
+    return ret;
+}
+#endif /* !NO_SHA256 */
+
+
+#if !defined(NO_AES) && defined(HAVE_AES_CBC)
+static int whTestCrypto_Aes(void* c)
+{
+    whClientContext* ctx   = (whClientContext*)c;
+    int              devId = WH_DEV_ID;
+    int              ret   = 0;
+    Aes              aes[1];
+    uint8_t          cipher[AES_BLOCK_SIZE]   = {0};
+    uint8_t          plainOut[AES_BLOCK_SIZE] = {0};
+    /* NIST SP 800-38B test vectors (same k128 / m used by the CMAC test
+     * in test/wh_test_crypto.c). Using a fixed vector keeps this suite
+     * self-contained, no RNG needed. */
+    const uint8_t key[] = {0x2b, 0x7e, 0x15, 0x16, 0x28, 0xae, 0xd2, 0xa6,
+                           0xab, 0xf7, 0x15, 0x88, 0x09, 0xcf, 0x4f, 0x3c};
+    const uint8_t iv[AES_BLOCK_SIZE] = {
+        0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+        0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f};
+    const uint8_t plainIn[AES_BLOCK_SIZE] = {
+        0x6b, 0xc1, 0xbe, 0xe2, 0x2e, 0x40, 0x9f, 0x96,
+        0xe9, 0x3d, 0x7e, 0x11, 0x73, 0x93, 0x17, 0x2a};
+
+    (void)ctx;
+
+    /* test aes CBC with client side key */
+    ret = wc_AesInit(aes, NULL, devId);
+    if (ret != 0) {
+        WH_ERROR_PRINT("Failed to wc_AesInit %d\n", ret);
+    } else {
+        ret = wc_AesSetKey(aes, key, sizeof(key), iv, AES_ENCRYPTION);
+        if (ret != 0) {
+            WH_ERROR_PRINT("Failed to wc_AesSetKey %d\n", ret);
+        } else {
+            ret = wc_AesCbcEncrypt(aes, cipher, plainIn,
+                    sizeof(plainIn));
+            if (ret != 0) {
+                WH_ERROR_PRINT("Failed to wc_AesCbcEncrypt %d\n", ret);
+            } else {
+                ret = wc_AesSetKey(aes, key, sizeof(key), iv,
+                        AES_DECRYPTION);
+                if (ret != 0) {
+                    WH_ERROR_PRINT("Failed to wc_AesSetKey %d\n", ret);
+                } else {
+                    ret = wc_AesCbcDecrypt(aes, plainOut, cipher,
+                            sizeof(cipher));
+                    if (ret != 0) {
+                        WH_ERROR_PRINT("Failed to wc_AesCbcDecrypt %d\n",
+                                ret);
+                    } else {
+                        if (memcmp(plainIn, plainOut, sizeof(plainIn)) !=
+                            0) {
+                            WH_ERROR_PRINT("Failed to match AES-CBC\n");
+                            ret = -1;
+                        }
+                    }
+                }
+            }
+        }
+        (void)wc_AesFree(aes);
+    }
+    if (ret == 0) {
+        WH_TEST_PRINT("AES CBC DEVID=0x%X SUCCESS\n", devId);
+    }
+    return ret;
+}
+#endif /* !NO_AES && HAVE_AES_CBC */
+
+
+static whTestFn _tests[] = {
+#ifndef NO_SHA256
+    whTest_CryptoSha256,
+#endif
+#if !defined(NO_AES) && defined(HAVE_AES_CBC)
+    whTestCrypto_Aes,
+#endif
+    NULL
+};
+
+whTestSuite whTestSuite_Crypto =
+    WH_TEST_SUITE("Crypto", _tests);
+
+#endif /* !WOLFHSM_CFG_NO_CRYPTO */

test-refactor/client-server/wh_test_crypto.h

@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2026 wolfSSL Inc.
+ *
+ * This file is part of wolfHSM.
+ *
+ * wolfHSM is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * wolfHSM is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with wolfHSM.  If not, see <http://www.gnu.org/licenses/>.
+ */
+/*
+ * test-refactor/wh_test_crypto.h
+ *
+ * Basic crypto test suite (minimal SHA256 round-trip).
+ */
+
+#ifndef WH_TEST_CRYPTO_REFACTOR_H_
+#define WH_TEST_CRYPTO_REFACTOR_H_
+
+#include "wolfhsm/wh_settings.h"
+
+#if !defined(WOLFHSM_CFG_NO_CRYPTO)
+
+#include "wh_test_runner.h"
+
+extern whTestSuite whTestSuite_Crypto;
+
+#endif /* !WOLFHSM_CFG_NO_CRYPTO */
+
+#endif /* WH_TEST_CRYPTO_REFACTOR_H_ */