feat: Add bulk ZIP export (#867) by stephanbuettig · Pull Request #222 · httptoolkit/httptoolkit-ui

stephanbuettig · 2026-04-11T06:43:23Z

Adds ZIP archive export for HTTP exchanges with 37 code snippet formats via @httptoolkit/httpsnippet. Includes format picker panel, Web Worker generation, and safe filename conventions.

Features:

ZIP export with selectable snippet formats (37 languages/clients)
Format picker with category grouping and popular defaults
Web Worker-based generation for non-blocking UI
Safe filename conventions matching existing HAR export pattern

New files: snippet-formats registry, export-filenames utility, download helper, zip-metadata model, zip-download-panel component.

Unit tests for snippet-formats and export-filenames included.

Extracted from #219 as requested by @pimterry.

@pimterry

Adds ZIP archive export for HTTP exchanges with 37 code snippet formats via @httptoolkit/httpsnippet. Includes format picker panel, Web Worker generation, and safe filename conventions. Features: - ZIP export with selectable snippet formats (37 languages/clients) - Format picker with category grouping and popular defaults - Web Worker-based generation for non-blocking UI - Safe filename conventions matching existing HAR export pattern New files: snippet-formats registry, export-filenames utility, download helper, zip-metadata model, zip-download-panel component. Unit tests for snippet-formats and export-filenames included. Extracted from httptoolkit#219 as requested by @pimterry.

stephanbuettig · 2026-04-11T08:00:40Z

✅ Manual Test Results — 2026-04-11

Both features were tested against a fresh clone of current upstream (main, commit 23a99520) using npm start.

ZIP Export (this PR)

All runs completed with 0 snippet errors:

Scenario	Formats	Snippets	Time	Size
Single request, all formats	37	37	87 ms	67 KB
Single large request, all formats	37	37	290 ms	156 KB
Single request, 7 formats	7	7	45 ms	8.5 KB

The ZIP download panel opens correctly, format selection persists across sessions, and the generated archives are valid and well-structured.

Batch export (PR #223, depends on this PR)

Scenario	Formats	Snippets	Errors	Time	Size
13 exchanges, 7 formats	7	91	0	254 ms	152 KB
14 exchanges, all formats	37	518	0	2.1 s	795 KB

Both features are production-ready and work correctly on the current upstream codebase. Ready for review and merge.

pimterry

I haven't gone through everything in detail yet, especially the actual UI component code, and I haven't tested this manually either, but I think this is a good set of bits to start with from a quick review. There's a strong outline here but there's going to be a good bit of work to properly integrate this into the codebase and make it maintainable for the future.

pimterry · 2026-04-13T22:35:47Z

+};
+
+// Build extended optGroups with ZIP at the top
+const exportOptionsWithZip: _.Dictionary<SnippetOption[]> = {


You don't need to import the whole of lodash for this one type, you can just use Record<string, SnippetOption[]>.

There are probably old examples of doing that elsewhere here - they predate Record being added to typescript, if you spot any along the way feel free to clean them up 😄.

pimterry · 2026-04-13T22:37:41Z

+ *
+ * Contains ALL available HTTPSnippet targets/clients organized by language
+ * category. The ZIP export pipeline, format picker UI, and batch toolbar
+ * all consume this registry.


Why is this here? HTTPSnippet already has a registry, we don't want to duplicate it, since the options available will change and this will get out of date very quickly.

pimterry · 2026-04-13T22:38:09Z

+
+    @action.bound
+    setZipFormatIds(ids: ReadonlySet<string> | string[]) {
+        this._zipFormatIds = Array.isArray(ids) ? [...ids] : [...ids];


This ternary does nothing? Both options are the same.

pimterry · 2026-04-13T22:41:51Z

+                metadata
+            } as Omit<GenerateZipRequest, 'id'>));
+        } catch (err) {
+            // postMessage can throw for unserializable data (MobX proxies, etc.)


When does this happen? We probably shouldn't catch this - we need to fix that instead. MobX proxies need to be filtered out etc. Otherwise we'll find that the export doesn't work in lots of common cases and data will silently go missing, which is a big problem.

Do you have any examples? Normally it's best to let this fail hard instead - that way it's easy to spot these issues in testing, and they'll show up in the Sentry error reports for debugging & tracking later as well.

Imo it's good to catch errors when they're due to expected issues like bad data or user input or configuration. If it can be due to implementation problems, we need to make sure that's very visible now in testing (crashing the app on purpose) and that we then fix all the problems to handle it.

pimterry · 2026-04-13T22:47:00Z

+        };
+
+        // Safety timeout: if the worker doesn't respond within 5 minutes,
+        // clean up the listener to prevent memory leaks.


This doesn't really make sense, it doesn't look like handler has any references that would cause leaks here, and this doesn't actually cancel the processing, it leaves it going indefinitely. If there's a real risk it could take this long it's a problem because it will block the worker completely.

That means every other worker operation (like decoding any compressed request or response body) will wait until this is finished. If these are that slow we'll need to implement actual cancellation, look into abort controllers for how signals for that kind of thing can work.

pimterry · 2026-04-13T22:47:27Z

+                harEntries,
+                formats,
+                metadata
+            } as Omit<GenerateZipRequest, 'id'>));


Why does this use a custom wrapper instead of callApi? I'd much prefer to keep everything using the same abstraction if we can. It doesn't have progress of course, or any abort support, but both could be added there instead of here, and then that would work for all worker calls which would be great.

pimterry · 2026-04-13T22:50:15Z

+                                    ),
+                                    cookies: [], // Included in headers already
+                                    ...(postData !== undefined ? { postData } : {})
+                                };


Don't we already have this same preprocessing logic in the normal export snippet generation? Is this intentionally different for some reason? I would've expected to just reuse that. Likely to give better results for users, since it guarantees the content you see in the Export card is the same thing per-request you see in the batch zip export.

pimterry · 2026-04-13T22:54:06Z

+ *   - HAR batch:   "HTTPToolkit_export_{date}_{count}-requests.har"
+ *   - ZIP archive: "HTTPToolkit_{date}_{count}-requests.zip"
+ *   - Snippet:     "{index}_{METHOD}_{STATUS}_{hostname}.{ext}"
+ */


Instead of just referencing the other name patterns elsewhere, we could just move the logic for all of these into here.

pimterry · 2026-04-13T23:00:21Z

+            expect(result.length).to.equal(1);
+        });
+    });
+


These tests just assert on a selection of hardcoded data values, which doesn't seem very useful.

I think this fail at least could probably just go away (along with the hardcoded data itself).

It would be useful to have a proper end to end test of zip generation though. We should be able to do that, we do similar testing including worker API calls in https://github.com/httptoolkit/httptoolkit-ui/blob/main/test/unit/workers/worker-decoding.spec.ts.

Doesn't need to cover every possible edge cases there, we just need a basic covering of the overall key flows to make sure the structure and key behaviours work. That's probably just the success case with a couple of examples and an error case (if there are scenarios where we expect this to fail). You can then use fflate in the test to check the expected output appears. No need to test on specific code snippet contents for specific inputs or anything like that (that's covered in a lot of detail already by httpsnippet's own tests) just that the whole flow glues together correctly.

pimterry · 2026-04-13T23:05:48Z

+// This is never passed to httpsnippet — it's only used for dropdown rendering.
+const ZIP_SNIPPET_OPTION: SnippetOption = {
+    target: ZIP_ALL_FORMAT_KEY as any,
+    client: '' as any,


These any are a bit suspicious. I think we should have some kind of better types that make this work properly, without this and witohut an extra special case functions like getExportFormatKey that just wraps getCodeSnippetFormatKey with one extra condition. We probably want something like export ExportOption = ZipExportOption | SnippetOption somewhere with some kind of discriminated union, and then to change the various references take either ExportOption or SnippetOption as appropriate, and discriminate to make all those types work correctly, without any.

@pimterry

Changes addressing @pimterry's review: Architecture: - DI pattern for ZipExportController (testable without module stubs) - Shared sanitizer: simplifyHarEntryRequestForSnippetExport (single source of truth for snippet export sanitization, used by both single-export and ZIP-export paths) - Cooperative cancellation via MessageChannel + yieldToEventLoop() - callApi abort now rejects immediately (no 5-min hang if worker is stuck before first yield); exportAsZip translates AbortError back to cancelled response to preserve the public API contract Bug fixes: - Cancellation race: abortListener in callApi now calls finalize() + reject(AbortError) immediately, matching timeout handler behavior - Listener ordering: emitter.once registered before worker.postMessage to prevent latent race with synchronous worker responses - Type safety: replaced `undefined as any` with conditional spread in buildUltraSafeRequest Code quality: - All comments and debug logs translated to English (upstream PR ready) - formatBytes JSDoc corrected (SI-style labels, not IEC) - ZIP_DEBUG flag defaults to false Tests: - New: snippet-export-sanitization.spec.ts (hop-by-hop headers, empty query params, cookie clearing, postData.text preservation) - New: zip-export-service.spec.ts (stale-run invalidation, reset during in-flight run) - All 808 tests pass, 0 failures

@action

…-pane Upstream refactored the class component to a functional component. Adapted ZIP export features (dialog state, export button, ZipExportDialog) to the new functional component pattern using React.useState instead of MobX @observable/@action decorators. PinIcon uses upstream's styled(Icon) .attrs() approach.

pimterry · 2026-04-23T13:57:41Z

The package-lock update wasn't commit, so which breaks the build here. Also it seems the end of export-filenames.ts is full of broken binary data so can't even be parsed.

Several source files contained trailing NULL bytes introduced by the cross-filesystem copy (Windows CIFS mount → Linux). This caused export-filenames.ts and others to be detected as binary, breaking the build. Affected files cleaned: export-filenames.ts, zip-export-formats.ts, export.ts, http-export-card.tsx, zip-export.spec.ts, ui-store.ts. Also adds the missing package-lock.json update for the fflate dependency added in package.json.

The previous null-byte cleanup (9210cb9) used tr -d which stripped null bytes but also truncated content that followed embedded nulls. 6 files were affected and are now restored from the verified local copy (808 tests passing): - ui-worker-api.ts: 288→321 lines (exportAsZip body was missing) - ui-worker.ts: 845→858 lines - zip-export-dialog.tsx: 851→864 lines - zip-export-service.ts: 371→393 lines - zip-manifest.ts: 77→79 lines - zip-export-service.spec.ts: 133→136 lines

- Reset executable bit (100755 → 100644) on three source files that were incorrectly marked during cross-filesystem copy - Remove trailing whitespace on final line of zip-export-dialog.tsx

stephanbuettig · 2026-04-23T20:15:35Z

Hi @pimterry, thanks for the review and sorry about the broken files. The issue was that I copied the source files from a Windows workspace into a Linux environment, which introduced null bytes into several files and dropped the package-lock. Took a few rounds to get everything clean, but it's all sorted now.

What changed in this update:

All 10 review points are addressed. The biggest structural changes: snippet preprocessing is now a shared function (simplifyHarEntryRequestForSnippetExport) used by both single-export and ZIP paths, the custom worker wrapper is gone in favor of extending callApi with optional abort/progress/timeout support, the hardcoded format registry is replaced by runtime derivation from HTTPSnippet, and cancellation is cooperative via MessageChannel instead of just a timeout. The test suite has proper worker round-trip tests modeled after worker-decoding.spec.ts.

Build issues (missing package-lock, file encoding, wrong permissions) are all fixed. 808 tests pass, 0 failures.

Happy to adjust anything!

Upstream bumped fast-xml-parser to ^5.7.0. Kept our fflate ^0.8.2 addition and regenerated package-lock.json.

socket-security · 2026-04-23T20:20:10Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	fflate@0.4.8 ⏵ 0.8.2	^-1		⁺¹

View full report

This was referenced Apr 11, 2026

feat: Add multi-select batch export (#76) #223

Closed

feat: Add ZIP export and batch selection (#76, #867) #219

Closed

pimterry reviewed Apr 13, 2026

View reviewed changes

pimterry mentioned this pull request Apr 17, 2026

Multi selection for intercepted request httptoolkit/httptoolkit#76

Closed

stephanbuettig added 2 commits April 18, 2026 23:21

pimterry force-pushed the main branch 2 times, most recently from 95f7fd7 to 4e6711d Compare April 21, 2026 19:38

stephanbuettig added 3 commits April 23, 2026 19:34

chore: fix file permissions and trailing whitespace

7b2e958

- Reset executable bit (100755 → 100644) on three source files that were incorrectly marked during cross-filesystem copy - Remove trailing whitespace on final line of zip-export-dialog.tsx

merge: resolve package.json/package-lock.json conflicts with upstream

a69c569

Upstream bumped fast-xml-parser to ^5.7.0. Kept our fflate ^0.8.2 addition and regenerated package-lock.json.

Conversation

stephanbuettig commented Apr 11, 2026

Uh oh!

stephanbuettig commented Apr 11, 2026

✅ Manual Test Results — 2026-04-11

ZIP Export (this PR)

Batch export (PR #223, depends on this PR)

Uh oh!

pimterry left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pimterry commented Apr 23, 2026

Uh oh!

stephanbuettig commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

socket-security Bot commented Apr 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stephanbuettig commented Apr 23, 2026 •

edited

Loading