Commit Graph

36 Commits

Author SHA1 Message Date
Siddharth Balyan
55c9f32060
fix(tui): width-aware markdown table rendering with vertical fallback (#26195)
* refactor(tui): thread cols through Md/StreamingMd/renderTable, update cache key

* feat(tui): three-tier width calc + full-line string rendering in renderTable

Replaces the old renderTable (L203-244) with:
- Empty table guard
- Ragged row normalization
- Three-tier column width calculation (ideal → proportional shrink → hard scale)
- Rounding remainder distribution
- Full-line string rendering (one <Text> per row, not per cell)
- wrap=truncate-end on all table lines
- All cells rendered as plain text via stripInlineMarkup

No wrapping or vertical fallback yet — those come in Phase 3 and 4.

* feat(tui): wrapCell with grapheme-safe hard-break + multi-line row rendering

Adds:
- Intl.Segmenter-based grapheme splitting (fallback to [...word])
- wrapCell() for width-correct word wrapping on stripped text
- Multi-line row rendering with LineEntry metadata (header/separator/body)
- Post-render safety condition (maxLineWidth computed, vertical fallback in Task 4)
- Non-wrapping path preserved for tables that fit at ideal widths

* feat(tui): vertical key-value fallback with scaled threshold + safety check

Wires:
- Scaled row-height threshold (numCols<=3: 8, <=6: 5, else: 4)
- Post-render safety check (maxLineWidth > available space)
- Header-only edge case
- Vertical format: bold headers, stripped cell text, clamped separator width
- Iterates headers (not rows) for consistent key-value fields on ragged rows

* test(tui): pass cols to Md in test helpers, add width-overflow assertions

- renderAtWidth now passes cols={columns} to <Md> so width-aware code paths
  are exercised in tests
- tableFuzz: every rendered line must fit within allocated width (stringWidth)
- tableRepro: separator regex updated to match truncation ellipsis
- stringWidth imported from @hermes/ink for CJK-correct assertions

* fix(tui): address adversarial review — comment tier 3 budget overshoot, eliminate redundant wrapCell

- Add comment on Tier 3 MIN_COL_WIDTH clamp exceeding budget (self-heals via safetyOverflow)
- Track tallestBodyRow during allEntries build pass instead of re-wrapping every cell
  in a second traversal (eliminates O(cells) of redundant stripInlineMarkup+stringWidth)

* fix(tui): pass cols to recursive fenced-markdown Md, fix test frame extraction

- Thread cols into <Md> for fenced markdown blocks (L734) so nested
  tables use the width-aware renderer instead of max-content path
- Fix renderAtWidth helpers to extract final Ink repaint frame instead
  of concatenating all intermediate frames (REPAINT_RE split)
- Add fenced-markdown-table fixture to tableFuzz (exercises the nested path)

* chore: remove repro test suites and tmux driver script

These were scaffolding for development/reproduction — not needed in the PR.
2026-05-15 20:25:56 -05:00
brooklyn!
75b428c852
feat(ui-tui): resolve markdown links to readable page titles (#24013)
* feat(ui-tui): resolve links to readable page titles

Mirror desktop pretty-link behavior in the TUI by resolving HTTP links to page titles with shared caching and safe fetch filters, plus slug-based fallbacks so chat links stay readable even when title fetch fails.

* refactor(ui-tui): tighten link-title fallback handling

Clean up the link-title resolver by hardening in-flight cleanup and clarifying title length limits, while adding focused coverage for HTML entity decoding and markdown-label fallback behavior.

* fix(ui-tui): block private-network targets in title fetches

Prevent automatic link-title resolution from requesting local or private hosts by rejecting RFC1918, link-local, ULA, and intranet-style hostnames before fetch, and add regression coverage for blocked host patterns.
2026-05-11 14:16:31 -07:00
Teknium
1d00716754
fix(cli,tui): align CJK / wide-char markdown tables (#23863)
CJK and emoji glyphs render as two terminal cells but JS String#length
and the model's own padding count them as one, so any markdown table
with Chinese / Japanese / Korean cells drifts right per row when a
real terminal renders it. Both surfaces fix this with a display-cell
width measurement (wcswidth on the Python side, stringWidth on the
TUI side).

Changes:
- agent/markdown_tables.py: new helper. realign_markdown_tables(text)
  detects markdown table blocks (header + |---| divider) and
  rewrites the row padding using wcwidth.wcswidth so every pipe and
  dash lines up across rows. No-op on text without tables.
- cli.py: hook the helper into _render_final_assistant_content for
  strip / render modes (raw passes through untouched), and into the
  streaming line emitter so live token-by-token rendering also
  produces aligned tables. A small two-buffer state machine in
  _emit_stream_text holds table rows until the block ends, then
  flushes them through the realigner so all rows pad to a single
  per-column width.
- ui-tui/src/components/markdown.tsx: renderTable now uses
  stringWidth (Bun.stringWidth fast path + East-Asian-width-aware
  fallback, already memoised in @hermes/ink) instead of UTF-16
  String#length for both column-width measurement and per-cell
  padding. Drops the comment that documented the bug as a deliberate
  limitation.

Validation:
- New tests/agent/test_markdown_tables.py (11): every rebuilt block
  shares pipe column offsets across rows for pure CJK, mixed
  CJK+emoji, ragged-row, and multi-table inputs.
- Updated tests/cli/test_cli_markdown_rendering.py: the existing
  strip-mode test asserted exact whitespace; rewritten to assert the
  alignment contract (cell content survives + every rendered row
  shares pipe offsets).
- New ui-tui markdown.test.ts case (1): rendered column-2 start
  offset is identical for the header + every body row, including
  the CJK row that drifted before the fix.
- Live: hermes chat -q with the user-reported screenshot prompt now
  produces a perfectly aligned table on the wire (header, divider,
  4 body rows including '通义千问', all pipes at identical columns).
2026-05-11 11:13:06 -07:00
brooklyn!
a7e7921dbc
fix(tui): trim markdown wrap spaces (#22062)
* fix(tui): trim markdown wrap spaces

Use trim-aware wrapping for markdown prose so word-wrapped continuation lines do not keep boundary spaces.

* fix(tui): simplify markdown wrap nodes

Keep trim-aware wrapping on the rendered markdown text node while leaving nested inline segments as plain virtual text.

* fix(tui): trim definition row wrapping

Apply trim-aware wrapping to markdown definition rows so continuation lines match other prose rows.

* fix(tui): trim list and quote wrapping

Put trim-aware wrapping on the rendered list and quote rows that own markdown inline layout.

* fix(tui): preserve markdown nesting with trim wrap

Move list and quote indentation into layout padding so trim-aware wrapping does not erase nested markdown structure.

* fix(tui): trim only soft wrap spaces

Change trim-aware wrapping to remove whitespace only at soft-wrap boundaries so original leading inline spaces stay verbatim.

* fix(tui): preserve extra boundary whitespace

Trim only one soft-wrap boundary whitespace character so wrap-trim avoids leading continuations without collapsing intentional spacing.

* fix(tui): align styled wrap-trim mapping

Update styled text remapping to skip the single whitespace removed at soft-wrap boundaries without dropping preserved indentation.

* fix(tui): clean wrap trim test helpers

Clarify boundary-trim wording and strip OSC escapes from markdown render test output.

* fix(tui): strip osc before ansi in markdown tests

Remove OSC escapes from raw render output before SGR/CSI cleanup so markdown render assertions stay plain text.
2026-05-08 20:51:34 -07:00
Austin Pickett
e4120d1e6d Merge remote-tracking branch 'origin/main' into fix/markdown
Made-with: Cursor

# Conflicts:
#	ui-tui/src/components/markdown.tsx
2026-04-28 22:01:02 -04:00
Austin Pickett
cb039ac000 fix: account for latex 2026-04-28 21:20:43 -04:00
brooklyn!
6b09df39be
fix(tui): restore macOS copy behavior and theme polish (#17131)
This PR groups the TUI fixes that restore macOS Terminal usability and clean up the theme/composer regressions:

- copy transcript selections on macOS drag-release so Terminal.app users can copy while mouse tracking is enabled
- copy composer selections on macOS drag-release; composer selection is internal to TextInput and does not use the global Ink selection bus
- keep IDE Cmd+C forwarding setup macOS-only, and make keybinding conflict checks respect simple when-clause overlap/negation
- force truecolor before chalk initializes (unless NO_COLOR / FORCE_COLOR / HERMES_TUI_TRUECOLOR opt-outs apply) so the default banner keeps its gold/amber/bronze gradient in Terminal.app
- move TUI surfaces onto semantic theme tokens and preserve skin prompt symbols as bare tokens with renderer-owned spacing
- render focused placeholders as dim hint text in TTY mode instead of inverse/selected-looking synthetic cursor text
2026-04-28 18:47:14 -05:00
Austin Pickett
c3d39feb3a feat(latex): latex in tui 2026-04-28 19:08:11 -04:00
Brooklyn Nicholson
50edbe6f46 review(copilot): say solid rule, not dashed 2026-04-28 15:49:35 -05:00
Brooklyn Nicholson
4689ace7cb review(copilot): clarify table-rule rationale (UTF-16 code units, not graphemes) 2026-04-28 15:49:15 -05:00
Brooklyn Nicholson
9eabc24e24 fix(tui): visually distinguish markdown table rows from prose (#15534)
Tables rendered through `<Md>` had no separator and no header weight,
so they read as a paragraph with extra whitespace.  This adds two tiny,
border-free changes that survive Ink's grapheme-approximate column
widths better than a full outline:

* Bold the header row, keeping the existing amber colour.
* Insert a dim `─`-dashed rule between the header and body rows.

We deliberately stay away from a full outline — column widths are
measured via `stripInlineMarkup(...).length`, which is grapheme-aware
but still off by a cell on East Asian wide characters and emoji-mid-
cell strings.  A header rule plus the existing 2-space column gap
gives the visual hierarchy the issue asks for without amplifying that
inaccuracy into a misaligned border.

Validation: `npm run type-check` clean, `npm test --run` 389/389.
2026-04-28 15:49:15 -05:00
Brooklyn Nicholson
b1c49d5e73 chore(tui): /clean recent perf work — KISS/DRY pass
24 files, -319 LoC. Behaviour preserved, 369/369 tests green.

- hermes-ink caches: shared lruEvict helper for the four parallel LRU
  caches (stringWidth, wrapText, sliceAnsi, lineWidth); touch-on-read
  stays inlined per cache; tightened output.ts skip-slice fast path.
- wheelAccel: trimmed provenance header, collapsed env parsing, ternary
  dispatch in computeWheelStep.
- perfPane: folded ensureLogDir into once-flag, spread-with-overrides
  for fastPath/phases instead of full rebuilds.
- env: extracted truthy() (used 4×).
- virtualHeights: collapsed user/diff/slash height bumps; trail+todos
  estimate.
- useInputHandlers: scrollIdleTimer cleanup on unmount, ?? undefined
  shorthand.
- useMainApp: dropped dead liveTailVisible IIFE and liveProgress
  indirection.
- appLayout, markdown, messageLine, entry: vertical rhythm, dropped
  narration comments, inlined one-shot vars.
- fix: empty catch blocks → /* best-effort */ for no-empty lint.
2026-04-26 20:38:47 -05:00
Brooklyn Nicholson
c370e2e1e5 perf(tui): cache stringWidth/wrapText/sliceAnsi + skip-slice when line fits clip
CPU profile (Apr 2026, real-user scroll on 11k-line session) showed three
hot loops in the per-frame render path:

  Output.get() per-frame walk:                 24% total
  └─ sliceAnsi(line, from, to) per write:     18% total
  stringWidth(line) chain (cached + JS):      14% total

All three were re-doing identical work every frame: same string → same
clipped slice → same width.

Fixes:

1. Memoize stringWidth (8k-entry LRU) for non-ASCII strings; ASCII fast-path
   skips the cache (inline scan beats Map.get for short ASCII, the >90%
   case). String.charCodeAt scan up to 64 chars is cheaper than the regex
   fallback.

2. Memoize wrapText (4k-entry LRU keyed by maxWidth|wrapType|text) — wrapAnsi
   is pure and the same content reflows identically every frame.

3. Memoize sliceAnsi (4k-entry LRU keyed by start|end|str) for the
   end-defined hot path used by Output.get().

4. Skip the slice entirely in Output.get() when the line already fits the
   clip box (startsBefore=false && endsAfter=false). Most transcript lines
   never exceed their container width, and tokenizing them just to slice
   (line, 0, width) was pure overhead. This single fast-path drops
   sliceAnsi from 18% → ~0% in the profile.

Also tighten virtualization constants (MAX_MOUNTED 260→120, OVERSCAN 40→20,
SLIDE_STEP 25→12) and cap historical-message render at 800 chars / 16
lines via HISTORY_RENDER_MAX_*; messages inside the FULL_RENDER_TAIL_ITEMS
window still render in full so reading-zone behavior is unchanged.

Validation, real-user CPU profile, page-up scroll on 11k-line session:

  Output.get() self-time:     24%   →   0.3%
  sliceAnsi total:            18%   →   not in top 25
  stringWidth family:         14%   →   ~3%
  idle:                     60.7%   →  77.3%

Frame timings (synthetic page-up profile harness):
  dur p95:   ~10ms   →  4.87ms
  dur p99:   25ms+   → 12.80ms
  yoga p99:  ~20ms   →  1.87ms

The remaining CPU in the profile is Yoga layoutNode + React commit,
which is the irreducible work for this UI tree size.
2026-04-26 19:28:09 -05:00
Brooklyn Nicholson
b97b4c4981 refactor(tui): clean markdown.tsx per KISS/DRY
- Drop the outer no-op capture group from INLINE_RE and restructure the
  source as an ordered list of patterns-with-index-comments so each
  alternative is individually greppable. Shift group indices in MdInline
  down by one accordingly.
- Inline single-use helpers (parseFence, isFenceClose, isMarkdownFence,
  trimBareUrl) and intermediate variables (path, lang, raw, prefix, body,
  depth, task body, setext match, etc.).
- Hoist block-level regexes used inside MdImpl (FENCE_CLOSE_RE, SETEXT_RE,
  BULLET_RE, TASK_RE, NUMBERED_RE, QUOTE_RE) to top-level consts so
  they're compiled once instead of per-line.
- Collapse the duplicate compact-vs-normal blank-line branches into one
  if/!compact gap call.
- Move Fence and MdProps types to the bottom per house style.
- Shorten splitTableRow → splitRow and use optional chaining in a few
  match sites.

No behavior change; 162/162 tests pass. Net -22 LoC.
2026-04-21 18:11:12 -05:00
Brooklyn Nicholson
43eb1153e9 fix(tui): don't swallow Kimi/Qwen ~! ~? kaomoji as subscript spans
The inline markdown regex had `~([^~\s][^~]*?)~` for Pandoc-style subscript
(H~2~O, CO~2~). On models that decorate prose with kaomoji like `thing ~!`
and `cool ~?` — Kimi especially — the opener `~!` paired with the next
stray `~` on the line and dim-formatted everything between them with a
leading `_` character, mangling markdown output.

Tighten the pattern to short alphanumeric-only content (`~[A-Za-z0-9]{1,8}~`)
since real subscript never contains punctuation, spaces, or long runs.
Same tightening applied to stripInlineMarkup so width measurement stays
consistent. Classic CLI was unaffected because it renders these literally.
2026-04-21 17:34:48 -05:00
Brooklyn Nicholson
136519a2c9 fix(tui): inject VS16 so text-default emoji render as color glyphs
Models frequently emit bare codepoints like U+26A0 (⚠), U+2139 (ℹ),
U+2764 (❤), U+2714 (✔), U+2600 (☀), U+263A (☺) which, per Unicode, have
Emoji_Presentation=No and render as monochrome text-style glyphs in
terminals unless followed by VS16 (U+FE0F). Agent output leaked through
the TUI like `⚠ careful` instead of `⚠️ careful`.

Added `ensureEmojiPresentation` (lib/emoji.ts): scans for the curated
set of text-default codepoints and appends VS16 when the next char is
not already VS16, ZWJ, or a keycap-enclosing mark. Idempotent and
fast-pathed by a Unicode-range regex so ASCII-heavy text is untouched.

Applied once at the top of `Md`'s line parse. Hermes-ink's stringWidth
already accounts for VS16, so cursor/layout stays correct.
2026-04-21 15:52:39 -05:00
Brooklyn Nicholson
97c2da2112 fix(tui): render MEDIA: as a clickable file chip, drop audio directive
The agent emits `MEDIA:<path>` to signal file delivery to the gateway,
and `[[audio_as_voice]]` as a voice-delivery hint. The gateway strips
both before sending to Telegram/Discord/Slack, but the TUI was rendering
them raw through markdown — which is also how the intraword underscore
bug originally surfaced (`browser_screenshot_ecc…`).

At the `Md` layer, detect both sentinels on their own line:
- `MEDIA:<path>` → `▸ <path>` with the path rendered literal and wrapped
  in a `Link` for OSC 8 hyperlink support (absolute paths get a
  `file://` URL, so modern terminals make them click-to-open).
- `[[audio_as_voice]]` → dropped silently; it has no meaning in TUI.

Covers tests for quoted/backticked MEDIA variants, Windows drive paths,
whitespace, and the inline-in-prose case (left untouched — still
protected by the intraword-underscore guard).
2026-04-20 17:11:54 -05:00
Brooklyn Nicholson
b17eb94907 fix(tui): don't italicize intraword underscores in markdown
The inline markdown regex matched `_..._` / `__...__` anywhere, so file
paths like `browser_screenshot_ecc1c3feab.png` got mid-path italics.

Require non-word flanking (`(?<!\w)` / `(?!\w)`) on underscore emphasis
so snake_case identifiers and paths render literally, matching the
CommonMark intraword rule. `*` / `**` keep intraword semantics.
2026-04-20 17:04:09 -05:00
Brooklyn Nicholson
5c8b291607 fix(tui): wrap markdown links in Link so Ghostty/iTerm/kitty get real OSC 8 hyperlinks
renderLink was discarding the URL entirely — it rendered the label as amber
underlined text and dropped the href. Result: Cmd+Click / Ctrl+Click did
nothing in any terminal, including Ghostty.

Now both markdown links `[label](url)` and bare `https://…` URLs are wrapped
in @hermes/ink's Link component, which emits OSC 8 (\\x1b]8;;url\\x07label\\x1b]8;;\\x07)
when supportsHyperlinks() returns true. ADDITIONAL_HYPERLINK_TERMINALS already
includes ghostty, iTerm2, kitty, alacritty, Hyper.

Autolinks that look like bare emails (foo@bar.com) now prepend mailto: in the
href so they open the mail client correctly.

Also adds a typed declaration for Link in hermes-ink.d.ts.
2026-04-18 14:39:24 -05:00
Brooklyn Nicholson
f8becbfbea feat(tui): per-language syntax highlighting in markdown code fences
Adds a minimal hand-rolled highlighter for ts/js/jsx/tsx, py, sh/bash, go, rust,
json, yaml, sql. Recognizes whole-line comments, single/double/backtick strings,
numbers, and per-language keyword sets. Unknown langs fall through to the current
plain rendering; the existing diff-specific colorization is preserved.

Closes the §8 "Markdown syntax highlighting is missing (only diff gets colored)"
finding from the TUI v2 audit without pulling in a highlighter library.
2026-04-18 09:48:38 -05:00
Brooklyn Nicholson
4b4b4d47bc feat: just more cleaning 2026-04-15 14:14:01 -05:00
Brooklyn Nicholson
4a260b51fe fix: deep markdown parsing 2026-04-13 15:01:15 -05:00
Brooklyn Nicholson
e03bef684e chore: fmt 2026-04-12 16:33:25 -05:00
Brooklyn Nicholson
24a498eb90 feat: better markdown 2026-04-11 17:15:36 -05:00
Brooklyn Nicholson
8760faf991 feat: fork ink and make it work nicely 2026-04-11 11:29:08 -05:00
Brooklyn Nicholson
b7d4ea1550 feat: better hyperlink formatting 2026-04-09 15:13:43 -05:00
Brooklyn Nicholson
b50d81f212 fix: diff colours 2026-04-08 12:11:55 -05:00
Brooklyn Nicholson
9c2c9e3a3e chore: fmt 2026-04-07 20:30:22 -05:00
Brooklyn Nicholson
d9d0ac06b9 chore: readme update 2026-04-07 20:24:46 -05:00
Brooklyn Nicholson
29f2610e4b tui updates for rendering pipeline 2026-04-07 20:11:05 -05:00
Brooklyn Nicholson
2d349bbf7a chore: fmt 2026-04-06 18:43:00 -05:00
Brooklyn Nicholson
afd670a36f feat: small refactors 2026-04-06 18:38:13 -05:00
Brooklyn Nicholson
5a5d90c85a chore: formatting etc 2026-04-03 20:14:57 -05:00
Brooklyn Nicholson
56a69e519b chore: uptick 2026-04-03 19:55:15 -05:00
Brooklyn Nicholson
1218994992 chore: uptick 2026-04-03 14:44:50 -05:00
Brooklyn Nicholson
bbba9ed4f2 feat: split apart main.tsx 2026-04-02 20:39:52 -05:00