NoteWorthy
A multimodal notes-to-LaTeX pipeline that streams Gemini-generated structure into a styled template, compiles it through configured HTTP backends, and returns PDF plus source.
- Role
- Full-stack developer
- Stack
- Next.js / TypeScript / Gemini File API / Server-Sent Events / LaTeX / Prisma / PostgreSQL
- 7,237
- supplied TeX lines
- byte-stable source artifact
- 63
- TikZ scenes
- plus 32 PGFPlots axes
- 110
- supplied PDF pages
- US-letter metadata
Why the conversion path matters
Dense mathematical notes are more than text: hierarchy, notation, diagrams, and relationships must survive the move into an editable document. NoteWorthy turns image notes into a generated LaTeX body, composes that body into a styled source file, and returns both source and a compiled PDF.
The implementation keeps the user-facing loop visible through streamed status updates, while each server boundary applies a narrower set of request, generation, composition, or response guards.
Current execution path
- Client-side compression begins only above a 20 MiB batch, with a 2,000 px maximum dimension and JPEG quality 0.82.
- POST /api/latex/generate accepts no more than ten files or 25 MiB, checks browser-declared MIME values for an image/ prefix, and emits Server-Sent Events.
- GoogleAIFileManager uploads images in parallel before streamed Gemini generation accumulates one LaTeX body.
- Files written for accepted requests are removed after generation completes or raises an error; early validation exits happen before those file writes.
Generation contract
Summary, Base, and Expansion modes change the conversion instruction, while Regular is the default model and Fast/Pro are premium-gated selectable alternatives mapped to Gemini identifiers. The prompt requests a body compatible with the template’s definition, note, theorem, question, solution, example, and TikZ grammar.
The streamed result is trimmed, fenced output and a returned document prefix are removed, and the cleaned body is handed to composition. This normalization is deterministic cleanup, not a separately implemented OCR or compile-repair stage.
Proposed / conceptual ML-CV layer
A disconnected dashed overlay explores possible perspective and illumination normalization, a math-aware layout graph, region-level uncertainty visualization, and compile-repair plus render-regression gates.
These components are design hypotheses only. They are not present in the inspected runtime, carry no accuracy or latency claims, and never share the solid visual path used for source-verified behavior.
Compile and output boundary
- The composition route replaces <content> in a 1,065-line styled template.
- The PDF route tries configured external HTTP compiler targets with backend-specific payloads, a 30-second request timeout, and bounded retries.
- Returned bytes must begin with the PDF file signature before the browser creates a preview URL.
- The interface retains the generated body, composed source, and PDF blob for copy, preview, and export.
Optional and historical rails
Authenticated users can optionally save PDF bytes and metadata through Prisma and combine selected saved documents. Neither operation is required for conversion.
The repository also retains a Socket.IO server and a Dockerized latexmk service. They document earlier or alternate approaches; the current client uses fetch plus SSE, and the current PDF route defaults to configured HTTP compiler backends.
Artifact evidence
The supplied main.tex contains 7,237 lines across three chapters and 62 numbered sections, including 63 TikZ scenes and 32 PGFPlots axes. It imports preamble, macros, and letterfonts and references two external images, so it is offered as source-as-is rather than a standalone compile bundle.
The supplied generated PDF reports 110 US-letter pages. Five full-page previews are shown below without asserting that the file is a complete or partial corpus, and without turning structural counts into model-quality claims.
Interactive system 01 / current runtime
Follow one conversion across every boundary.
The retained hook name says “WebSocket,” but the executable client now uses a streamed HTTP request. Isolate each layer to inspect the real request path, its limits, and the optional persistence rail.
The current path carries image notes through client preparation, guarded upload, streamed Gemini generation, template composition, external HTTP compilation, and browser output.
BoundarySolid paths are current defaults. Dashed nodes are selectable or authenticated options; the subdued lower rail is retained historical code.
Interactive system 02 / artifact evidence
Source, structure, and render stay connected.
The supplied TeX and PDF are treated as evidence, not a quality score. Counts come from the files; the selected views expose the grammar, plotting structures, and final page forms behind them.
Artifact 01 / source
A large TeX corpus with an explicit dependency boundary.
- 7,237 source lines
- 3 chapters
- 62 numbered sections
- SHA-256 6ff659b0…e6cbab
The supplied main.tex is offered as-is. The three imported files and two referenced images are not included, so this is not described as a standalone compile bundle.
\documentclass{report}
\input{preamble}
\input{macros}
\input{letterfonts}
\chapter{Vector Fields $f:\mathbb{R}^n \rightarrow \mathbb{R}^n$}
\section{Vector Fields}Selected pages / supplied PDF
Full pages, at their intrinsic portrait ratio.
Five pages span boxed theorem grammar, equations, 3D surfaces, polar construction, and vector fields. Nothing auto-advances, and every preview is contained without a landscape crop.
Selected supplied PDF page 15 of 110. A 3D parametric helix above theorem and question structures generated from the source grammar.
Evidence bundle / direct artifacts
Inspect the supplied files directly.
The PDF is the exact file behind the previews. The TeX source is available unchanged and keeps its external dependency boundary visible.
PDF / 110 pages
Supplied generated PDF
The exact US-letter artifact used for every page preview on this case study.TeX / 7,237 lines
Supplied main.tex
The source artifact as provided, including its external import and image references.Current
SSE is the live transport.
The browser fetches the generation route with text/event-stream, then composes and compiles after completion.Optional
Library features require identity.
SavedPdf persistence and document combination sit outside the default conversion path.Historical / alternate