Speed is engineered.
I optimize LLM serving, GPU kernels, retrieval, and runtimes—turning bottlenecks into measured gains.
Don’t compute it twice.
Native-hot: 2.42 s, faster still. Paths are schematic, not live telemetry.
Persistent-cache speedup
NVIDIA A40 · 16K prefix · C8
Lower kernel latency
Best of five measured QMM configurations
Faster wavefunction grids
Versus the vectorized pyquante2 path
Feel the
difference.
One GPU. Three serving paths.
Explore what persistent prefix reuse changes—and where native-hot caching still wins.
Faster batch completion
with a persistent-cache hit.
18.26 seconds less batch time
Native-hot is faster. This result measures prefix reuse in a fresh process, not a universal serving speedup.
Mean batch wall time · Three repetitions per workload
Recorded values. The playback does not run a benchmark.
Numbers.
Not adjectives.
Nine results across four layers.
Open a study to see the change, its context, and the evidence behind it.
01Persistent KV reuse
4.38×Batch wall-time speedup
Persistent KV reuse
The engineering
Reuse a stored prefix in a fresh serving process instead of recomputing it.
A40 · Qwen2.5-7B · BF16 · 16K prefix · concurrency 8
Same-process native-hot cache was faster: 2.42 s. This is a persistent-cache result, not a blanket vLLM speedup. Three repetitions per cell; no statistical-significance claim.
Recorded comparison
02Quantized matrix multiplication
38.1%Lower kernel latency
Quantized matrix multiplication
The engineering
Change tile selection for large affine quantized matrix products.
M4 Max · 4-bit weights · BF16 activations · M=2048
Five tested configurations report 5.6–38.1% lower latency. Kernel-level measurements, not end-to-end model speedups. Submitted; not merged.
Explore five measured configurations
03Fuzzy search. 100K documents.
38.7×Warm-query speedup
Fuzzy search. 100K documents.
The engineering
Replace the JavaScript search hot path with compiled Mojo kernels.
M4 Max · 100K docs · 16-code-unit patterns · warm
Warm results range from 8.8× to 38.7× across nine cells. The accelerated configuration uses 16 threads; this is not a single-thread comparison.
Recorded comparison
04Faster lexical ranking
1.10–1.95×Against optimized bm25s
Faster lexical ranking
The engineering
Precompute sparse scores, reduce allocation overhead, and batch queries.
M4 Max · 1K–100K docs · 5–20 query terms
Scoring variants differ between libraries. Numerical parity is measured against rank_bm25, not bm25s. The smaller optimized-baseline range is the headline.
Supporting measurement
05Molecular wavefunction grids
72×Against vectorized baseline
Molecular wavefunction grids
The engineering
Evaluate a molecular-orbital grid with a compiled numerical kernel.
M4 Max · single thread · benzene / 6-31G* · 50³ grid
A 50³ single-orbital grid, not an entire chemistry simulation. The much larger legacy-PyQuante ratio is intentionally not the headline.
Recorded comparison
06Filtered vector retrieval
4,099QPSConcurrency 8 · recall 1.0000
Filtered vector retrieval
The engineering
Use physical partitioning to avoid scanning unrelated vectors.
Cohere-100K · 768 dimensions · cosine · 1% filter
Serial p99 is reported separately as 1.5 ms. It is not p99 at concurrency 8. Connector PR is open; this is not leaderboard certification.
Supporting measurement
0727B-model serving on one GPU
781.3tok/sAggregate output throughput
27B-model serving on one GPU
The engineering
Serve a 27B model at 50-way concurrency on one H100.
H100 80GB · Gemma 3 27B · FP16 · 550 in / 256 out
Taken from the supplied August 26, 2026 documentation snapshot. The live documentation could not be rechecked for this edition.
Supporting measurement
08Warm, deterministic execution
0.83–0.93msDocumented warm NPV call
Warm, deterministic execution
The engineering
Keep a signed compute runtime resident between typed function calls.
One representative operation · warm resident runtime
Excludes first-ever runtime provisioning, documented at about 110 s. Not end-to-end agent latency or a benchmark of every function.
Supporting measurement
09Memory-bounded phrase search
19.51MBPeak RSS · scored stress test
Memory-bounded phrase search
The engineering
Replace explosive position expansion with a linear-memory matching path.
Repeated-token phrase stress case · scored search
The baseline is a watchdog lower bound, not a measured exact peak. No memory-reduction percentage is inferred. Submitted; not merged.
Supporting measurement
Author-published, workload-specific results. Open PRs are proposed contributions, not upstream adoption. Recorded evidence is from the November 9, 2026 source review; benchmarks have not been independently rerun.
INDEPENDENT AI SYSTEMS & PERFORMANCE ENGINEER
Angel Marinov.
I build and optimize the parts of AI systems where execution cost accumulates: inference paths, GPU kernels, search, and runtime internals.
Selected work includes persistent vLLM KV-cache reuse, compiled Mojo retrieval and scientific kernels, and proposed upstream changes to quantized matrix multiplication and phrase-query memory.
Agentic iteration.
Human ownership.
Coding agents are part of my daily workflow. The architecture, engineering decisions, validation, and final review stay with me.
01Find the actual bottleneck
Start with the workload, hardware, and target metric. Measure the existing path before deciding what should change.
02Change the mechanism
Work where the cost is: data layout, allocation, dispatch, memory traffic, caching, or the algorithm itself. Use agents to explore; use evidence to choose.
03Keep correctness intact
Numerical agreement, semantic behavior, and lifecycle correctness belong beside the speed result. A faster wrong answer is not an improvement.
04Ship something reviewable
A focused change, a clear comparison, and explicit trade-offs. Work remains inside the client's approved environments and AI-tool policies.
The work behind the numbers
Bring a
bottleneck.
or a runtime constraint.Find me on GitHub