An ESP-DL model listing may say "ESP32-P4 supported," but that does not answer the question a camera product has to answer: will a result arrive while it is still useful?
Chip support, successful loading, sufficient memory, fast individual inference, and acceptable response time with the camera, display, and network all running are separate checks. Passing the first one does not certify the others.
Start with a specific job, such as detecting a person near an entrance or reading a fixed instrument display. Get a minimal model path working, then measure the complete path on the intended board, sensor, resolution, and concurrent workload. This article uses Espressif's public documentation to explain that process. It does not claim measured FPS, power consumption, temperature, recognition accuracy, or production reliability for any untested device.
Choose the job before choosing a model
The current ESP-DL Model Zoo lists object detection, face and pedestrian detection, face recognition, classification, pose estimation, OCR, and other tasks with chip support markers. Those entries do not have interchangeable inputs or outputs. A detector returns candidate locations; a classifier returns categories; an OCR pipeline has to turn image regions into usable text. Asking only how many models the chip "can run" skips the interface your application actually needs.
Consider an entrance device. Face detection answers whether a face appears in a frame. Identity recognition adds feature extraction and matching, plus enrollment, thresholds, and a response to false matches. If the product only needs to wake a display when someone approaches, adding identity recognition creates privacy and error-handling work without improving that task. Define the action to be taken on a result before selecting a model with more features than the application requires.
OCR has a similar split. Reading digits at a known location on a panel is different from reading arbitrary labels at varying angles and under glare. The first task can narrow the crop and expected character set. The second needs to deal with localization, perspective, blur, and potentially multiple languages. An OCR entry marked for P4 in the official list is a credible integration starting point. It is not evidence that a particular camera, lens, and mounting angle will read your labels accurately. That conclusion needs actual images and a record of omissions, substitutions, and rejected reads.
The list also includes non-vision tasks. Their presence does not provide performance evidence for a vision pipeline. For a camera product, lock down the image source, the desired output event, and the maximum acceptable delay first. Only then does a model shortlist become meaningful.
What "supported" can — and cannot — mean
The first level is repository-level chip support: a model or example is marked for ESP32-P4. Record the model version, dependency versions, target chip, and file source. This prevents a later library update from silently changing what was actually assessed. It does not certify an arbitrary board configuration or an arbitrary input stream.
The next level is runtime compatibility. ESP-DL's .espdl model and TensorFlow Lite Micro's .tflite model are different artifacts. The ESP-VISION P4 guide describes separate execution paths and task-specific APIs. Copying a model that worked elsewhere to an SD card does not make it loadable by ESP-DL. Check the conversion toolchain, target chip, operator support, tensor dimensions, types, and input/output conventions rather than relying on the filename extension alone.
The ESP-DL dl::Model::test(), profile_memory(), and profile_module() functions are C++ model APIs. ESP-VISION's espdl and tflite examples show a separate Python application path. Both address related validation questions, but their calls cannot simply be mixed into one program. Choose the runtime that the product firmware actually uses, and instrument that runtime's own interfaces.
The third level is application fitness. A successful test() compares a model's result against test values exported with the model. It does not include exposure changes, color conversion mistakes, motion blur, business thresholds, or display lag from a real camera. Calling that result "validated entrance recognition" would claim much more than the test covers.
Decide where the model lives before finalizing updates
ESP-DL documents three broad loading routes: embed an .espdl file in application rodata, store it in a dedicated Flash partition, or load it from an SD card. Embedding is straightforward but ties the model to an application reflash and can enlarge the application partition. A dedicated partition allows a different model-update flow but requires partition sizing and compatibility planning. SD storage makes file replacement convenient, while adding mounting, file integrity, and read-speed conditions. These are release and maintenance trade-offs, not an inherent ranking of accuracy or speed.
| Route | Convenient when | What it costs you |
|---|---|---|
| Embedded in app rodata | Model never changes independently of firmware | Any model change means reflashing the app; app partition grows |
| Dedicated Flash partition | Model updated separately from firmware | Partition sizing, compatibility checks, version negotiation |
| SD card | Frequent model swaps, large artifacts | Mount, integrity verification, read-speed and card-quality worst cases |
If a deployed device must update its model without updating the main application, design that boundary before the firmware image and partition map are frozen. Decide how the device rejects a corrupted model, how it returns to the previous compatible version, and how the firmware checks model/API compatibility. The public API establishes that several loading options exist. It does not establish that any of them has passed your product's power-loss or update testing.
Keeping parameters in Flash or copying them into faster memory is another explicit trade-off. The ESP-DL documentation describes param_copy=false as a way to reduce RAM copies, with a possible inference-speed penalty. A model that starts only after disabling the copy has not automatically met a response-time requirement. Compare both configurations with the same input, clock, and concurrent tasks before accepting the memory saving.
File size is not the application's memory requirement
A common sizing mistake is to subtract a model file's size from total PSRAM and call the remainder "free for the camera." The file does not account for intermediate tensors, parameter copies, input and output buffers, captured frames, transforms, display buffers, or other tasks. Some allocations may also need a particular memory type or contiguous region. Plenty of total PSRAM does not guarantee that the next allocation a driver needs will succeed. If the product also runs a GUI, framebuffer allocations compete directly with the model workspace.
ESP-DL's profile_memory() reports model structures, copied parameters, variables, and other overhead across internal RAM, PSRAM, and Flash. Use it as a model-level measurement, not as the entire application's peak. Log free and peak memory before and after camera initialization, model loading, display initialization, and network activity. A failure observed after display startup is much easier to explain when the memory change at each boundary is known. The combined profile() call can also show where model-layer time is spent.
A simple lower bound helps expose hidden buffers. A 640×480 RGB565 frame with no extra stride contains 614,400 pixel bytes; two complete buffers contain 1,228,800 bytes, about 1.17 MiB. That calculation excludes alignment, driver-private allocations, a resized model input, and display-side copies. It says nothing about which memory region the buffers occupy. Its purpose is to flag that frames can consume memory on the same scale as a compact model's workspace — not to predict your board's measured peak.
The model input also deserves an explicit assumption. A tightly packed 224×224×3 tensor at one byte per element needs at least 150,528 bytes. A different camera format may require a transform buffer before inference. Layout, alignment, preprocessing, and runtime allocation policy can all raise the observed number. Write the assumptions beside the estimate; "AI needs roughly 2 MB" is not a reproducible memory plan.
An inference benchmark is not a response-time benchmark
A camera result travels through capture, format conversion, resize or crop, inference, post-processing, business rules, and finally display or event delivery. profile_module() reports latency inside the model's layers. It does not include that entire path. Reporting only its total cannot explain why a bounding box still trails a moving part on a screen.
The input may need a color conversion and resize before reaching the model. A detection result may then need thresholding, box decoding, and non-maximum suppression. Camera and display ownership can add copies or queueing. None of these stages is automatically dominant, but none is safely assumed to be free. Put timestamps around each stage in the same clock domain and record end-to-end percentiles, not only model time.
"Real time" has to come from the job. A meter reader that samples once per second cares about correct readings, worst-case response, and retry behavior. A device that controls access may care far more about tail latency and false activation. A live video overlay also needs input rate, processed rate, queued frames, and age of the output image. Mean inference time alone can look healthy while the frame queue grows and the displayed decision becomes stale.
Separate model bring-up from a real camera workload
First, use a fixed input to check that the model loads, tensors match, and the model produces the expected result. ESP-DL's test() works when test inputs and outputs were included during model export. A shipping model may omit those values to save space, so the absence of built-in test values is not itself evidence of a broken model. This stage isolates conversion and runtime errors.
Second, connect the intended camera. Record sensor, lens, resolution, pixel format, exposure range, crop, and channel ordering. Preserve failed frames for analysis. If the model behaves on public example images but misses objects under actual glare, motion, or viewing angle, shaving a few milliseconds from inference will not solve the product problem. Accuracy and timing must be accepted independently.
Third, enable the display, network, logs, and continuous capture. On the same model and board settings, compare isolated and concurrent operation. Record memory peaks, end-to-end latency percentiles, dropped frames, temperature, and recovery from faults. The duration of a meaningful sustained test follows the product's duty cycle. A short demonstration is not a reliability test, and public documentation cannot supply your device's pass criterion.
Quantization changes both fit and failure modes
The ESP-DL quantization guide describes .espdl as a quantized deployment format with 8-bit, 16-bit, and mixed-precision options exported through ESP-PPQ. A smaller quantized artifact does not by itself prove unchanged application accuracy. Small objects, low-contrast digits, and motion-blurred frames may be exactly the inputs that reveal a changed error distribution.
Hold the task, preprocessing, and evaluation set constant while comparing quantization choices. Check that each model exports and loads, that board-level correctness checks pass where test data is available, and that the relevant categories or characters still meet the application error budget. Then compare memory and timing.
If a smaller variant loses recall for a safety-relevant target, an attractive average accuracy figure will not rescue the decision. If a higher-precision variant exhausts working memory, changing a recognition threshold is not a memory fix.
There may also be a .tflite route for the task. ESP-VISION treats TensorFlow Lite Micro and ESP-DL as distinct runtimes. The TFLite model needs the enabled operators and correct quantized input/output handling. Verify equivalent input and output semantics before comparing their file sizes, working memory, and elapsed times. Otherwise the experiment compares two different models or preprocessing paths rather than two runtime options.
Carry the sample into product firmware one boundary at a time
An example often places capture, model invocation, and drawing in one demonstration loop. Product firmware may also own touch input, communications, logging, updates, and power management. Before copying the loop, decide which task owns each camera buffer, where a frame becomes a model tensor, and which component consumes the result. Unclear ownership can appear to work on the first frame and then fail when a reused buffer is overwritten or an allocation remains live.
Convert the camera image according to the model's actual tensor contract. Check width, height, channel order, data type, normalization, crop policy, and the coordinate mapping back to the original image. ESP-VISION exposes preprocessing parameters and tensor metadata queries that can help with this check. If boxes are consistently shifted although an object is found, inspect crop and resize coordinates before deciding that the model itself is inaccurate.
Keep the result interface separate from the business event interface. A detection box is transient frame state; an alarm may travel over a network and be persisted. Sending an event for every frame containing the same object binds network traffic and user notifications to inference frequency. A sensible design is to pass a timestamped candidate with confidence and source-frame identity to business logic, then apply deduplication and retry policy there.
Preserve checkpoints as integration becomes more complex: readable file, model load, fixed-input output, real-camera input, display overlay, business event, network concurrency, and sustained operation. Add one variable at a time. When the next checkpoint fails, the previous one remains a useful baseline. Enabling a new camera, model, display, and wireless stack all at once may feel faster but makes a memory fault difficult to attribute.
Write an acceptance record that can explain a failure
The useful deliverable is not a sentence saying "P4 supports YOLO." It is a record tied to board and PSRAM/Flash configuration, firmware and model versions, storage location, camera conditions, pixel transformations, threshold settings, concurrent tasks, clock rate, and failed samples. Keep model-only latency separate from application response time; keep mean values separate from high-percentile delays. This makes a later regression attributable rather than anecdotal.
The gates below add one new class of uncertainty at a time. They are a validation sequence, not results of testing performed for this article.
If the model will not load, verify conversion target, file version, storage path, partition, and filesystem before replacing the board. If it fails only after extended operation, inspect memory peaks, fragmentation, ownership, and restart behavior. If boxes lag behind the scene, time capture, preprocessing, inference, post-processing, and display separately, and check whether frames queue up. Treating all three symptoms as "P4 is too slow" wastes debugging time because they point to different parts of the system.
An approval decision can therefore be conditional:
- The model has a compatible conversion or official component route.
- The target board retains enough usable memory with all required functions active.
- Real inputs meet the error budget.
- Tail latency and sustained operation satisfy the application's deadline.
Passing an earlier condition cannot stand in for a later one. Mark an unmeasured condition "pending," not with an invented FPS estimate.
If the requirement is an open-ended large language model, long context, several high-resolution video streams, or a fixed frame-rate promise before any target-board testing, the Model Zoo's P4 support marker is not a sufficient basis for a product commitment. Narrow the task, consider a different compute platform, or budget a board-level benchmark first. Compact and well-defined vision tasks are a reasonable area to investigate on P4. Suitability for a particular product still rests on the actual model, memory map, camera path, and measured results.
Sources
- Espressif ESP-DL Model Zoo
- ESP-DL model loading, correctness testing, and profiling
- ESP-DL quantization and export
- ESP-VISION: introducing a model on ESP32-P4
Top comments (9)
The split between "supported", "loads", "fast once" and "fast enough with the camera, display and network running" is the right one, and it's rare to see a post say outright that it isn't claiming FPS for devices it hasn't tested.
I'd add two checks to the measured path. They're about method, not about the P4, which I haven't measured. First, report the distribution, not the mean. On a Raspberry Pi 5, a much bigger device, so only the method carries over, YOLO26n with half an hour on one process plus half an hour on four gave an average of 347 ms, and not one inference landed between 268 and 397 ms. The mean described a frame that never happened. Second, run it long enough to get hot: a short test can pass on a board that only starts slowing down after ten minutes.
For the entrance-device case, which number would you use to decide "the result is still useful": the p95 of the full path, or the time from someone appearing to the display waking up?
The 347 ms mean with nothing between 268 and 397 ms is the most useful number in this thread, and it makes the point better than any argument: you were reporting the latency of a frame that never occurred. That is the same failure mode as quoting a single peak RSS, just in the time domain.
On the method additions, I'd take both and add a third. Distributions first, absolutely, but report them per stage rather than only end-to-end, because the camera-to-inference slice and the inference-to-display slice have different tail shapes. A p95 that lives in capture tells you to fix exposure or the queue, while one that lives in inference tells you to shrink the model. Reporting only the full-path p95 hides which of the two you bought. Second, running long enough to get hot is the check most people skip, and it's the one that separates a number from a spec. I'd go further and say pick the duration from the deployment, not from the test: a device that runs eight hours a day needs an eight-hour soak at the target ambient temperature, and the interesting output is the drift between the first ten minutes and the last ten.
Third, and this is specific to the P4 class of parts: record whether the board was doing anything else. With a single camera and no display, the CPU has slack; add a display refresh and a network upload and the same model gets a different p95 without a single line of model code changing.
On your question, I'd use neither alone, because they answer different questions and the product fails on the second one. The p95 of the full path is the number for capacity planning and for regression testing across firmware revisions. The time from a person appearing to the display waking up is the number that decides whether the result is still useful, and it will always be worse than the p95 because it includes the detector's own confirmation and the display's refresh interval. My rule is to set the acceptance threshold on the wake-to-visible number and then use the p95 as the instrument I track to see if I am drifting toward it. If the wake-to-visible time is the one you gate on, you can't accidentally ship a build that passes tests while feeling slow to the person standing at the door.
One caveat worth stating, since you flagged that no inference landed in that 268-397 ms gap: a 60-point sample is enough to show the mean is a poor summary, but it isn't enough to trust a p95. If you want a tail number, sample at least a few hundred runs, or say "the distribution was bimodal" and leave the tail claim out.
Per-stage distributions are the better version of my first check: an end-to-end p95 can't tell you whether to fix the queue or shrink the model. Gating on wake-to-visible and tracking the p95 as the instrument is a rule I'll borrow.
One correction on the sample, since your caveat depends on it: it wasn't 60 points. The Pi run had 25,898 inferences, 13,813 with one process and 12,085 with four, and every one was kept. The empty gap between 268 and 397 ms is in that full set. Your caveat still holds at the extremes, though: with one process the p99 was 137.7 ms and the maximum 267.9, and only 51 inferences went past 150 ms, so the far tail rests on few points.
Fair correction, and the 25,898 inferences matter more than the number I was arguing about. My "60 points" was inferred from the gap being described as empty, not measured by me, so I'll take your figure.
What changes with your numbers is the shape of the caveat, not its existence. With 12,085 inferences at one process and a p99 of 137.7 ms against a max of 267.9, the body of the distribution is well established and the far tail is not: 51 samples past 150 ms in 12,085 is 0.4%, which means the p99.9 lives on roughly 12 points. So the honest claim is that the median and p95 are solid, the p99 is indicative, and anything past the max you measured is unmeasured rather than impossible. That is a narrower claim than "the tail rests on few points", and it is also more useful, because it tells a reader exactly which percentile they can plan against.
The four-process run is the more interesting one. 13,813 versus 12,085 is only a 14% throughput gain for 4x the processes, and if the per-inference latency still spread the same way, the contention cost is being paid in the tail rather than the mean. That is the same conclusion this article reaches about the P4 camera pipeline: adding parallel work does not shrink the queue, it relocates it. If you still have the raw run, the p95-to-max ratio for one process versus four would settle it in a single number.
On per-stage reporting, one practical note from our side. On the P4 we could only get clean per-stage numbers by timestamping at three points (CSI frame-ready, inference done, DSI vsync) and accepting that the camera timestamp has its own jitter. If your Pi harness timestamps only at the API boundary, the stage split will look tighter than the hardware actually is.
Small fix first: the counts are the other way around. 13,813 is one process and 12,085 is four, and both phases ran 30 minutes, so four processes didn't gain throughput, they lost it: 7.68 inferences per second with one, 6.72 with four.
The ratio you asked for, from the raw run: max over p95 is 1.98 with one process (135.2 to 267.9 ms) and 1.15 with four (608.3 to 702.3 ms). The p99.9 is 167.7 and 662.1. So on this Pi the contention was paid in the body, not the tail: every inference took about 4.6 times longer, and the distribution got relatively tighter. The one-process run is the one with the loose tail.
On timestamps, your warning applies in full: these numbers are inference only, with random input. No camera and no display, so there is no stage split to look tighter; there is only one stage. Your three points (frame-ready, inference done, vsync) are what it would take to say anything about the whole system.
Thanks for the correction, and the direction matters: I had read 13,813 as the four-process run, which made 4x processes look like a 14% win. With one process at 7.68 inf/s and four at 6.72 inf/s, four processes actually lose 12.5% of throughput while every inference gets ~4.6x slower. That is a much stronger result than the one I was building on, and it changes what the Pi is telling you: this is not a tail-latency problem, it is a serialization problem. Four processes on a Pi sharing one memory bus and one set of cores are not adding capacity, they are adding context switches and cache pressure to the same budget.
The max-over-p95 ratio is the cleanest number in the thread. 1.98 for one process versus 1.15 for four says the one-process run has the loose tail and the four-process run is comparatively tight around a much worse center. For planning that is the opposite of what people usually assume: concurrency bought predictability at the cost of the body. If you are sizing a system, the one-process configuration is the one where a p95 budget can still be blown by a single bad inference, and the four-process configuration is the one where you already blew it on every inference. Neither is a tail story.
What I would want next, if the harness is already there: the p50 and p95 for each configuration, not just max and p99.9. With p95 of 135.2 ms and max 267.9 ms on one process, the ratio is 1.98, but p99.9 at 167.7 ms sits close to p95, which suggests the tail is thin and a handful of outliers do all the work. If p50 is around 120 ms, then one process has a ~2x spread between typical and worst, and that is a configuration you can still defend. Four processes at p95 608.3 ms and p99.9 662.1 ms has almost no spread at all, which usually means a hard constraint, not contention noise: something is scheduling at a fixed interval and everything is queuing behind it.
On timestamps, we agree completely, and your framing is the right one. Inference-only with random input is one stage, and a one-stage number cannot say anything about a camera-to-display budget. The reason we bother with the three-point split on the P4 is not precision for its own sake: it is that the camera timestamp and the vsync timestamp fail in different ways (dropped frames versus missed deadlines), and an end-to-end number hides which one you are paying for. On a Pi with no camera and no display, there is nothing to hide, so the single-stage number is honest as long as it is labeled that way, which yours is.
If you do run the camera path later, the thing worth measuring is not the mean but the gap between inference-done and vsync on each frame. That gap is where the pipeline either has headroom or does not, and it is the number that decides whether a second model fits.
Your p50 guess was close: 129.7 ms with one process and 595.4 with four. So one process spreads about 2.1x from typical to worst, and four only about 1.2x.
But before reading a scheduler into the four-process run, I have to correct my own framing, because I went back to the harness: the two phases didn't run the same configuration. The one-process phase used ONNX Runtime's default intra-op threads, so a single inference was spread across all four cores. The four-process phase ran one thread per process, one process per core. So 595 ms is mostly the single-core latency of YOLO26n on this Pi, not four inferences queuing for the same cores, and "the contention was paid in the body" was the wrong way for me to put it. Each process had a core to itself, which is also why that distribution is so tight: nothing was waiting its turn.
Where contention does show is in the gap. If a core were simply a quarter of the machine, four single-thread runs would land near 4 x 130 = 521 ms; they landed at 596, about 14% worse, the same loss as the throughput (7.68 vs 6.72 per second). My guess is memory bandwidth and the shared L3, with four copies of the weights instead of one, but I didn't measure that, so it stays a guess. The heat is consistent with it: the four-process phase peaked at 80.7 °C with the clock holding 2,400 MHz, while the one-process phase reached 85.1 °C and throttled.
Agreed on the inference-done-to-vsync gap. That's the number I'd want before claiming a second model fits.
Updating the harness and saying so is the part most threads never get to, so take the correction as given: if the one-process phase was intra-op parallel across four cores and the four-process phase pinned one thread per core, then 595 ms is single-core YOLO26n latency and my serialization reading was wrong. Withdraw it. The tight distribution on four processes now makes sense without a scheduler story: each process owned a core and had nothing to wait for, while the one-process run had four threads contending for shared cache and a single clock budget, which is exactly the shape that produces a 2.1x body spread and a throttled 85.1 °C peak.
What is left standing is the gap, and it is a better result than the one either of us started with. Four single-thread processes landing at 596 ms against a 4 x 129.7 = 519 ms "quarter of a machine" expectation is 15% over, and it lines up with the 12.5% throughput loss, which means the two numbers are the same measurement seen from both ends rather than two independent confirmations. That is the number I would put in a post: on this class of device, per-core scaling is not free and the tax shows up as a double-digit percentage even when nothing is queuing.
Memory bandwidth and shared L3 with four copies of the weights is the right first hypothesis, and you can test it without new hardware. Run two processes instead of four and look at where the latency lands: if it is near 2 x 130 = 260 ms plus a similar percentage rather than half of 596, the shared-resource story holds and the tax is roughly proportional to process count. If two processes land much closer to 260 ms flat, then four was a threshold event, which would point at L3 capacity rather than bandwidth, because four weight copies is where the working set stops fitting. Either answer is worth having, and the two-process run is 30 minutes of the harness you already have.
One caveat on the thermal comparison, since it cuts against the conclusion: 85.1 °C with throttling on one process versus 80.7 °C at a held 2,400 MHz on four means the single-process run was the one thermally limited. So part of its 129.7 ms p50 may be throttle, not just intra-op contention, and the honest comparison needs the one-process run with threads pinned to a single core too, warmed to the same steady state. That would give four numbers instead of two: single-core single-process, single-core four-process, all-core single-process, all-core four-process. The 2x2 is what separates "concurrency tax" from "thermal headroom", and right now the thread is attributing both to memory.
The inference-done-to-vsync gap stays the open question for the P4 work, and your Pi result actually sharpens what to look for: if per-core scaling costs 15%, the interesting quantity is not the mean gap but how much of it is left when the pipeline stalls. A gap that is comfortable on average and near zero at p99 means there is no room for a second model regardless of the average, which is the trap the single-stage numbers hide.
One correction to the two-process prediction, and one number on the thermal caveat.
With one thread per process, two processes shouldn't land near 2 x 130. Each inference still runs on a single core, so both should sit near single-core latency, around 520 ms, plus whatever the shared resources cost with two copies of the weights instead of four. That still makes two processes a good test of bandwidth versus L3, just with 520 as the baseline instead of 260.
Which points at the cell we're actually missing: one process, one thread. Right now 521 is 4 x 130, and that assumes the four-thread run scaled perfectly. If intra-op parallelism lost anything, the true single-core time is below 521, and the four-process tax is bigger than 14%, not smaller. So 14% is a floor.
On the throttle: the harness logs in five-minute windows. The first window of the one-process phase ran at a mean real clock of 2,397 MHz with no throttle flags, p50 128.3 ms. The last one, throttled, averaged 2,283 MHz, p50 131.2 ms. So throttling moved the one-process p50 by about 2%, which doesn't change the gap.