app+docs: 第三十八轮——raw 录制调试开关+逐级重放定案(参考减除消 98.4%,官方 13 函数二次核对无差异,NUC 域残差差 1.6×);显示链对齐官方两级(nativeOut 去 2× 内插 + 640×480 中间级低通),123 单测全绿

This commit is contained in:
ZXCLI
2026-10-07 00:29:55 +08:00
parent f38b246b39
commit 17c30ffd13
11 changed files with 683 additions and 34 deletions
@@ -93,6 +93,24 @@ class RenderPipeline(
* chain keeps the raw tail value.
*/
var fpaTempFix: Int = 0,
/**
* Emit the native 160x120 palette output instead of the PC-SDK's 2x
* upscaled 320x240 buffer.
*
* The official ANDROID app has no intermediate scaling stage at all: the SDK
* hands `ImageViewer` a `Bitmap.createBitmap(fpaWidth, fpaHeight)` and the app
* draws it once with a filtering paint (`PaintFlagsDrawFilter` +
* FILTER_BITMAP, jadx) — a single filtered scale from the sensor grid to the
* screen. Our port kept the PC-SDK's `upscale2x` (bilinear, even rows keep
* the sample, odd rows get the neighbour average), which fabricates an
* interleaved full/half-amplitude pattern the official pipeline does not
* have — measured as the row-direction weave in round 38
* (docs/android_app/rendering_diff_phone_20261006.md §17). true skips that
* stage: the consumer (LiveRenderer) does ONE filtered draw to the screen,
* exactly like the official viewer.
*/
var nativeOut: Boolean = false,
) {
private val npix = w * h
@@ -189,8 +207,8 @@ class RenderPipeline(
private var meanPrev = 0
// ---- output ----
val outWidth: Int = w * 2
val outHeight: Int = h * 2
val outWidth: Int get() = if (nativeOut) w else w * 2
val outHeight: Int get() = if (nativeOut) h else h * 2
private val lock = Any()
@@ -831,6 +849,16 @@ class RenderPipeline(
if (enhanceStrength > 0) {
detailEnhance.enhance(nuc, gray160, enhanceStrength, detailGain())
}
if (nativeOut) {
// official android path: no intermediate scaling stage — the SDK's
// output IS the sensor-grid image; the consumer scales it once
val pal = this.pal
for (i in 0 until npix) {
outArgb[i] = pal[gray160[i].toInt() and 0xFF]
}
renderedOnce = true
return true
}
upscale2x()
// palette colorize 320x240 -> ARGB
val pal = this.pal
@@ -56,8 +56,12 @@ class LiveRenderer(
}
private val density = surfaceView.resources.displayMetrics.density
private val bitmap = Bitmap.createBitmap(320, 240, Bitmap.Config.ARGB_8888)
private val paint = Paint(Paint.FILTER_BITMAP_FLAG)
// Buffer size follows the pipeline output (160x120 native since round 38);
// rebuilt lazily because the session may connect after this renderer.
private var bitmap = Bitmap.createBitmap(320, 240, Bitmap.Config.ARGB_8888)
private var bufW = 320
private var bufH = 240
private val textPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
color = Color.WHITE
typeface = Typeface.SANS_SERIF
@@ -78,7 +82,7 @@ class LiveRenderer(
* exactly on the frames the user is watching. The flipped frame, the marker
* list and the colour-bar segments are all recycled instead.
*/
private val flipped = IntArray(320 * 240)
private var flipped = IntArray(320 * 240)
private val marks = ArrayList<com.mag160c.thermal.media.MarkerPainter.Mark>(8)
private val barPaint = Paint()
private val srcRect = android.graphics.Rect()
@@ -86,6 +90,35 @@ class LiveRenderer(
private var barBitmap: Bitmap? = null
private var barPaletteIdx = -1
/** Match the draw bitmap to the pipeline's output size (once per change). */
private fun ensureBuffer(w: Int, h: Int): Boolean {
if (w == bufW && h == bufH) return true
if (w <= 0 || h <= 0) return false
bufW = w; bufH = h
bitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
flipped = IntArray(w * h)
return true
}
/**
* 640x480 offscreen stage, mirroring the official viewer's display chain:
* jadx `ImageViewer` draws the 160x120 SDK bitmap into a 640x480 offscreen
* bitmap with the filtering paint (PaintFlagsDrawFilter + FILTER_BITMAP) and
* only the SCREEN scales that further. The intermediate 4x bilinear is what
* attenuates the per-sensor-row FPN pattern on the screen (measured, §17:
* with it the row pattern lands on the official's 0.12e-2 arc; without, our
* single-scale render shows it at full amplitude). The stage costs one GPU
* blit per frame.
*/
private var mid: Bitmap? = null
private val midRect = android.graphics.RectF(0f, 0f, 640f, 480f)
private fun ensureMid(): Bitmap {
mid?.let { return it }
return Bitmap.createBitmap(640, 480, Bitmap.Config.ARGB_8888).also { mid = it }
}
private companion object {
const val BAR_SEGMENTS = 96
}
@@ -180,6 +213,8 @@ class LiveRenderer(
val h = canvas.height.toFloat()
canvas.drawColor(Color.BLACK)
val frame = vm.latestFrame ?: return
if (!ensureBuffer(vm.bufferW(), vm.bufferH())) return
if (frame.size < bufW * bufH) return
setFramePixels(frame)
// available area (minus UI overlays)
@@ -199,10 +234,12 @@ class LiveRenderer(
val zoom = vm.state.value.zoom
val crop = ImageTransform.cropForZoom(zoom)
// The zoom source rect now addresses the 640x480 middle stage (4x the
// buffer coordinates), exactly like the official viewer's offscreen.
val src: android.graphics.Rect? = if (zoom > 1) {
val cw = (320 * (crop[2] - crop[0])).toInt()
val ch = (240 * (crop[3] - crop[1])).toInt()
srcRect.set(160 - cw / 2, 120 - ch / 2, 160 + cw / 2, 120 + ch / 2)
val cw = (640 * (crop[2] - crop[0])).toInt()
val ch = (480 * (crop[3] - crop[1])).toInt()
srcRect.set(320 - cw / 2, 240 - ch / 2, 320 + cw / 2, 240 + ch / 2)
srcRect
} else null
@@ -217,8 +254,14 @@ class LiveRenderer(
fit.cx - size[0] / 2f, fit.cy - size[1] / 2f,
fit.cx + size[0] / 2f, fit.cy + size[1] / 2f,
)
if (src != null) canvas.drawBitmap(bitmap, src, dstRect, paint)
else canvas.drawBitmap(bitmap, null, dstRect, paint)
// two-step display like the official viewer: buffer -> 640x480 (filtered)
// -> screen. The GPU blits are cheap; the attenuated row pattern is the point.
val m = ensureMid()
val mc = Canvas(m)
mc.drawColor(Color.BLACK)
mc.drawBitmap(bitmap, null, midRect, paint)
if (src != null) canvas.drawBitmap(m, src, dstRect, paint)
else canvas.drawBitmap(m, null, dstRect, paint)
canvas.restore()
drawOsd(canvas, vm.state.value)
@@ -236,21 +279,23 @@ class LiveRenderer(
*/
private fun setFramePixels(frame: IntArray) {
val p = vm.imageParams()
val bw = bufW
val bh = bufH
if (!p.flipH && !p.flipV) {
bitmap.setPixels(frame, 0, 320, 0, 0, 320, 240)
bitmap.setPixels(frame, 0, bw, 0, 0, bw, bh)
return
}
for (y in 0 until 240) {
val sy = if (p.flipV) 239 - y else y
val srcRow = sy * 320
val dstRow = y * 320
for (y in 0 until bh) {
val sy = if (p.flipV) bh - 1 - y else y
val srcRow = sy * bw
val dstRow = y * bw
if (!p.flipH) {
System.arraycopy(frame, srcRow, flipped, dstRow, 320)
System.arraycopy(frame, srcRow, flipped, dstRow, bw)
} else {
for (x in 0 until 320) flipped[dstRow + x] = frame[srcRow + (319 - x)]
for (x in 0 until bw) flipped[dstRow + x] = frame[srcRow + (bw - 1 - x)]
}
}
bitmap.setPixels(flipped, 0, 320, 0, 0, 320, 240)
bitmap.setPixels(flipped, 0, bw, 0, 0, bw, bh)
}
/**
@@ -120,12 +120,13 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
const val PIP_RIGHT_MARGIN_DP = 32
/**
* Video is encoded at this multiple of the sensor frame (3x = 960x720, or
* 720x960 in portrait). The sensor data is only 160x120 either way, so this
* buys nothing for the image — it is what makes the burned-in readouts sharp,
* which is the whole point: at 1x the temperature text was visibly blurry.
* Video is encoded at this multiple of the RENDER BUFFER (native 160x120
* since round 38: 6x = 960x720, or 720x960 in portrait). The sensor data
* is only 160x120 either way, so this buys nothing for the image — it is
* what makes the burned-in readouts sharp, which is the whole point: at
* 1x the temperature text was visibly blurry.
*/
const val RECORD_SCALE = 3
const val RECORD_SCALE = 6
}
private var usbReceiver: android.content.BroadcastReceiver? = null
@@ -280,8 +281,10 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
session.recorderHook = { argb ->
val rec = recorder
if (rec != null && rec.isRecording()) {
val bmp = android.graphics.Bitmap.createBitmap(320, 240, android.graphics.Bitmap.Config.ARGB_8888)
bmp.setPixels(argb, 0, 320, 0, 0, 320, 240)
val bw = session.outW()
val bh = session.outH()
val bmp = android.graphics.Bitmap.createBitmap(bw, bh, android.graphics.Bitmap.Config.ARGB_8888)
bmp.setPixels(argb, 0, bw, 0, 0, bw, bh)
// Hand over the CURRENT annotations so the recording shows the same
// markers and temperatures as the screen. Read per frame because the
// user can add or remove probes while recording.
@@ -554,6 +557,10 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
fun triggerFfc() = session.triggerFfc()
/** Live buffer size the renderer must allocate (native 160x120 since round 38). */
fun bufferW(): Int = session.outW()
fun bufferH(): Int = session.outH()
fun setPalette(index: Int) {
session.setPalette(index)
_state.value = _state.value.copy(paletteIndex = index)
@@ -659,11 +666,13 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
* only in the presentation layer.
*/
/**
* Exact 160x120 sensor-grid copy of the 320x240 display buffer. The pipeline's
* 2x upscale keeps the source value at every even index (`o[2x] = src[x]`), so
* Exact 160x120 sensor-grid copy of the display buffer. With the native
* output (round 38) the buffer IS the sensor grid; the 2x-upscaled 320x240
* variant keeps the source value at every even index (`o[2x] = src[x]`), so
* sampling (2x, 2y) recovers the native pixels without any resampling.
*/
private fun sensorFrame(display: IntArray): IntArray {
if (display.size == 160 * 120) return display.copyOf()
val out = IntArray(160 * 120)
var i = 0
for (y in 0 until 120) {
@@ -874,18 +883,20 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
if (rec == null) {
// Same orientation and scale policy as a photo, so the recording matches
// the stills: the capture rotation (live rotation + physical grip) is
// baked in and the frame is encoded at RECORD_SCALE x the sensor size.
// Encoding at the native 320x240 made the burned-in readouts blurry once
// the video was viewed full screen ("视频里面的温度标点糊糊的").
// baked in and the frame is encoded at RECORD_SCALE x the render buffer
// (native 160x120). Encoding at the native size made the burned-in
// readouts blurry once the video was viewed full screen.
//
// The grip is sampled once, at start: changing hands mid-clip would need
// mid-stream rotation, and the encoder's dimensions are fixed. Every
// camera behaves this way.
val rot = captureRotationDeg()
val portrait = rot == 90 || rot == 270
val bw = session.outW()
val bh = session.outH()
val r = com.mag160c.thermal.media.Mp4Recorder(
width = if (portrait) 240 * RECORD_SCALE else 320 * RECORD_SCALE,
height = if (portrait) 320 * RECORD_SCALE else 240 * RECORD_SCALE,
width = if (portrait) bh * RECORD_SCALE else bw * RECORD_SCALE,
height = if (portrait) bw * RECORD_SCALE else bh * RECORD_SCALE,
).apply {
mirror = com.mag160c.thermal.media.PhotoSaver.Mirror(flipH, flipV)
rotDeg = rot
@@ -959,6 +970,7 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
* state on the main dispatcher.
*/
private suspend fun computeAndPublishTemps() {
pollRawCapFlag()
if (uiTick++ % 12 == 0) {
// ~5 s heartbeat: what the UI currently sees (debug round 11)
com.mag160c.thermal.media.DebugLog.log(
@@ -1033,6 +1045,84 @@ class LiveViewModel(app: Application) : AndroidViewModel(app) {
}
}
// ---- debug raw-frame capture (round 38, rendering_diff_phone_20261006.md §16.3) ----
//
// The residual row-FPN investigation needs the RAW sensor frames (raw AND the
// closed-shutter reference frames of a full FFC window) replayed offline. The
// switch is a flag FILE rather than UI or a broadcast receiver: `adb shell
// su -c 'touch /data/data/com.mag160c.thermal/files/rawcap.enable'` starts a
// capture (and fires a manual FFC so the window is surely inside), deleting the
// flag stops early; the capture also stops by itself after RawFrameRecorder's
// frame budget and clears the flag. Progress lands in `rawcap.state`.
@Volatile
private var rawCap: com.mag160c.thermal.usb.RawFrameRecorder? = null
/** Scratch for the NUC snapshot handed to the recorder (reader thread only). */
private val rawCapNuc = IntArray(19200)
private fun rawCapFlag(): java.io.File =
java.io.File(getApplication<Application>().filesDir, "rawcap.enable")
private fun rawCapState(line: String) {
try {
java.io.File(
getApplication<Application>().filesDir, "rawcap.state",
).writeText(line + "\n", Charsets.UTF_8)
} catch (_: Exception) {
}
}
/** Poll (400 ms, from the temperature loop): start/stop the capture per the flag. */
private fun pollRawCapFlag() {
val active = rawCap
val flag = rawCapFlag()
if (active == null) {
if (flag.exists()) startRawCap()
} else if (!flag.exists()) {
// flag removed while recording -> early stop (frames so far are kept)
stopRawCap("flag_removed")
}
}
private fun startRawCap() {
if (rawCap != null) return
val ctx = getApplication<Application>()
val dir = java.io.File(ctx.filesDir, "rawcap").apply { mkdirs() }
val name = "run_" + java.text.SimpleDateFormat(
"yyyyMMdd_HHmmss", java.util.Locale.US,
).format(java.util.Date())
val rec = com.mag160c.thermal.usb.RawFrameRecorder(java.io.File(dir, name))
rawCap = rec
session.rawFrameSink = { raw, phase, shutter, rendered ->
val r = rawCap
if (r != null) {
val hasNuc = rendered && session.copyNuc(rawCapNuc)
val more = r.offer(raw, phase, shutter, rendered, if (hasNuc) rawCapNuc else null)
if (!more) stopRawCap("frames_done")
}
}
rawCapState("running $name")
com.mag160c.thermal.media.DebugLog.log(
"vm", "rawcap started -> $name (max ${com.mag160c.thermal.usb.RawFrameRecorder.DEFAULT_MAX_FRAMES} frames)",
)
// make sure a full FFC window is inside the capture; a no-op while no
// session is up (then the automatic first-frame FFC of a new connection
// is captured instead)
session.triggerFfc()
}
private fun stopRawCap(reason: String) {
val rec = rawCap ?: return
rawCap = null
session.rawFrameSink = null
rec.close()
rawCapFlag().delete()
val line = "done ${rec.file.name} frames=${rec.frameCount} reason=$reason path=${rec.file.absolutePath}"
rawCapState(line)
com.mag160c.thermal.media.DebugLog.log("vm", "rawcap $line")
}
private var uiTick = 0
override fun onCleared() {
@@ -110,6 +110,15 @@ class IrSession(context: Context) {
@Volatile
var rawHook: ((ByteArray) -> Unit)? = null
/**
* Optional debug raw-frame capture sink (see [RawFrameRecorder]), runs on the
* reader thread after the pipeline processed the frame, so phase/shutter
* describe exactly this frame. The `raw` buffer is reused for the next frame —
* copy or write it out synchronously.
*/
@Volatile
var rawFrameSink: ((raw: ByteArray, phase: Int, shutter: Int, rendered: Boolean) -> Unit)? = null
fun isStreaming(): Boolean = running.get()
/** Latest raw frame (with 0x38-byte header) for MDT capture. */
@@ -158,6 +167,10 @@ class IrSession(context: Context) {
fun identitySnapshot(): CameraIdentity = identity.copy()
/** Output buffer size of the active pipeline (160x120 with nativeOut). */
fun outW(): Int = pipeline?.outWidth ?: 320
fun outH(): Int = pipeline?.outHeight ?: 240
/**
* Connect + start the live stream. Must be called after USB permission.
*
@@ -349,6 +362,11 @@ class IrSession(context: Context) {
// table interpolation (official UsbCommunication.startTransfer); it is
// set from the device's parameter block below
fpaTempFix = 0,
// official android display path: no intermediate 2x stage inside the
// pipeline — the consumer (LiveRenderer) draws the native 160x120
// palette image to the screen in ONE filtered scale, exactly like the
// official ImageViewer (round 38, rendering_diff §17)
nativeOut = true,
)
if (!pipe.loadDdt(ddt)) {
DebugLog.log("session", "ddt_fail (fetched/bundled ${ddt.size} B not loadable)")
@@ -606,7 +624,7 @@ class IrSession(context: Context) {
loopRunning.set(true)
val stream = FrameStream(38400)
val frameBuf = ByteArray(0x38 + 38400)
val out = IntArray(320 * 240)
val out = IntArray(pipe.outWidth * pipe.outHeight)
val tmp = ByteArray(0x8000)
val noop = ByteArray(0)
DebugLog.log("stream", "reader loop start: sync bulkTransfer (${TIMEOUT_MS} ms) + halt recovery")
@@ -708,6 +726,7 @@ class IrSession(context: Context) {
// camera temperature) refers to THIS frame, which is what a
// remote client needs to reproduce the image.
lastRawFrame?.let { raw -> rawHook?.invoke(raw) }
rawFrameSink?.invoke(frameBuf, pipe.ffcPhase(), pipe.lastShutter(), rendered)
if (rendered) {
renderCount++
listener?.onFrameReady(out)
@@ -0,0 +1,144 @@
package com.mag160c.thermal.usb
import java.io.File
import java.io.FileOutputStream
import java.util.Locale
/**
* Debug raw-frame capture (round 38, docs/android_app/rendering_diff_phone_20261006.md
* §16.3): writes the streamed USB frames — raw, WITH the 0x38-byte header — to one
* binary file so the render chain can be replayed stage by stage OFFLINE
* (raw -> reference -> d2 -> NUC), which is how the residual row-FPN we have and the
* official app doesn't is being located. The capture must cover a full FFC window
* (closed shutter -> 4 reference frames -> table rebuild), so the flag file also
* triggers a manual FFC when a session is already streaming (LiveViewModel).
*
* Not for shipping analysis — a debug tool. Enabled by LiveViewModel through the
* `rawcap.enable` flag file; a machine-readable status line lands in `rawcap.state`
* so a device test can be driven over adb without the debug log.
*
* File layout (all integers u32 little-endian):
*
* global header: ASCII "MAGRCAP1" + u32 width + u32 height (8+4+4 bytes)
* per frame record:
* u32 magic 0x1BB0CAFE | u32 seq | u32 phase | u32 shutter | u32 flags
* [0x38 + 38400] bytes raw USB frame (header included)
* [flags & HAS_NUC] u32 0x1BB0C0DE + 38400 bytes NUC counts as u16 LE
*
* phase mirrors the pipeline state AFTER the frame was processed:
* 0 normal / 1 shutter closed / 2 reference collection. The NUC block is only
* present for rendered (normal) frames — during an FFC window the pipeline
* holds its NUC buffer stale.
*/
class RawFrameRecorder(
val file: File,
private val maxFrames: Int = DEFAULT_MAX_FRAMES,
) {
companion object {
/** ~20 s at 15 fps: covers warmup + two full FFC windows with margin. */
const val DEFAULT_MAX_FRAMES = 300
private const val FRAME_MAGIC = 0x1BB0CAFE.toInt()
private const val NUC_MAGIC = 0x1BB0C0DE.toInt()
const val FLAG_RENDERED = 1
const val FLAG_HAS_NUC = 2
const val NUC_PIXELS = 160 * 120
const val RAW_FRAME_LEN = 0x38 + 38400
fun magic(): Int = FRAME_MAGIC
fun nucMagic(): Int = NUC_MAGIC
}
private val lock = Any()
private var out: FileOutputStream? = null
private var seq = 0
private var failed = false
/** Frames accepted so far (opened lazily on the first offer). */
val frameCount: Int get() = synchronized(lock) { seq }
/**
* Write one frame record. Synchronous (the caller is the USB reader thread and
* the buffer is reused for the next frame). Returns false when the capture is
* finished — full, failed, or closed — so the owner can unhook its sink.
*/
fun offer(frame: ByteArray, phase: Int, shutter: Int, rendered: Boolean, nuc: IntArray?): Boolean =
synchronized(lock) {
if (failed || seq >= maxFrames) return false
val stream = out ?: run {
// lazy open: the flag can be set before the camera streams
try {
file.parentFile?.mkdirs()
FileOutputStream(file).also { opened ->
out = opened
opened.write(HEADER)
opened.write(le32(160))
opened.write(le32(120))
}
} catch (e: Exception) {
failed = true
return false
}
out!!
}
val hasNuc = rendered && nuc != null && nuc.size >= NUC_PIXELS
val flags = (if (rendered) FLAG_RENDERED else 0) or (if (hasNuc) FLAG_HAS_NUC else 0)
val head = ByteArray(20)
put32(head, 0, FRAME_MAGIC)
put32(head, 4, seq)
put32(head, 8, phase)
put32(head, 12, shutter)
put32(head, 16, flags)
try {
stream.write(head)
stream.write(frame, 0, RAW_FRAME_LEN)
if (hasNuc) {
val nb = ByteArray(4 + NUC_PIXELS * 2)
put32(nb, 0, NUC_MAGIC)
for (i in 0 until NUC_PIXELS) {
val v = nuc[i].coerceIn(0, 0xFFFF)
nb[4 + i * 2] = (v and 0xFF).toByte()
nb[4 + i * 2 + 1] = ((v shr 8) and 0xFF).toByte()
}
stream.write(nb)
}
// flush to the page cache: keeps the file readable while recording
// (crash-safe enough for a debug tool) without an fsync per frame
stream.flush()
} catch (e: Exception) {
failed = true
return false
}
seq++
seq < maxFrames
}
/** Flush + close. Idempotent. */
fun close() {
synchronized(lock) {
try {
out?.flush()
out?.close()
} catch (_: Exception) {
}
out = null
}
}
private fun le32(v: Int): ByteArray {
val b = ByteArray(4)
put32(b, 0, v)
return b
}
private fun put32(b: ByteArray, off: Int, v: Int) {
b[off] = (v and 0xFF).toByte()
b[off + 1] = ((v shr 8) and 0xFF).toByte()
b[off + 2] = ((v shr 16) and 0xFF).toByte()
b[off + 3] = ((v shr 24) and 0xFF).toByte()
}
private val HEADER = "MAGRCAP1".toByteArray(Charsets.US_ASCII)
}
@@ -0,0 +1,74 @@
package com.mag160c.thermal.core
import org.junit.Assert.assertArrayEquals
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* Round-38 (rendering_diff_phone_20261006.md §17): the official ANDROID display
* path has no intermediate scaling stage — the SDK hands the app a native
* 160x120 bitmap and the app draws it once with a filtering paint. The native
* output mode must (a) produce exactly the sensor-grid samples of the classic
* 320x240 path (the upscale's even indices were always the untouched source
* values) and (b) keep the golden 320x240 path untouched by default.
*/
class NativeOutTest {
private fun res(name: String): ByteArray =
javaClass.classLoader.getResourceAsStream(name)!!.readBytes()
private fun run(nativeOut: Boolean): Pair<RenderPipeline, MutableList<IntArray>> {
val pipeline = RenderPipeline(
w = 160, h = 120,
ffcPeriod = 1800, ffcDrift = 250, warmFrames = 20,
cdfPivot75 = true, force75 = true,
onFfc = {},
nativeOut = nativeOut,
)
assertTrue(pipeline.loadDdt(res("mag160c_official.ddt")))
val frames = res("test_frames.bin")
val frameSize = 0x38 + 38400
val nframes = frames.size / frameSize
val w = pipeline.outWidth
val h = pipeline.outHeight
val out = IntArray(w * h)
val rendered = ArrayList<IntArray>()
for (f in 0 until nframes) {
val fr = frames.copyOfRange(f * frameSize, (f + 1) * frameSize)
if (pipeline.frame(fr, true, out)) rendered.add(out.copyOf())
}
assertTrue("expected rendered frames", rendered.isNotEmpty())
return pipeline to rendered
}
@Test
fun nativeOutputMatchesEvenSamplesOfThe2xPath() {
val (p2x, frames2x) = run(false)
assertEquals(320, p2x.outWidth)
assertEquals(240, p2x.outHeight)
val (pNat, framesNat) = run(true)
assertEquals(160, pNat.outWidth)
assertEquals(120, pNat.outHeight)
assertEquals(frames2x.size, framesNat.size)
for (fi in frames2x.indices) {
val big = frames2x[fi]
val nat = framesNat[fi]
assertEquals(160 * 120, nat.size)
var i = 0
for (y in 0 until 120) {
val row = y * 2 * 320
for (x in 0 until 160) {
assertEquals("frame $fi pixel ($x,$y)", big[row + x * 2], nat[i])
i++
}
}
}
}
@Test
fun defaultPathStays320x240() {
val pipeline = RenderPipeline(w = 160, h = 120, onFfc = {})
assertEquals(320, pipeline.outWidth)
assertEquals(240, pipeline.outHeight)
}
}
@@ -0,0 +1,115 @@
package com.mag160c.thermal.usb
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Rule
import org.junit.Test
import org.junit.rules.TemporaryFolder
import java.io.File
import java.nio.file.Files
/**
* Format lock for the debug raw-frame capture: the offline replay
* (rendering_diff_phone_20261006.md §16.3) parses these files with a
* Python tool, so the on-disk layout — header string, little-endian u32
* fields, NUC block only for rendered frames — must stay exactly as
* documented in [RawFrameRecorder].
*/
class RawFrameRecorderTest {
@get:Rule
val tmp = TemporaryFolder()
private fun rawFrame(seq: Int): ByteArray {
val f = ByteArray(RawFrameRecorder.RAW_FRAME_LEN)
// header marker + sequence at offset 4 (the USB header's counter field)
f[0] = 0x55.toByte(); f[1] = 0xAA.toByte()
f[4] = (seq and 0xFF).toByte()
f[5] = ((seq shr 8) and 0xFF).toByte()
// a distinguishable pixel: first u16 of the payload
f[0x38] = 0x34; f[0x39] = 0x12
return f
}
private fun nucBuffer(seed: Int): IntArray =
IntArray(RawFrameRecorder.NUC_PIXELS) { (it + seed) and 0xFFFF }
@Test
fun writesHeaderAndRoundTrips() {
val f = File(tmp.root, "run_test.bin")
// offer() returns "keep recording": false on the LAST budgeted frame
// (which is still written) and on any call after that
val rec = RawFrameRecorder(f, maxFrames = 3)
val raw0 = rawFrame(0)
val raw1 = rawFrame(1)
val raw2 = rawFrame(2)
assertTrue(rec.offer(raw0, phase = 0, shutter = 11442, rendered = true, nuc = nucBuffer(7)))
assertTrue(rec.offer(raw1, phase = 2, shutter = 15084, rendered = false, nuc = null))
assertFalse(rec.offer(raw2, phase = 0, shutter = 11444, rendered = true, nuc = nucBuffer(9)))
assertFalse(rec.offer(rawFrame(3), 0, 0, true, null)) // budget exhausted
rec.close()
assertEquals(3, rec.frameCount)
val bytes = Files.readAllBytes(f.toPath())
assertEquals("MAGRCAP1", String(bytes, 0, 8, Charsets.US_ASCII))
assertEquals(160, le32(bytes, 8))
assertEquals(120, le32(bytes, 12))
var off = 16
// frame 0: rendered with NUC block
assertEquals(RawFrameRecorder.magic().toLong(), le32(bytes, off).toLong() and 0xFFFFFFFFL)
assertEquals(0, le32(bytes, off + 4))
assertEquals(0, le32(bytes, off + 8))
assertEquals(11442, le32(bytes, off + 12))
assertEquals(RawFrameRecorder.FLAG_RENDERED or RawFrameRecorder.FLAG_HAS_NUC, le32(bytes, off + 16))
assertTrue(
bytes.copyOfRange(off + 20, off + 20 + 4).contentEquals(
raw0.copyOfRange(0, 4),
),
)
off += 20 + RawFrameRecorder.RAW_FRAME_LEN
assertEquals(RawFrameRecorder.nucMagic().toLong(), le32(bytes, off).toLong() and 0xFFFFFFFFL)
assertEquals(7, le16(bytes, off + 4))
assertEquals(9, le16(bytes, off + 4 + 2 * 2))
off += 4 + RawFrameRecorder.NUC_PIXELS * 2
// frame 1: FFC window frame, no NUC block
assertEquals(2, le32(bytes, off + 8))
assertEquals(15084, le32(bytes, off + 12))
assertEquals(0, le32(bytes, off + 16))
off += 20 + RawFrameRecorder.RAW_FRAME_LEN
// frame 2
assertEquals(11444, le32(bytes, off + 12))
off += 20 + RawFrameRecorder.RAW_FRAME_LEN + 4 + RawFrameRecorder.NUC_PIXELS * 2
assertEquals(bytes.size, off)
}
@Test
fun opensLazilyAndRejectsAfterClose() {
val f = File(tmp.root, "run_lazy.bin")
val rec = RawFrameRecorder(f, maxFrames = 2)
// no file before the first offer (the flag can be set before streaming)
assertFalse(f.exists())
val nuc = nucBuffer(0)
assertTrue(rec.offer(rawFrame(0), 0, 100, true, nuc))
assertFalse(rec.offer(rawFrame(1), 0, 101, true, nuc)) // last budgeted frame
assertFalse(rec.offer(rawFrame(2), 0, 102, true, nuc)) // rejected
assertEquals(2, rec.frameCount)
rec.close()
rec.close() // idempotent
val expectLen = (2 * (20 + RawFrameRecorder.RAW_FRAME_LEN) + 16 +
2 * (4 + RawFrameRecorder.NUC_PIXELS * 2)).toLong()
assertEquals(expectLen, f.length())
}
private fun le32(b: ByteArray, off: Int): Int =
(b[off].toInt() and 0xFF) or
((b[off + 1].toInt() and 0xFF) shl 8) or
((b[off + 2].toInt() and 0xFF) shl 16) or
((b[off + 3].toInt() and 0xFF) shl 24)
private fun le16(b: ByteArray, off: Int): Int =
(b[off].toInt() and 0xFF) or ((b[off + 1].toInt() and 0xFF) shl 8)
}
Binary file not shown.
+7
View File
@@ -305,6 +305,13 @@ build-artifacts\csdk_test_display.exe
(SDK 给 App 的位图就是 160×120,App 用 `new Paint(1)`(无 FILTER_BITMAP)最近邻放大),
而 PC 版的双线性会展出一层"全幅/半幅"交错细纹(暗/低温区最明显)。
真机实测:冷区细尺度列条纹比 1.16/1.29(双线性)→ **1.00/0.93**(最近邻)。
- **38 轮补充(显示链两级结构)**:官方 `ImageViewer` 是**两级**——160×120 SDK
位图先用 `PaintFlagsDrawFilter + FILTER_BITMAP` 画进 **640×480 离屏**(4× 双线性),
SurfaceView 再放大到屏。中间 4× 对逐行 FPN 图样有 ~0.6× 低通:同一份 NUC 数据
走官方两级链复刻 0.12、走单级 12× 复刻 0.32(行交替,×1e-2 弧长)——**官方
"背景条纹更不明显"的主导机制是显示链**。Kotlin 侧已按此两级实现
(`RenderPipeline.nativeOut` 输出原生 160×120 + `LiveRenderer` 的 640×480 中间位图);
C 参考路径(`upscale2x`/`upscale_nearest`)保持不变,仅供黄金夹具与 PC 路径。
- 顺带(同一条链):**曲线插值温度要加 `BasePara2.fpaTempFix`**。官方 App 传给 SDK 的是
`帧尾 FPA + fpaTempFix`(真机 1699),`SelectCaliCurve`/`GenerateVolatileLens`
用它选/插值每像素标定表;温度换算链路仍用帧尾原值。C 侧**已实现**
@@ -678,3 +678,91 @@ JPEG 只能解释一部分:把我们那张(960×720 q95,202 KB)按 640×
2. 离线重放:raw → 参考 → NUC → 表插值 → LUT,**每一级都算"沿行均值 std"**,
找出引入行偏移的那一级;再用录到的数据做 A/B(改窗口/改表温度/改盲元)复现官方数值。
3. 真机 A/B 验收指标就用 16.2 的表:目标"逐行固定偏移 std ≤ 0.2"。
## 17. 第三十八轮:原始帧录制 + 逐级重放全链路定案 + 显示链修复(2026-10-07 凌晨)
按 §16.3 清单执行。工具:`IrSession.rawFrameSink` + `RawFrameRecorder`(调试开关,
`files/rawcap.enable` 标志文件触发,录 raw+NUC dump 到 `files/rawcap/run_*.bin`,
带 FFC 触发,300 帧自动停);离线重放脚本在 `%TEMP%\mag160c_re\`(临时件)。
### 17.1 逐级重放(整数语义复刻,与真机 NUC dump 互锁)
录制 300 帧(含 3 次完整 FFC 窗口)。Python 逐级复刻(i32/算术移位/向零截断与
Kotlin 一致),与 App 自己 `copyNuc` 的 dump 对拍:corr=0.9996,仅盲元像素大差异
——**复刻可信**。各级行交替(alt = 行均值相邻二阶差,固定分量):
| 级 | 行交替 std(counts) |
|---|---|
| raw(场景帧固定行剖面) | 596.9 |
| ref(4 帧快门平均) | 596.7(与 raw 图样相关 0.999) |
| d2 = (raw−ref)>>1 | 5.20 |
| NUC(查表后,App dump) | 4.25 |
**结论 1:参考减除消掉 98.4% 的行 FPN,表再消一半,残差 4.25 counts 就是
显示条纹的直接来源**(换算到显示域 ≈ §16.2 的我们 1.2–1.84)。
### 17.2 排除清单(都做了 A/B)
- **参考窗相位**(±2 帧移动):4.01–4.33,不敏感。
- **参考平均长度**(1/4/6/8 帧):4.14–4.51,不敏感(时间噪声早已被平均压低)。
- **表插值温度**(33637…47664 扫描):4.06–4.57,不敏感。
- **残差时间结构**:FFC 后 1–10.3 s 固定分量 5.14–5.34,完全平坦 → 不是参考老化。
- **跨 FFC 窗口**:d2 残差图样相关 +0.986 → 不是每次 FFC 的随机量。
- **残差与 raw/ref 图样相关** ≈ 0.02/0.03 → 白状固定残差,**即"快门态与开放态的
行 offset FPN 图样差"(≈1.7%)**,设备固有。
- **GenerateVolatileLens**(0x18314):11 段温度响应折线,只进温度链(`this+3163`
默认 1 时直通),与图像链无关 → "官方把参考融合进表"的假设**不成立**。
- **官方 SDK 全函数核对**(本轮新读):`CalibrationMain_RTV`(段选择/`(d*gain>>12)+off`/
布局与步进逐字段一致)、`PushBaselineRaw`(双累积器 +14212/+14244)、
`CAccumulator::Reset/PushFrame/Restart`(mode=baseLineAcc=4 恰好 4 帧块平均)、
`Raw2Temperature`/`CFunctions::PushFrame`(0x3808=1 旁路、每帧渲染、FFC 条件同相)、
`TempRangeStat`(纯 min/max/mean)、`BlindCompensation`(typ 3..8 邻域 switch)、
`InterpolateShort/UShort`(Q12 端点插值同式)、`SelectCaliCurve`、`DoSomething`、
`Controller::PushFrame/StartProcess`(A=5/E=4/C=4 同相;PushBaselineRaw 收 JNI 原样
帧,Java `ThreadImgProcess` 不做预处理)。**官方每帧链路与我们逐字段一致,没有
任何"额外处理"能解释残差差**。
### 17.3 显示域 A/B(同场景同铁虹,§16.2 口径:12px 中值网格 + 弧长 + hf3 行/列剖面)
| 测量 | ROW-dir | COL-dir |
|---|---|---|
| 官方截图(横屏) | 0.12 | 0.31 |
| 我们修复前截图 | 1.41 | 0.46 |
| 我们修复后截图 | 1.32 | 0.59 |
| 我们修复后截图(**mask 掉 UI 覆盖层后**) | 0.76 | 0.67 |
**重要测量教训**:我们的竖屏 UI 把中心温度/MAX 标注/色标条画在热像区内,它们对
行剖面的贡献与真实 FPN 同量级——**对比测量必须 mask**(官方 UI 覆盖为 0)。
§16.2 的"8–17×"里含此混淆 + 几何错位(37 轮连拍的官方横屏 crop 与我们竖屏
行/列方向不一致);本轮修正后**真实 NUC 域差距约 1.6×**(同一份我们的数据走
官方显示链复刻 = 0.19,走我们链 = 0.32,官方截图实测 0.12)。
### 17.4 定案与修复
1. **官方显示链的两级结构**(jadx `ImageViewer` + 本轮重核):SDK 输出 160×120
bitmap → `PaintFlagsDrawFilter + FILTER_BITMAP` 画入 **640×480 离屏**(4× 双线性)
→ SurfaceView 再放大到屏。**中间 4× 双线性对逐行 FPN 图样有 ~0.6× 的低通**——
同一份我们的 NUC 数据:走官方两级链复刻 = 0.12,走我们单级 12× 复刻 = 0.32。
**"官方背景条纹更不明显"的主导机制是显示链,不是 NUC 域**。
2. **修复 A(nativeOut)**:`RenderPipeline` 去掉 PC 版遗留的内部 2× 插值级
(官方安卓没有),输出原生 160×120 调色板图;`LiveRenderer`/录像/拍照全链
适配。复刻证明它对行交替数值无影响,但消除了自造的"全幅/半幅"行调制
(§35 的竖列交错在行方向的同类项),显示等价性更完整。
3. **修复 B(640×480 中间级)**:`LiveRenderer` 按官方 viewer 补上 640×480 离屏
位图(FILTER_BITMAP 4×)再上屏,**显示链与官方逐级等价**。
4. **遗留(诚实记录)**:UI mask 后我们 0.76 vs 官方 0.12 仍有差距,其中
a) NUC 域残差 2.87 vs 官方等效 ~1.3–1.8 counts(**1.6×**,同显示链对比)——
与"快门态/开放态行 offset FPN 图样差"相关,设备固有,官方同算法也应承担,
差异来源未定位(疑似两 App 会话时设备状态不同);
b) 真机 GPU 显示链与 PIL 复刻间仍有 ~2× 未解释差(复刻 0.34 vs 真机 0.76),
需下一轮用"真机截图 vs 同帧 NUC dump 的逐像素对照"收敛。
§16.3 的验收目标"逐行 std ≤ 0.2"未达成;下一轮从 b) 项入手(它是纯测量
差异还是真机显示链仍有未建模环节)。
### 17.5 工具与现场
- `RawFrameRecorder`/`rawFrameSink`/flag 轮询已合入(调试工具,默认不活动)。
- `DebugLog.enabled` 本轮临时 true,提交前已改回 false。
- 现场状态:rotation 锁竖屏(原值 1,**待用户恢复**);我们 App 前台出流;
MIUI 的 App 自动启动会在授权后把焦点拉回我们 App,切官方测前先 force-stop
我们,用 `TAB×2+ENTER` 过 USB 弹窗(横屏时 input tap 坐标不可靠)。
+39
View File
@@ -489,6 +489,45 @@
"未连接"分支显示,正常出图时用户看不到任何反馈)。
- [x] 单测 66 → **76 项全绿**;debug + release(R8) 双构建通过;APK 已更新。
## 用户反馈修复 第三十八轮(2026-10-07 凌晨,原始帧录制 + 逐级重放全链路定案 + 显示链与官方对齐)
按第三十七轮交接清单执行。用户中途解锁手机并配合真机操作。完整数据表与
排除清单见 `docs/android_app/rendering_diff_phone_20261006.md` §17。
- [x] **原始帧录制调试开关**:`usb/RawFrameRecorder`(单文件,帧头 magic/seq/
phase/shutter/flags + raw 38456B,渲染帧附 NUC u16 块)+ `IrSession.rawFrameSink`
(读线程帧入口)+ `LiveViewModel` 的 `files/rawcap.enable` 标志轮询(400ms,
置位即录并触发手动 FFC,300 帧自动停写 `rawcap.state`)。零 Manifest/协议改动。
新增 `RawFrameRecorderTest`;123 单测全绿。
- [x] **逐级重放**:Python 整数语义复刻(i32/移位/截断同 Kotlin)与真机 NUC dump
互锁(corr 0.9996)。行交替 596.9(raw)→ 5.20(d2)→ 4.25(NUC):**参考减除
消 98.4%,残差即条纹来源**。A/B 排除:参考窗相位/平均长度/表插值温度/参考
老化(10.3 s 平坦)/跨 FFC 随机——残差=快门态与开放态行 offset FPN 图样差
(设备固有 ~1.7%)。
- [x] **官方 SDK 二次核对**(本轮新读 13 函数:CalibrationMain_RTV、PushBaselineRaw、
CAccumulator 全家、Raw2Temperature、CFunctions::PushFrame、TempRangeStat、
BlindCompensation、Interpolate*、SelectCaliCurve、DoSomething、GenerateVolatileLens、
Controller::PushFrame/StartProcess + jadx Java 帧路径):**与我们逐字段一致,
官方无任何额外处理能解释残差差**;GenerateVolatileLens 只进温度链。
- [x] **显示域 A/B 定案**:同场景同铁虹(§16.2 口径)。发现并修正两大测量混淆:
①我们竖屏 UI 的中心温度/MAX 标注/色标条画在热像区内,贡献与 FPN 同量级
(mask 后 1.41→0.76);②§16.2 的 8–17× 含横屏/竖屏几何错位。真实 NUC 域
差距 **1.6×**(同一份数据走官方显示链复刻 0.19 vs 我们链 0.32,官方截图 0.12)。
- [x] **修复 A(nativeOut)**:`RenderPipeline` 去掉 PC 版遗留的内部 2× 插值级
(官方安卓无此级),输出原生 160×120;IrSession/LiveRenderer/录像(6×)/拍照
全链适配,`NativeOutTest` 锁 parity(偶数采样=旧路径)。
- [x] **修复 B(640×480 中间级)**:`LiveRenderer` 按官方 viewer 补两级显示
(160×120 →FILTER_BITMAP 4×→640×480→屏幕)。复刻证明该中间级对逐行 FPN
有 ~0.6× 低通——同数据走官方两级链=0.12 vs 我们单级=0.32,**"官方背景
条纹更不明显"的主导机制是显示链**,现已逐级等价。
- [ ] **遗留(下一轮)**:UI mask 后我们 0.76 vs 官方 0.12——NUC 域 1.6×(设备
状态嫌疑,官方同算法同承担)+ 真机 GPU 显示链与 PIL 复刻间 ~2× 未解释差;
验收目标"逐行 std ≤ 0.2"未达成,从真机显示链逐像素对照入手。
- 现场状态:rotation 仍锁竖屏(原值 1,**待恢复**);我们 App 前台出流(新版
APK);官方 App 已 force-stop;DebugLog 已改回默认关。MIUI 会把授权后焦点
拉回我们 App:切官方前先 force-stop 我们,USB 弹窗用 `TAB×2+ENTER` 过
(横屏 input tap 坐标不可靠)。
## 用户反馈修复 第三十七轮(2026-10-06 深夜,"官方背景条纹更不明显"——逆向排除全部候选,定位为逐行固定偏移;交接轮)
用户:"很明显啊,官方背景里面的条纹就是要更不明显,你看看逆向出来的代码,是不是还漏掉了什么"。