Sense · Derive · Insight

SENSE AI Truth Verification System

SENSE is a browser-based multimodal behavioral analysis system developed by Yanshu Tech (衍数科技). Using only a webcam and a microphone, it reads facial micro-expressions, voice acoustics, remote heart rate (rPPG) and gaze behavior in real time, generates AI interview questions from your context, and produces a structured truth-score and risk report.

Launch SENSE How it works
DeveloperYanshu Tech (衍数科技)
PlatformWeb browser, no install
Signals4 multimodal channels
LanguagesEnglish & Simplified Chinese

What is SENSE?

SENSE (SENSE AI Truth Verification System) is a demonstration of multimodal biometric analysis. Instead of relying on a single signal, it fuses four families of behavioral data collected during a structured Q&A session: facial micro-expressions, voice acoustics, physiological signals estimated from video, and behavioral cues. An AI layer generates context-aware questions and summarizes the session into a truth score, a risk level, key suspicions and a recommendation.

SENSE runs entirely in a web browser and needs no dedicated hardware beyond a camera and a microphone. It is designed for entertainment, research and interview-training scenarios — not for legal, medical or employment decisions.

SENSE is not a certified lie detector or a medical device. All scores are probabilistic estimates and may be inaccurate. See the full disclaimer.

How SENSE works

  1. Create a session Choose a remote secure link (valid for 1 hour, report delivered by email) or launch the live monitor interface for on-site use, and set the question limit: 5 (short), 10 (standard) or 20 (detailed).
  2. Calibrate a baseline The participant reads a short sequence aloud while SENSE records a resting acoustic and visual baseline for that person.
  3. AI generates contextual questions Based on the background information you provide, the system generates relevant questions and presents them one at a time.
  4. Real-time multimodal analysis Spoken answers are transcribed with the SenseVoice multilingual speech model [10], while SENSE continuously analyzes micro-expressions, vocal stress markers, estimated heart rate and gaze/head/hand behavior, and displays live biometrics.
  5. Final report At the end of the session SENSE outputs a truth score, risk level, key suspicions and a recommendation summary.

Core technologies

FACIAL

Micro-expression analysis

High-speed analysis of involuntary facial muscle movements based on the Facial Action Coding System (FACS), including brow furrow (AU4), smiling (AU12) and mouth-press stress indicators. [1][2]

VOICE

Voice acoustics

Prosodic analysis of pitch (F0), jitter, shimmer and harmonics-to-noise ratio (HNR) as indicators of vocal tension and instability. [8][9]

PHYSIOLOGICAL

rPPG heart rate

Remote photoplethysmography extracts pulse signals from subtle pixel-level color changes in the face — no wearable or contact sensor required. [5][6][7]

BEHAVIORAL

Behavioral cues

Tracking of gaze deviation, head-pose stability, freezing responses and hand-to-face contact as additional behavioral indicators. [2][3][4]

Methodology & validation status

Every capability SENSE displays is tied to a published method or a documented model. The table states what is measured, what it is based on, and how far the evidence goes.

Capability Measured signal Method / basis Validation status
Facial action units Brow furrow (AU4), smile (AU12), cheek raise (AU6), mouth press 52 blendshape scores from MediaPipe Face Landmarker, mapped to FACS-style action units [1][2] Model-based proxy; not certified FACS coding
Voice acoustics Pitch (F0, 75–600 Hz), local jitter, local shimmer, HNR Praat / Parselmouth algorithms [8][9] Standard acoustic measures; stress interpretation is heuristic
Heart rate (rPPG) Pulse waveform estimated from facial pixel color changes rPPG methods from the literature [5][6][7] Non-medical estimate; affected by lighting and motion
Behavioral cues Gaze offset, blink rate, shoulder stability, hand-to-face contact MediaPipe Face / Pose / Gesture landmark models [2][3][4] Behavioral indicators only
Answer transcription Spoken answers in Chinese and English SenseVoice multilingual ASR (ONNX, quantized) [10] Open-source speech model
Data transport Video and audio frames streamed to the analysis server TLS transport (HTTPS) [11] Standard TLS; not end-to-end encrypted
Truth score & risk level Weighted score over the signals above, plus an LLM summary Published internal thresholds, compared against a personal baseline (see below) Not a validated deception test [12][13][14]

Scoring model (published thresholds)

  • Voice stress: local jitter > 6% scores 2 points and > 3.5% scores 1; local shimmer > 25% scores 2 and > 16% scores 1; pitch > 320 Hz combined with jitter > 2.5% scores 1. A frame is flagged only when the total exceeds 2 points and persists for at least 3 analysis cycles.
  • Heart rate: compared with the personal baseline — more than 40% above baseline adds 15 penalty points, more than 25% adds 10, more than 13% adds 5.
  • Answer evasion: complex questions answered with fewer than 5 Chinese characters or 3 English words, together with phrases such as "I don't know", are flagged.
  • Calmness bonus: a normal-length answer with no voice, heart-rate or behavioral flags earns a 5–10 point bonus.
SENSE does not claim to detect lies. Peer-reviewed reviews find that individual physiological and behavioral signals do not reliably identify deception [12][14], and that facial movements cannot be read as direct indicators of emotion or intent [13]. SENSE is an entertainment and research demonstration.

Read the full methodology, scoring thresholds and references →

Use cases

  • Interview training — practice high-pressure interviews with live behavioral feedback.
  • HR and compliance drills — simulate structured integrity interviews for training purposes.
  • Multimodal research — explore how facial, vocal and physiological signals correlate during questioning.
  • Entertainment and events — run a "truth machine" experience with remote links or live monitor mode.

Pricing

SENSE uses a credit wallet. Each measurement session consumes credits; top-up packs are available in three sizes and payment is currently supported via WeChat Pay.

10 CR
¥ 10.00
50 CR
¥ 50.00
100 CR
¥ 100.00

Open the app and click the wallet icon to top up.

Frequently asked questions

What is SENSE?

SENSE is a browser-based multimodal behavioral analysis system developed by Yanshu Tech (衍数科技). Using only a webcam and microphone, it combines facial micro-expression analysis, voice acoustics, remote heart-rate estimation (rPPG) and gaze/head/hand behavior tracking to generate interview questions and a structured truth-score and risk report.

Is SENSE a lie detector?

No. SENSE is a demonstration of multimodal biometric analysis intended for entertainment and research purposes. It is not a certified lie detection tool or a medical device, and its results are probabilistic estimates that can be affected by lighting, camera quality and individual differences.

What hardware do I need to use SENSE?

A computer or phone with a working webcam and microphone, and a modern browser (Chrome or Edge recommended). No special sensors are required: heart rate is estimated from facial video using rPPG.

How does SENSE estimate heart rate without a wearable?

SENSE uses remote photoplethysmography (rPPG): tiny periodic color changes in facial skin caused by blood flow are extracted from camera pixels and converted into a pulse waveform. This is not a medical-grade measurement.

Which languages does SENSE support?

The SENSE interface and its AI-generated questions are available in Simplified Chinese and English.

What does the final SENSE report include?

A truth score, a risk level, key suspicions flagged from the multimodal signals, and a recommendation summary. In remote-link mode the report can be emailed to a specified recipient.

Is my video or audio stored by SENSE?

Video and audio are processed in real time to extract biometric features, and transport is encrypted over HTTPS (TLS). Do not upload sensitive or personally identifiable documents into the context engine.

How much does SENSE cost?

SENSE uses a credit wallet. Credit packs of 10, 50 and 100 credits are available, and each measurement session consumes credits. Payment is currently supported via WeChat Pay.

References

  1. Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press.
  2. Google AI Edge. MediaPipe Face Landmarker — face landmarks and 52 blendshape scores. ai.google.dev/edge/mediapipe
  3. Google AI Edge. MediaPipe Pose Landmarker. ai.google.dev/edge/mediapipe
  4. Google AI Edge. MediaPipe Gesture Recognizer. ai.google.dev/edge/mediapipe
  5. Verkruysse, W., Svaasand, L. O., & Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics Express, 16(26), 21434–21445. doi:10.1364/OE.16.021434
  6. Wang, W., den Brinker, A. C., Stuijk, S., & de Haan, G. (2017). Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering, 64(7), 1479–1491. doi:10.1109/TBME.2016.2609282
  7. de Haan, G., & Jeanne, V. (2013). Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 60(10), 2878–2886. doi:10.1109/TBME.2013.2266196
  8. Teixeira, J. P., Oliveira, C., & Lopes, C. (2013). Vocal acoustic analysis – jitter, shimmer and HNR parameters. Procedia Technology, 9, 1112–1122. doi:10.1016/j.protcy.2013.12.124
  9. Boersma, P., & Weenink, D. Praat: doing phonetics by computer. fon.hum.uva.nl/praat
  10. An, K., et al. (2024). FunAudioLLM: Voice understanding and generation foundation models for natural interaction between humans and LLMs (SenseVoice). arXiv:2407.04051. arxiv.org/abs/2407.04051
  11. IETF (2018). RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3. rfc-editor.org/rfc/rfc8446
  12. National Research Council (2003). The Polygraph and Lie Detection. The National Academies Press. doi:10.17226/10420
  13. Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. doi:10.1177/1529100619832930
  14. DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129(1), 74–118. doi:10.1037/0033-2909.129.1.74

Try SENSE in your browser

No installation. Just a webcam, a microphone and a quiet, well-lit environment.

Launch SENSE 中文介绍