Method, sources, and limitations
How image analysis, traditional interpretation rules, and source selection shape the experience—and where their limits begin.
Overview
Tongue Reader is an educational demonstration of one narrow part of Chinese medicine: observing the tongue and relating visible signs to broader pattern frameworks.
Users review estimated visible signs, then a rule-based mapping surfaces associations drawn from Chinese medicine teaching. The output represents the app's source-informed mapping, not a practitioner assessment.
The project does not use automated image analysis to validate, rank, or replace Chinese medicine. It uses technical tools to make a traditional observational framework easier to explore while keeping its limits visible.
Design goals
- Encourage careful observation without turning isolated signs into verdicts.
- Connect the interactive reading with deeper study of patterns, diagnosis, channels, and history.
- Present Chinese medicine as a living, internally diverse medical tradition with cultural, historical, and clinical context.
- Use sources according to the questions they can responsibly answer rather than placing knowledge traditions on a single hierarchy.
How the reading engine works
Image modes send each photo to the TongueSAM segmentation model first. TongueSAM runs on a GPU service hosted by Modal and isolates the tongue from the rest of the image. If that service cannot return a usable mask, the browser uses MediaPipe to crop the mouth region. In the same Modal request, four classifiers estimate body color and coating properties. The browser adds lightweight shape and color checks, and the user reviews every suggested feature before interpretation.
Source-annotated rules weight combinations of signs. When references support more than one interpretation, the reading preserves that ambiguity.
How photos are handled
A photo submitted for automated analysis passes through a protected Vercel server route to the model service on Modal. The app processes it in memory for that request and does not save it to a database, object store, browser storage, or application log. The service returns a tongue mask and classifier probabilities; lighter color and shape checks then run in the browser.
Vercel and Modal may process ordinary request metadata as part of operating their infrastructure. Avoid submitting a photo if you do not want it transmitted to those services; manual observation remains available without uploading an image.
Open-source model foundations
Tongue segmentation uses the MIT-licensed TongueSAM project, pinned to a specific upstream revision. Its pipeline combines a YOLOX prompt generator with SAM ViT-B. Four ResNet50-SE classifiers adapted from the AGPL-licensed TongueDiagnosis project estimate tongue-body color and coating color, thickness, and greasiness. Full provenance and license references are recorded in the project's third-party notices.
Disclaimers
This educational guide is not designed or validated as a medical device and does not provide diagnosis, complete pattern identification, or treatment.
Image models estimate visible features; a rule-based mapping, not a practitioner assessment, generates the reading.
Chinese medicine assessment combines the Four Examinations and the person's history and circumstances. This app covers only a narrow part of looking.
Do not use this demo to replace or delay appropriate care. Consult qualified Chinese medicine and other health practitioners for individualized care within their respective areas of practice.
Safety framing draws on NCCIH and the WHO Global Traditional Medicine Strategy 2025–2034.
Sources
Sources are grouped by purpose: Chinese medicine theory and practice, history, image research, and health and safety context.
Source policy: this app does not place Chinese medicine and biomedicine on a single ladder of authority. It asks which source is appropriate to each claim. Traditional concepts and associations are grounded in Chinese medicine teaching; claims about cameras and automated analysis are grounded in imaging and reliability research; safety language draws on professional and public-health guidance.
Institutional health and safety guidance
Public-health, safety, and care-context sources
Used for safety language, evidence boundaries, practitioner guidance, and discussion of how traditional and other health systems may be used responsibly.
National Center for Complementary and Integrative Health (NCCIH)
Traditional Chinese Medicine: What You Need To Know
- Used for
- Medical disclaimers, evidence framing, treatment-safety cautions, and guidance not to replace or delay appropriate care.
- Limitations
- A public-health overview rather than a source for detailed traditional pattern correspondences.
World Health Organization
Global Traditional Medicine Strategy 2025–2034
- Used for
- Contemporary framing of traditional medicine through evidence, safety, regulation, cultural respect, and people-centred care.
- Limitations
- A global policy framework; it does not evaluate this app or establish the meaning of individual tongue signs.
Peer-reviewed research on reliability and imaging
Empirical research for image and measurement questions
Used to characterize what standardized tongue photographs may preserve, where observer agreement varies, and why image quality and capture conditions matter.
Wang et al. (2020), JMIR mHealth and uHealth
Intra-Rater and Inter-Rater Reliability of Tongue Coating Diagnosis in Traditional Chinese Medicine Using Smartphones
- Used for
- Smartphone-image reliability, classification standardization, and caution around tongue-body color and slippery coating.
- Limitations
- Assesses clinician agreement under a defined scheme; it does not establish that an automated tool can reproduce full TCM pattern differentiation.
O'Brien and Birch (2009), Journal of Alternative and Complementary Medicine
A Review of the Reliability of Traditional East Asian Medicine Diagnoses
- Used for
- The app's emphasis on diagnostic variability, context, and avoiding definitive pattern claims from isolated signs.
- Limitations
- A reliability review, not an evaluation of this specific rule engine or its outputs.
Jiang et al. (2021), BMC Medical Informatics and Decision Making
Tongue Image Quality Assessment Based on a Deep Convolutional Neural Network
- Used for
- Image-quality considerations including exposure, blur, staining, posture, and standardized capture.
- Limitations
- Evaluates image-quality screening, not the quality of a complete practitioner assessment or the usefulness of treatment.
Lo et al. (2012), Evidence-Based Complementary and Alternative Medicine
The Study on the Agreement between Automatic Tongue Diagnosis System and Traditional Chinese Medicine Practitioners
- Used for
- Background on agreement testing between computerized tongue features and practitioner observation.
- Limitations
- An earlier system and limited study design; its findings should not be generalized to every tongue-reading system.
Historical and textual scholarship
Historical sources for chronology and intellectual context
Used for chronology, classical-text development, and the modern institutional formation of Chinese medicine.
Paul U. Unschuld
Medicine in China: A History of Ideas
- Used for
- The history of Chinese medical ideas, classical development, and contextual interpretation of medical systems.
- Limitations
- A scholarly historical interpretation rather than a guide to contemporary practice.
Paul U. Unschuld
Huang Di Nei Jing Su Wen: Nature, Knowledge, Imagery in an Ancient Chinese Medical Text
- Used for
- Dating, composition, and intellectual context of the Suwen and its foundational concepts.
- Limitations
- Focuses on textual history and interpretation rather than contemporary practice or outcomes research.
Nathan Sivin
Traditional Medicine in Contemporary China
- Used for
- The relationship between classical medicine, twentieth-century standardization, and contemporary Chinese medical education.
- Limitations
- Published in 1987; useful historically but not a current account of regulation or practice.
UNESCO Memory of the World Register
Ben Cao Gang Mu (Compendium of Materia Medica) nomination
- Used for
- Documentary-history context for Li Shizhen's Bencao Gangmu and its international cultural significance.
- Limitations
- Supports documentary heritage claims, not the safety or efficacy of materia medica.
Traditional clinical and educational references
Chinese medicine sources for theory and practice
These sources support descriptions of TCM concepts, tongue signs, pattern associations, and practitioner reasoning within Chinese medicine. They are evaluated for their standing and usefulness within that tradition, not by asking them to serve as biomedical research.
Giovanni Maciocia
The Foundations of Chinese Medicine, 3rd edition
- Used for
- Pattern theory, tongue features, diagnostic nuance, and established English-language clinical terminology.
- Limitations
- An influential English-language practitioner textbook representing a particular synthesis and translation of Chinese medicine.
- Used for
- Worked examples of multi-feature tongue interpretation and ambiguity handling.
- Limitations
- Illustrative teaching cases rather than a representative account of every lineage or practitioner approach.
Sacred Lotus
Chinese medicine theory, pattern, channel, and tongue references
- Used for
- Cross-referencing coating categories, Zang-Fu patterns, the Four Examinations, channel relationships, and observation guidance.
- Limitations
- A practitioner educational reference; interpretations may reflect its authors' teaching approach.
- Used for
- Pattern definitions, relationship mapping, mechanisms described within TCM, and plain-language summaries.
- Limitations
- A commercial practitioner-informed knowledge base; its summaries are cross-checked against other Chinese medicine references.
- Used for
- The app's feature taxonomy for body color, coating, shape, moisture, and the traditional tongue-region map.
- Limitations
- A secondary online teaching reference with limited publication metadata; important claims are cross-checked where possible.
Yosan University
Traditional Chinese Medicine Theory
- Used for
- Introductory scaffolding for TCM theory, meridians, and educational terminology.
- Limitations
- Institutional teaching content rather than primary historical scholarship or a comprehensive account of all TCM lineages.
Yin Yang House
Pattern and relationship reference
- Used for
- Pattern adjacency and examples of relational rather than one-symptom reasoning.
- Limitations
- A practitioner website used as a secondary teaching reference; relationship maps reflect its own editorial model.
Supplementary consumer-facing material
Plain-language support
Used only to help identify accessible everyday language. It is not used alone to establish historical claims, traditional pattern correspondences, or image-analysis limitations.
- Used for
- Occasional plain-language inspiration for describing lived experience.
- Limitations
- Consumer-facing material whose language is used selectively and cross-checked against dedicated Chinese medicine references.