Home
  • Bio
  • Publications
  • Projects
  • Blogs
  • Experience/Education
  • Talks
  • Projects
    • Effective RNA Velocity Analysis: Math, Implementation, Benchmark, and Discovery of Meaningful Tumor Trajectories from Real-World Single-Cell Genomics
    • Single-Cell Genomic and Clinical Data Analyses Pinpointed Targets and Drove the Preclinically Validated Novel Strategy To Prevent/Treat kidney cancer.
    • Quantify Un-/Under- Annotated Features (hERVs, Transgenes) from (Single-Cell) Transcriptomics: Math Algorithms, Implementation and Turnkey Workflows
    • Integrated Analysis of Spatial Genomics and scRNAseq Identifies Occult Cervical Cancer Subtypes: Why Current Billion Dollar Screening Programs Fail the Most At-Risk Patients
    • Improve UMI Normalization that Determines the Downstream Interpretation of Single-Cell Transcriptomes
    • Infection-Driven Uterine Tumorigenesis: Integrated Spatial and Single-Cell Profiling Validated in Patients and Preclinical Models
  • Projects
  • Experience
    • Integrated single-cell, genomic, and clinical analyses enabled a preclinically validated strategy to prevent or treat kidney cancer
    • Unveiling asymptomatic infection in situ using lung slice models
    • Hyperactivated YAP1 Drives an Invasive EMT Subtype of Cervical Squamous Cell Carcinoma
    • Hyperactivated YAP1 is essential for sustainable progression of renal clear cell carcinoma
    • Targeting the disrupted Hippo signaling to prevent neoplastic renal epithelial cell immune evasion
    • HPV-YAP1 oncogenic alliance drives malignant transformation of fallopian tube epithelial cells
    • Single-cell transcriptomics reveals novel strategies for PRCC prevention and treatment
    • Human papillomavirus targets the YAP1-LATS2 feedback loop to drive cervical cancer development
    • Nutritional and Biological Action of Polyphenol-Extracts from Colombian Passiflora Ligularis Juss: In Vivo Study to Evaluate Weight Gain and Inflammation Control
    • Studying early Mycobacterium Tuberculosis infection in situ using lung slice models
    • Dried Fruit Consumption and Cancer
    • Aronia berry supplementation mitigates inflammation in T cell transfer-induced colitis by decreasing oxidative stress
    • Dietary prevention of colitis by aronia berry is mediated through increased Th17 and Treg
    • Cellular uptake and trans-enterocyte transport of phenolics bound to vinegar melanoidins
    • Effect of laboratory-scale decoction on the antioxidative activity of Zhenjiang Aromatic Vinegar: The contribution of melanoidins
    • Interactions of β-conglycinin (7S) with different phenolic acids—impact on structural characteristics and proteolytic degradation of proteins
  • Computational Insights
    • Deriving the Poisson Log-Likelihood and MLEs for Single-Cell RNA-seq
    • How Lagrange Multipliers 'Absorb' Dependencies in Constrained Optimization
    • Understanding DeepSeek's Multi-Head Latent Attention- One Trillion Dollar Math Trick
    • 🎓 Deep Dive into Transformer (GEMORNA) for mRNA Design
    • 🧬 Dynamic RNA velocity model-- (1) math solutions
    • 🧬 Dynamic RNA velocity model-- (2) parameter inference
    • 🧬 Dynamic RNA velocity model-- (3) post hoc velocity graph
    • 🧬 Dynamic RNA velocity model-- (4) latent time
    • 🧬 Dynamic RNA velocity model-- (5) global time normalization
    • 🧬 Dynamic RNA velocity model-- (6) computational handling in implementation
    • 🧬 Dynamic RNA velocity model-- (7) Gillespie Stochastic Simulation Algorithm
    • 🧬 Dynamic RNA velocity model-- (8) effective scVelo analysis
    • 🧬 Math derivation for steady-state RNA velocity model
    • 🧬 Math Derivation of CME-defined Stochastic Model of RNA Velocity
    • 🎓 Math Rationale for ELBO/KL in Bayesian Inference and VAEs
    • 📊 Bypassing Marginals Using Conjugate Priors in Bayesian Inference-- Math Derivation
    • Understanding edgeR: A Deep Dive into Statistical Methods for RNA-seq Differential Expression Analysis
    • 🛠️ Generate and validate ERV gtf that matches the latest reference genome
    • 🧬 A snakemake pipline-- quantify hERV, transgenes and genomics from 10x scRNAseq
    • 🧬 A snakemake pipline-- quantify hERV, transgenes and genomics from well-based single-cell RNAseq
    • 🧬 Dissect sctransform()
    • 🧬 Quantify hERV and transgenes from Bulk-RNAseq
    • 🧮 Math Derivation of a Top-Ranked scRNA-seq Imputation Method (SAVER)
    • ⚡ SeqWins, An R Package for Flexible Base Trimming and Comprehensive FASTQ Analysis on Windows
    • 💨 Modeling the deposition of infectious agents suggests that aerosolized drugs offer enhanced specificity
    • 🧬 EM algorithm, EM with max a posterior, full Bayesian inferences (variational Bayesian) used in auto-coder
    • Understanding Normalization in Deep Learning: A Complete Guide
    • 🧬 Smooth tests of goodness of Fit used in Stijn Hawinkel ’s plos one paper
    • 🧬 how good is negative binomial, why AI works for biology
    • 🧬 Seurat to analysis large sample size of bulk RNAseq. Discuss Wixcox is better for clinical samples
    • 🧬 Deep Neural Network to integrate scRNAseq and spatial genomics
    • 🧬 Spatial genomics presentation
    • 🧬 Bayesian abundance statistics in PRCC
    • 🧬 Train a high-performance Yolov8 (CNN) model for histological analysis: data augmentation and combating overfitting by Active Learning
    • 🧬 CCA reasoning and integration discussion
    • 🧬 Clinical relevance evaluation in PRCC
    • 🧬 Expressive cord diagram in PRCC
    • 🧬 Determine whether it’s noise or true stem like cells in PRCC
    • 🧬 quantify xenograft and host gene expression from the same sample
    • 🧬 counteract treatment-specific background in differential expressed gene analysis (huangcong paper)
    • Deriving Kendall's Tau from The Population Perspective (Copulas)
    • Deriving Kendall's Tau from The Sample-Based Perspective (Discrete)
    • Gaussian Copulas vs Multivariate Normal
    • Is the Copula Independent of Batch Effects in scDesign3?
    • Understanding Clayton Copula Conditional Sampling: From Theory to Practice
    • Understanding Copulas and Joint Densities: From Theory to Practice
    • Understanding Copulas in scDesign3: From Gaussian to Vine Copulas
    • Understanding Copulas: Separating Dependence from Marginal Distributions
    • Understanding Gaussian Copulas and Vine Copulas
    • Understanding Inverse Transform Sampling: From Theory to Practice
    • Why Gaussian Copulas Look Dense at Corners but Have Zero Tail Dependence
    • Neo4j Vector Indexing: A Technical Deep Dive with Concrete Examples
    • Why Likelihood Ratio Statistics Follow Chi-Square Distributions: A Deep Dive
    • GSEA Input Guide: Should You Use All Genes or Only DEGs with p < 0.05?
    • Mathematical Proof: RSS/σ² ~ χ²ₙ₋ₚ
    • Understanding Degrees of Freedom in Linear Regression
    • Understanding the t² = F Identity in Linear Regression: A Complete Mathematical Proof
  • learning
    • KL Divergence, Cross Entropy
    • Skip-gram vs CBOW: Understanding Word2vec Training Examples
    • BWT alignment
    • copula
    • Diffusion Model
    • Dirichlet Distribution
    • docker
    • GNN and CNN
    • Lagrange
    • Lecture 14 - Expectation-Maximization Algorithms | Stanford CS229 Machine Learning
    • linear algebra
    • LoRA
    • Method of characteristics
    • Separation of Variables
    • sql_b-tree_asyn
    • Startup Mechanics
    • U-Net
    • Free Unlimited Custom Domain Email Addresses with Gmail and Cloudflare
    • Learn JavaScript
    • Learn Python
learning
KL Divergence, Cross Entropy

KL Divergence, Cross Entropy

Oct 12, 2025·
Jiyuan (Jay) Liu
Jiyuan (Jay) Liu
· 1 min read

“交叉熵”如何做损失函数？打包理解“信息量”、“比特”、“熵”、“KL散度”、“交叉熵”

Last updated on Oct 14, 2025
NLP Word2vec Machine Learning Deep Learning
Jiyuan (Jay) Liu
Authors
Jiyuan (Jay) Liu
Computational Biologist
Skip-gram vs CBOW: Understanding Word2vec Training Examples Oct 12, 2025 →
© 2026 Me. This work is licensed under CC BY NC ND 4.0