Comparing full cumulative distribution functions avoids reliance on parametric normality assumptions.

Download image · 1200 × 630 PNG

Research steps

Compare conditional message distributions

  1. Extract category weights; split at conditioning-variable median
  2. Build cumulative distributions for both subsets
  3. Calculate Kolmogorov–Smirnov statistics and directional p-values

Illustrative summary of the source-described procedure. The source tests first-order stochastic dominance between conditional message distributions. No direction of dominance or statistical result is asserted here.

See source →

How

When to use

Researchers need to evaluate whether one online communication category consistently dominates another across a dataset of user messages. Standard mean comparisons fail to capture distributional shifts and rank orderings across full probability distributions. Non-parametric stochastic dominance testing compares cumulative distribution functions to determine whether communication patterns differ systematically across user groups or performance levels.

Test for first-order stochastic dominance between conditional message distributions using the non-parametric Kolmogorov-Smirnov statistical framework.

  1. Extract natural language processing probability weights for target message categories across all platform messages.
  2. Partition messages or entrepreneurs into high and low subsets based on median threshold values of a conditioning variable.
  3. Construct empirical cumulative distribution functions for the target category across both conditioned sub-samples.
  4. Calculate maximum algebraic distance statistics between the two empirical cumulative distribution functions.
  5. Evaluate dual null hypotheses of directional dominance using p-values to accept or reject first-order stochastic dominance.

Why it works

The non-parametric test determines whether one sentiment or business communication profile statistically dominates another across all quantiles.

Example

Data scientists evaluated over 140,000 messages sent by 4,958 entrepreneurs across 49 African countries. Testing business content conditional on sentiment yielded p-values of 0.0, 1.0, 0.0, proving that messages with low sentiment strongly dominate high-sentiment messages in business content in the first-order stochastic dominance sense.

About the study

Participants

140,000+ messages sent by 4,958 entrepreneurs across 49 African countries; 1,016 active M0 completers.

Study Design

Non-parametric Kolmogorov-Smirnov stochastic dominance testing comparing empirical cumulative distribution functions across NLP message categories and performance metrics.

Results

Messages with low sentiment dominate high-sentiment messages in business content with p-values of 0.0, 1.0, 0.0. High-performing entrepreneurs write messages with significantly higher business content (p-values 0.0, 0.0, 0.29 for extensive quality) and private audience targeting (p-values 0.0, 0.0, 0.96 for extensive quality).

Source

See source →