L1 LASSO regularization forces coefficients of non-essential terms to zero, creating sparse models.

Download image · 1200 × 630 PNG

Research steps

Select text indicators across repeated models

  1. Keep words and two-word phrases occurring at least 10 times
  2. Fit 200 to 1,000 LASSO models on resamples with replacement
  3. Keep indicators with non-zero weights in over 50% of fits

Illustrative summary of the source-described procedure. The source uses penalised logistic regression for semantic categories. Selection frequency identifies stable text indicators, not causal effects.

See source →

How

When to use

Organizations need to identify key words and phrases that signal specific message categories in high-dimensional text datasets. High-dimensional text contains thousands of sparse vocabulary terms that cause standard regression models to overfit. Running regularized regression across repeated random sub-samples isolates the most stable predictors.

Identify informative text indicators by running regularized logistic regressions over repeated random sub-samples of text data.

  1. Extract words and two-word phrases that occur at least 10 times across all platform messages.
  2. Select target binary or ordinal semantic categories as dependent variables in a logistic regression model.
  3. Draw repeated random sub-samples of message data with replacement between 200 and 1000 times.
  4. Fit an l1-penalized LASSO logistic regression model on each random sub-sample using varying regularization weights.
  5. Aggregate the estimated coefficient vectors across all fitted models.
  6. Select regressors that receive non-zero coefficient weights in over 50% of the fitted models as informative indicators.

Why it works

Bootstrapping across random sub-samples ensures that selected words remain consistent predictors regardless of sample variation.

Example

Data scientists analyzed over 140,000 messages written by 4,958 African entrepreneurs to isolate key terms for business focus, sentiment, and audience scope. Running 200 to 1,000 bootstrapped LASSO models identified product, project, business, market, and cost as stable high-business indicators with selection scores above 0.5.

About the study

Participants

140,000+ messages written by 4,958 entrepreneurs across 49 African countries.

Study Design

Bootstrapped fitting of 200 to 1,000 l1-penalized LASSO logistic regressions on random sub-samples drawn with replacement, selecting regressors with non-zero weights in >50% of iterations.

Results

The procedure selected 10 primary words for high business content (product, project, business, market, idea, need, customer, service, plan, cost) and distinct indicator sets for low business content, neutral sentiment, encouragement, and audience scope.

Source

See source →