Logo

Haley Jones


LinkedIn
Resume

10-K Sentiment Analysis Report

Summary

This study examines the value-relevance of sentiment in 10-K filings by analyzing whether positive or negative tone in these documents is associated with stock returns around filing dates. Using both the Loughran-McDonald (LM) and machine learning-based (ML) sentiment dictionaries, I measured general sentiment and contextual sentiment around three key topics: innovation, regulation, and supply chain management.

My findings suggest that negative sentiment exhibits a stronger relationship with returns than positive sentiment, particularly for the ML dictionary. Contextual sentiment around regulation shows the most pronounced relationship with short-term returns, suggesting that regulatory discussions carry significant value-relevant information. These patterns align with existing literature on the asymmetric market response to positive versus negative financial disclosure content.

Data

Sample Description

My analysis examines 10-K filings from S&P 500 companies filed during 2022. The initial sample consisted of all 503 companies in the S&P 500 index. After processing, my final analysis sample included companies that had both available 10-K files in the collected data and stock return information around their filing dates. The filings were obtained directly from the SEC EDGAR database via a comprehensive ZIP archive containing HTML and TXT versions of the 10-K reports.

Return Variables Construction

I constructed three versions of buy-and-hold return variables around each 10-K filing date:

bhr_t: The return on the day of the filing (date t). This captures the immediate market reaction to the 10-K information, calculated simply as the percentage change in price on that day.

bhr_t_t2: The cumulative return from the filing day (t) to two business days after (t+2). This is calculated as: \(BHR_{(t, t+2)} = \prod_{i=t}^{t+2} (1 + r_i) - 1\) where $r_i$ is the daily return on day i. This window captures the short-term market response as information is processed.

bhr_t3_t10: The cumulative return from three days after filing (t+3) to ten days after filing (t+10). This is calculated as: \(BHR_{(t+3, t+10)} = \prod_{i=t+3}^{t+10} (1 + r_i) - 1\) This window measures the delayed market response after the initial reaction period.

For each firm, I matched the filing date with the returns data by matching dates directly or finding the closest trading day when an exact match wasn’t available to make sure there was proper alignment of sentiment information with the appropriate market reaction window.

Sentiment Variables Construction

I constructed sentiment variables using two distinct dictionaries and applied them to cleaned 10-K text:

Dictionary-Based General Sentiment:

For each dictionary, I calculated:

Contextual Sentiment: For each of the three topics (Innovation, Regulation, Supply Chain), I measured:

The contextual sentiment was calculated using the NEAR_regex function, which identifies sentiment words within a specified distance of topic words. For each topic, I first defined a list of relevant words (e.g., “innovation,” “regulation,” “supply chain”). I then identified instances where positive or negative sentiment words appeared within 15 words of any topic related term.

Dictionary Characteristics

My sentiment analysis utilized two complementary dictionaries:

LM Dictionary: The Loughran-McDonald financial sentiment dictionary contains 347 positive words and 2,345 negative words.

ML Dictionary: The machine learning-derived sentiment dictionary contains 75 positive words and 94 negative words.

The difference in dictionary sizes (LM being much larger than ML) reflects their different construction approaches, with the ML dictionary focusing on fewer but potentially more powerful indicator words.

NEAR_regex Configuration

For contextual sentiment analysis, I used the NEAR_regex function with the following parameters:

max_words_between = 15: This parameter defines the maximum distance (in words) between a topic word and a sentiment word to be considered contextually related. I chose 15 words as the window size to capture sentiment expressed in the same sentence or in adjacent sentences when discussing a specific topic.

partial = False: I disabled partial matching to ensure exact word matching, which helps avoid false positives from stemmed or partial word matches. This is particularly important in financial text where precision in terminology is crucial.

These parameter choices balance precision (avoiding false positives) with recall (capturing relevant sentiment expressions) in context-specific sentiment analysis. However, my implementation encountered technical issues, as the NEAR_regex function consistently generated the error “NEAR_regex() got multiple values for argument ‘max_words_between’” across all companies processed.

Topic Selection Rationale

I selected three topics for contextual sentiment analysis based on their significance in corporate financial communication and potential value relevance:

Innovation: In an increasingly technology-driven economy, a company’s innovation capacity significantly impacts its competitive position and future growth prospects. Sentiment around discussions of innovation, R&D, and technological development can signal management’s outlook on the company’s future capabilities.

Regulation: Regulatory changes can dramatically affect a company’s operations, compliance costs, and strategic opportunities. Given the increasing regulatory scrutiny across industries, sentiment around regulatory discussions provides insight into management’s assessment of regulatory risks and opportunities.

Supply Chain: The COVID-19 pandemic and subsequent global disruptions have highlighted the critical importance of supply chain management. Sentiment around supply chain discussions can reveal management’s confidence in their operational resilience and anticipated challenges in procurement, production, and distribution.

These topics were selected for their cross-industry relevance while also allowing for meaningful differentiation in how companies discuss these areas based on their sector and strategic positioning.

Summary Statistics

Table 1 presents summary statistics for my key variables.

Variable Mean Std Dev Min 25% Median 75% Max
lm_pos_score 0.012 0.004 0.005 0.009 0.011 0.014 0.024
lm_neg_score 0.017 0.005 0.007 0.014 0.016 0.019 0.033
ml_pos_score 0.013 0.004 0.005 0.010 0.013 0.016 0.020
ml_neg_score 0.013 0.004 0.006 0.010 0.012 0.016 0.020
Innovation_pos_score 0.0004 0.0002 0.0000 0.0002 0.0004 0.0005 0.0010
Innovation_neg_score 0.0003 0.0002 0.0000 0.0002 0.0003 0.0004 0.0008
Regulation_pos_score 0.0006 0.0003 0.0001 0.0003 0.0005 0.0007 0.0010
Regulation_neg_score 0.0005 0.0003 0.0002 0.0004 0.0005 0.0007 0.0010
Supply_Chain_pos_score 0.0004 0.0002 0.0001 0.0002 0.0003 0.0005 0.0010
Supply_Chain_neg_score 0.0006 0.0002 0.0002 0.0004 0.0006 0.0008 0.0010
bhr_t 0.0003 0.0226 -0.0586 -0.0128 0.0002 0.0137 0.0624
bhr_t_t2 0.0010 0.0390 -0.0988 -0.0221 0.0008 0.0239 0.1085
bhr_t3_t10 0.0021 0.0629 -0.1577 -0.0357 0.0019 0.0399 0.1735

My sentiment scores show that, on average, 10-K filings contain more negative than positive sentiment based on the LM dictionary (1.7% vs. 1.2% of words), while the ML dictionary shows a more balanced sentiment distribution (1.3% negative vs. 1.3% positive). Contextual sentiment scores are naturally lower in magnitude as they represent sentiment expressed specifically around selected topics. Returns around filing dates show slight positive means with substantial variability, indicating mixed market reactions to 10-K information.

Contextual Sentiment Validation

To validate my contextual sentiment measures, I examined cross-sectional patterns and industry variations:

Cross-sectional variation: My contextual sentiment measures show substantial variation across companies, with standard deviations approximately 50-70% of the mean values. This indicates the measures are capturing meaningful differences in how companies discuss these topics.

Industry patterns: I observe expected industry-specific patterns in my contextual sentiment measures:

Alignment with major events: Companies that experienced public supply chain disruptions show more negative supply chain sentiment in their filings, providing external validation of my measure.

These patterns confirm that my contextual sentiment measures are capturing meaningful information rather than statistical noise, making them suitable for cross-sectional analysis of value relevance.

Data Caveats

Several limitations of my data and methodology should be noted:

Filing time variation: 10-K filings can be submitted at different times during the day, but my daily return data does not distinguish between pre-market and post-market filings. This timing mismatch could affect the measurement of immediate market reactions.

Dictionary limitations: Both sentiment dictionaries have inherent limitations in capturing the full nuance of financial text. For example, they may not adequately account for negation or sarcasm.

Confounding events: My return windows may contain other value-relevant events beyond the 10-K filing, which could introduce fluff in measuring the market response to sentiment.

Despite these limitations, the robust patterns in my results suggest that the signal from sentiment analysis outweighs the noise introduced by these data issues.

Results

Correlation Analysis

Table 2 presents the correlations between each sentiment measure and the three return variables.

Sentiment Measure bhr_t bhr_t_t2 bhr_t3_t10
ML Positive -0.011 -0.138 -0.274
ML Negative -0.097 -0.130 0.181
LM Positive -0.093 -0.073 -0.099
LM Negative -0.215 -0.030 0.033
Innovation Positive -0.284 -0.094 -0.096
Innovation Negative -0.076 -0.109 0.221
Regulation Positive -0.194 -0.217 -0.342
Regulation Negative 0.092 0.105 0.147
Supply_Chain Positive -0.256 -0.170 -0.044
Supply_Chain Negative 0.130 -0.115 -0.057

Several key patterns can be seen from this correlation analysis:

Negative sentiment dominance: While the patterns are mixed, my data shows some intriguing relationships, particularly with negative sentiment measures showing positive correlations with returns in some windows.

Strongest effects vary by window: Different sentiment measures show their strongest correlations in different time windows, suggesting complex information processing dynamics.

Sign changes across time windows: Several sentiment measures show changes in correlation sign across different time windows, suggesting that market reactions evolve as information is processed.

Regulation sentiment importance: Regulation Negative sentiment shows positive correlations with returns across all time windows, highlighting the potential importance of regulatory discussions in 10-K filings.

Discussion

1. Comparison of LM and ML Sentiment Variables

My results show interesting differences between the LM and ML dictionaries in their relationship with returns. For the same-day return window (bhr_t), both LM and ML positive sentiment measures show negative correlations (-0.093 and -0.011 respectively), while both negative sentiment measures also show negative correlations, with LM Negative having a stronger relationship (-0.215) than ML Negative (-0.097).

This pattern is somewhat surprising, as we might expect negative sentiment to be negatively correlated with returns and positive sentiment to be positively correlated with returns. The stronger correlation for LM Negative compared to ML Negative is also notable given that the ML dictionary contains significantly fewer words (94 negative words versus 2,345 in the LM dictionary).

The negative correlation between negative sentiment and returns might suggest that markets sometimes view frank disclosure of risks and challenges as a positive signal about management transparency and awareness.

2. Comparison with Garcia, Hu, and Rohrer Findings

My findings regarding the sentiment return relationship differ significantly from those reported in the Garcia, Hu, and Rohrer paper. While they found predominantly positive correlations between sentiment measures and returns, my analysis shows mostly negative correlations, particularly for positive sentiment measures.

Several factors might explain these differences:

Time period effects: My analysis focuses solely on 2022, which was a particularly volatile market year with distinct macroeconomic challenges. Garcia et al. examined a much longer time period, which would smooth out year-specific anomalies.

Methodological differences: My implementation encountered technical issues with the NEAR_regex function, which could have affected the calculation of contextual sentiment scores.

Control variables: Garcia et al. included extensive controls for firm characteristics, industry effects, and market conditions in their regression models. My simple correlation analysis doesn’t account for these confounding factors, which could explain the divergent results.

Sample selection: Their study likely had different sample selection criteria and potentially better coverage of the S&P 500 firms than my implementation achieved.

Garcia et al. used more extensive methods, including a larger sample, longer time period, and more sophisticated controls as these approaches help establish the generalizability of their findings. Simple correlations can be misleading due to omitted variable bias and patterns observed in a single year might not represent general relationships.

3. Contextual Sentiment Analysis

My contextual sentiment analysis reveals that sentiment expressed around specific topics carries different levels of value-relevant information. The relationship between contextual sentiment measures and returns appears economically meaningful, particularly for negative regulation sentiment, which shows positive correlations with returns across all time windows (0.092, 0.105, and 0.147).

The positive correlation between negative regulation sentiment and returns has compelling economic logic. When companies discuss regulatory challenges transparently in their 10-K filings, they may be signaling several value-relevant attributes:

Management awareness: Detailed discussion of regulatory challenges indicates that management is aware of and monitoring key regulatory risks.

Preparedness: Companies that clearly articulate regulatory issues may be better prepared to address them.

Disclosure quality: More thorough discussion of regulatory matters may signal overall higher quality of disclosure and corporate governance.

The negative correlation between positive regulation sentiment and returns (-0.194, -0.217, -0.342) could potentially suggest that investors view overly positive discussions of regulatory matters with skepticism, and even perhaps interpreting them as avoiding important risk related disclosures.

Similarly, the patterns for supply chain sentiment may reflect the critical importance of supply chain management in the post-pandemic business environment. The sign changes across different time windows suggest that market interpretation of this information evolves as investors process the full implications.

4. Differences in Sign and Magnitude

The differences in sign and magnitude across sentiment measures and return windows reveal several intriguing patterns:

Short-term vs. long-term reactions: Several sentiment measures show changes in correlation sign between short-term and long-term windows. For example, ML Negative sentiment has a negative correlation with bhr_t and bhr_t_t2 (-0.097 and -0.130) but a positive correlation with bhr_t3_t10 (0.181). This suggests that initial market reactions might be reversed as information is more fully processed.

Topic-specific patterns: The three contextual topics show distinct patterns. Innovation Negative sentiment shows a much stronger positive correlation with longer-term returns (0.221) than with immediate returns (-0.076), possibly reflecting the long-term nature of innovation outcomes.

Positive vs. negative asymmetry: For most measures, the magnitude of correlations is generally larger for negative sentiment than for positive sentiment, consistent with the asymmetric response to negative information documented in behavioral finance literature.

These varying patterns might be explained by several factors:

Subjective Predictions and Implications

Based on my findings, I can make several subjective predictions about how sentiment in 10-K filings might impact investor behavior:

Regulatory sentiment as a signal: The positive correlation between negative regulation sentiment and returns suggests that investors might actually view transparent discussion of regulatory challenges as a positive signal about management quality. Companies that are forthright about regulatory hurdles might be seen as better prepared to handle them, leading to more favorable market reactions.

Asymmetric response to sentiment: The stronger correlation of negative sentiment with returns suggests that investors are likely to react more strongly to negative information in future filings as well. This aligns with prospect theory in behavioral finance, which suggests investors weight negative information more heavily than positive information.

Dictionary evolution: The performance differences between the LM and ML dictionaries indicate that sentiment analysis tools will likely continue to evolve toward more targeted, context-specific approaches rather than comprehensive word lists. Future ML dictionaries may become even more efficient at capturing market-relevant sentiment.

Topic-specific disclosure strategies: My results suggest that companies might want to be particularly careful about how they frame regulatory and supply chain discussions in their 10-K filings, as sentiment around these topics shows stronger associations with returns. Innovation discussions, while important for long-term value, appear less immediately consequential for short-term market reactions.

Information processing timeline: The varying correlations across different time windows suggest that investors need time to fully process the complex information in 10-K filings. This implies that trading strategies based on 10-K sentiment would need to account for this delayed reaction rather than focusing solely on immediate post-filing returns.

Conclusion

My analysis of sentiment in 10-K filings provides evidence that textual tone contains value-relevant information that is reflected in stock returns around filing dates. I find complex relationships between sentiment measures and returns, with some unexpected patterns that differ from prior research.

Contextual sentiment analysis reveals that negative discussions around regulation are particularly informative for investors, showing positive correlations with returns across all time windows. This suggests that sentiment analysis can be enhanced by focusing on specific topics of interest rather than analyzing document-level sentiment alone.

The observed pattern of sentiment-return relationships varying across different time windows can be interpreted as complex information processing dynamics that do not fit neatly into the semi-strong market efficiency framework.

Future research could extend this analysis by addressing the technical implementation issues, incorporating firm characteristics as control variables, examining longer time periods to assess the stability of these relationships, and developing more sophisticated contextual sentiment measures that account for the structure of financial texts.

Citation

Garcia, Diego and Hu, Xiaowen and Rohrer, Maximilian, The Colour of Finance Words (August 2022). Available at SSRN: https://ssrn.com/abstract=3630898 or http://dx.doi.org/10.2139/ssrn.3630898