Political social listening with open sources (OSINT)
How I built a daily social listening system for Argentine and Brazilian politics: which open sources it collects, how it classifies tone and narratives with Python and language models, and which privacy safeguards it applies.
Matías Gaglio ·
Leer en españolOrganize the public conversation before interpreting it.
Political information is abundant but scattered: articles from hundreds of outlets, video comments, search interest, fact-checks and social networks with very different dynamics. I built a system that collects those open sources every day, normalizes them and turns them into aggregate indicators for about forty politicians in Argentina and Brazil: mention volume, tone, dominant narratives, attention spikes and exposure to reputational risk terms.
The dashboard is a working tool and is not published on this site. What follows explains how it works and what kind of reading it supports.
Only open, free sources.
Each source has its own Python collector, with retries, rate limits and a status log for every run. If one source fails, the rest keep going and the dashboard reports it.
Google News and RSS
Searches per politician and per fact-checker, plus RSS feeds from outlets across the spectrum, each labeled with its editorial line.
YouTube Data API
Public comments on recent political videos, by topic search and by channel.
Bluesky, Mastodon and Reddit
Posts from public APIs that do not require privileged access.
Google Trends, GDELT and Wikipedia
Search interest, international coverage and page views for each politician as attention signals.
From thousands of texts to signals you can explain.
Collect
GitHub Actions runs the pipeline daily and stores texts in a private cache, outside the public repository.
Classify
pysentimiento (RoBERTuito and BERTweet models on PyTorch) labels tone, emotion and aggression in Spanish and Portuguese. If the model is unavailable, auditable fallback lexicons take over.
Group
TF-IDF and NMF factorization in scikit-learn detect narratives: clusters of terms that appear together and how their share changes day by day.
Alert
Volume spikes by z-score, risk terms per politician and a simple 0 to 100 reputational pressure index.
The pressure index combines four components with visible weights: negative tone, aggression, risk terms and volume acceleration. I prefer a simple formula anyone can rebuild over an opaque score nobody can challenge.
Spot near-identical messages without accusing anyone by mistake.
To flag possible coordinated activity, the system compares posts using character n-grams and cosine similarity. Two different accounts posting nearly identical texts (90% similarity or more) within an hour form an edge in a NetworkX graph.
A single match is not enough, because two people can copy the same headline. The same pair of accounts has to match on at least two different messages. In the period analyzed no group met that bar, and the dashboard says so instead of forcing a finding.
What this approach makes readable.
| Detected narrative | Items |
|---|---|
| Milei and Bullrich | 734 |
| Lula and Alckmin | 659 |
| Pope Leo XIV and Villarruel | 608 |
| The Bolsonaro family | 579 |
| Kicillof, Máximo Kirchner and Buenos Aires province | 406 |
- Overall tone: 22.5% negative, 69.4% neutral and 8.1% positive, classified by the model across all 4,216 items.
- Spikes: on November 4 volume was 3.2 standard deviations above the period average.
- Qualitative reading: the model separates topics, but the same phrase can be support, irony or rejection. Findings are checked by reading examples before drawing conclusions.
Publish aggregates, never people.
- Texts stay out of the repository: posts and comments live in a temporary workflow cache; only percentages and counts reach the repository.
- Pseudonymized authors: accounts are replaced with hashed identifiers before analysis.
- Legitimate sources: only public APIs and open data, with no access to private information.
- Traceability: every indicator keeps the link to the public source behind it so it can be verified.