Run an Analysis

If you're ready to run an analysis, that's because you already installed mastoscore and set up your analysis with a configuration file.

The easiest way to run analyses is through the Backstage web interface:

./scripts/backstage.sh

Open http://localhost:5173 and use the interface to:

Using Command Line

If you prefer the command line, you can run each phase manually. There's an example.ini file that shows you what the options are. You can also go to the code repo and look at the ini/ directory to see some of the actual INI files I use at various times.

The command line takes a minimum of 2 arguments: the INI file and the action you want to do. It also takes an optional --debug parameter that will display some information. Without the --debug option, it's pretty quiet. With --debug=info you get informative progress messages. With --debug=debug you get a lot of noise.

1. Fetch

First, you go fetch all the toots related to your hashtag. This will take time. For #monsterdon, my laptop connects to around 113 different servers, fetching 40 toots at a time. Thanks to the way federation works, you end up fetching the same toots many times! On a Monsterdon Sunday, there are often around 2000 toots. But the fetch module will frequently fetch 80,000 or more. That's because when someone at mastodon.social toots something related to #monsterdon it is federated to a lot of servers. All 113 servers will send me a copy of that toot. There's a bunch of work in the fetch module to de-duplicate and handle all that. Fetching those 80,000 toots takes my laptop around 26 minutes. On the other hand, some less popular watch parties like #iliketowatch and #kungfusat will complete in a minute or less.

mastoscore --debug=info mastoscore.ini fetch

When this is done, you'll have a bunch of JSON files in your journaldir directory, organized in a hierarchical structure based on the event date (YYYY/MM/DD). See the fetch module for a description of the structure and the files.

2. Analyse

Analyse reads all the JSON files created by the fetch command and analyses them. Mostly this deduplicates the posts and then calculates all the top scores. It creates an analysis.json file.

mastoscore mastoscore.ini analyse

3. Graphs

The two graph types—histogram and wordcloud—can be generated individually or at the same time. Generally speaking, the histogram is right the first time, but the wordcloud needs to be tuned. So sometimes you run one, and sometimes you run both.

graphs

This will generate both the histogram file showing the timeline of toots over time and the wordcloud from the posts. See the graph module and the wordcloud module for more information.

mastoscore mastoscore.ini graphs

Take a look at the wordcloud alt-text file. You'll see text like:

Top 10 most frequent words were:
pumpkinhead: 625, pumpkin: 96, monster: 76, kid: 72, gonna: 62, think: 55, really: 54,
know: 52, magiccrystal: 52, time: 49,

You may find that some of the popular words aren't much fun to include (like yeah or really). So edit your INI file and change the stopwords parameter. Add those words to the list and rerun the wordcloud command to get a new wordcloud. There's no good way to do this other than to make a wordcloud, see what comes out, and then decide what you want to keep and remove. A lot of times I find a #monsterdon event for a film like The Howling (1981) will have a bunch of variations on the hashtag: #Howling, #TheHowling, and #TheHowling1981. I usually exclude all but one of them.

Histogram only

Running the graph command will generate just the histogram. There's a little bit of randomness, since the font is selected randomly from available fonts. But otherwise, it should come out exactly the same.

mastoscore mastoscore.ini graph

Wordcloud

This generates just the wordcloud and its alt text. Note that it always overwrites any existing image. It doesn't save it. And the word cloud is always random. So if you get one you like, be sure to save it before running this a second time.

mastoscore mastoscore.ini wordcloud

4. Post

At this point, you have everything you need to post. The post command will connect to your Mastodon server and post the analysis in a series of linked posts. Note that I developed this on infosec.exchange, which allows 11,000 characters in a toot and direct HTML. It's likely that if you're on a server with the standard limit of 500 characters, this will totally fail. It will try to create a gigantic post and then it will get rejected. It would be useful to support that case.

mastoscore mastoscore.ini post

The post command will post both the analysis text and the graph images (histogram and wordcloud) as a threaded conversation.