Tools of the Trade – When Are Software Tools Mentioned in Computational Text Analysis Research?

The use of computational methods for text analysis has been rapidly gaining a foothold in computational social science. Yet, little is known about the reporting practises of the software tools utilised in this field. This research note investigates the factors influencing the likelihood of software tools being mentioned in social science journal articles. To this end, we reviewed 406 journal articles, representing all computational text analysis articles published in top social science journals between 2016 and 2020, and identified all software tools reported. We explore both article-level and tool-level characteristics and investigate their association with the mentioning of software tools. Our findings indicate a consistent pattern of tool reporting across time and disciplines, while authors who detail methodological validation within their papers are more inclined to mention the software tools employed. Furthermore, we observe that software tools with greater accessibility tend to receive more frequent mentions, while the maintenance status of the tools does not significantly impact their frequency of being mentioned. We discuss the implications of these results for the further development of computational text analysis in social science and especially communication research.