Semrush is a data-driven SaaS platform which helps its users with search engine optimization, pay-per-click, content, social media and other marketing activities. Our array of tools is based on data, a lot of data. Five years ago, our marketing experts realized that not only does knowing the most popular search keywords help in planning ad campaigns for our clients and users, it also reveals information about peoples’ interests, and sometimes even their hidden desires.
After all, what do you do when you need information in 2023? You search the web. People may lie in their surveys, they may even lie to their friends, but they are unlikely to be anything but truthful in a Google search.
We decided to try offering this data to journalists and using it for external studies to increase brand awareness. And here’s what we’ve got.
Supporting Data Journalism: The First Steps
To start promoting the brand in this way, we needed to understand how exactly we could help journalists. We also needed to complete this quickly, so that it would not distract from our other marketing tasks.
By using the data in our product, we can track various socio-economic trends. For example, statistics on search queries, which usually help digital marketers adjust their advertising campaigns, can also show you which brand is the most-searched around the world. And if we look at the traffic drawn by various sites, we can determine which retailer can expect the biggest demand on Black Friday. This information is hard to find in the open, but it often reflects reality better than, say, targeted surveys.
We monitor all major new sources daily in order to find topics where our data could be useful. Way back in 2018, there was strong growth in Bitcoin’s value — it seemed like everybody and their dog wrote about it. We did the simplest thing we could think of: we calculated the correlation between the price of Bitcoin and the number of related search queries available from Semrush. As a result, our information was published at Business Insider.
Then we looked at what happens after pieces like this are published. It turned out that people were starting to search for our brand on the internet — they were becoming interested in our tool and even buying it. Odd as it may seem, the journalist who published an article about Bitcoin based on our data, boosted interest in our brand — our brand searches grew by 20% that month.
How Do We Work With the Data?
In the beginning, it was not yet clear how we could most effectively work with the data we had. We needed to gather information manually, directly from our tool’s interface. For example, just to determine which fashion retailer was the most popular in a particular year, we had to enter the website of every retailer into our Traffic Analytics, copy the result into a table, repeat this action for every site on a monthly basis, and then average and sort these numbers. That could take up half the working day. So, Semrush decided to seek out experts who could perform these tasks faster and more effectively, while bringing some new analytics ideas.
That is how the Research Agency Team was created, allowing us to get our new data-based approach flowing. As the team forged on, we did quite a bit to automate and structure the way we work with our data. For example, knowledge of programming languages is hugely beneficial for this — similarly to most data analysts and data scientists, we use Python.
We first wrote our own internal library that would help us solve everyday tasks and generate standard reports, all in just a few lines of code. It connected everything in one place, to provide a single view for uploading data from different sources, ranging from tables and SQL queries to official and internal APIs.
When we first began working with journalists, we found that often they are seeking not just numbers but also attractive visualizations to place in their articles. Initially, we drew graphs using basic visualization libraries (matplotlib, seaborn), but frankly the visual results were not too impressive and it took us a long time.
That is why we wrote our own separate tool in Python based on Plotly and an HTML editor to provide stylish visualizations of our data. Together with the Marketing Design Team, we developed branded templates that met all the requirements of our Brand book, and then we translated those templates into code. Now generating a graph takes literally just a minute.
Data Studies for Product Promo and Marketing Campaigns
We also realized that on top of our data for general media, we wanted to spend time on narrower, more detailed research. The intended audience for this research would be our potential users. Instead of merely claiming that our tool offers a lot of useful data and we know how to work with it, we could prove this with case studies. In this way, we would attract new users to our product, and at the same time we also share some valuable observations and insights from our area of knowledge.
One of the first studies was dedicated to Voice Search SEO. Colleagues suggested that we analyze the search queries that users were making with voice search. At that time, we did not have that data internally at Semrush, so we decided to collect it from scratch.
We got some voice assistants and wrote a script that would make our computers ask these voice assistants questions (altogether fifty thousand of them!) and then record the results. Over several months our electronic friends talked with each other — taking up a meeting room at our office — and then, with the help of Semrush tools, we collected the remaining information on our search queries.
Long story short, we obtained a huge dataset and some interesting research, which was presented at one of the major SEO conferences — SEO Search Marketing Expo, and at other events.
Every year Black Friday represents a very “hot” topic. For us it is particularly interesting because it draws the attention of both general and business-oriented media. We gather information on the most popular products and brands that people search for on Google and Amazon in conjunction with Black Friday, we track the traffic to ecommerce sites during this hectic period, and we analyze the ad spend of online shops. We make an effort to not only gather interesting statistics but also analyze trends, and to give our readers insights that they can use in, for example, advertising and promoting their products.
At year’s end we traditionally release a study of what resonated in the world over the preceding twelve months. A simple example: we look at the fastest-growing search queries, we analyze what topics have been covered by major news sites, we draw up a list of the most popular stars, politicians, music albums, and much more. These studies of ours have proven very successful, and in large part due to the engaging infographics created by our Design Team, they have attracted a lot of attention and mentions, especially on social media. Our last study on results for 2021 saw 640 publications and media mentions around the world.
Now we put out more than fifteen niche studies each year in various areas: SEO, Local SEO, Content Marketing, Social Media Marketing, we review the latest developments in the field (for example, Google Search Engine updates), and we work together with partners to expand the range of data that we analyze (e.g. data from app stores and YouTube).
To avoid getting bogged down in routine tasks, we have automated our process to the max. Now we have our own internal web service that downloads data from the required sources in a format that is simple, clear and does not require any specific technical knowledge. Consequently, the task regarding fashion retailers that we described above, now takes no more than one minute to generate the report.
It has become very fast and easy to get data for various activities; our colleagues use it for pitching to journalists as well as for carrying out internal research.
How to Plan a Study That Brings Real Value
Working with data in our field is not like analytics in other sectors. Marketing and public relations are very dynamic areas: often we do not have much time to come up with a lot of hypotheses and test them.Teamwork, manipulation of data, ideas, and brainstorming is what lies behind each of our products. For us, research is a product on its own, so the way we perform research has much in common with how we develop products.
When we embark on any study, we have to find answers to several questions:
- Who are we performing the study for?
- What question are we looking to answer in the study — what value will we bring?
- How are we going to do it?
Thus, it is the same as what we do for products.
Let’s discuss the first point. Our research has two kinds of users: the customer’s team and the ultimate reader. We have to take the interest of both into account as we work on a project. For example, the marketing team has to have a good understanding of how to read and present the data that we have. Readers, on the other hand, must find some value in our research, and this value must be clearly shown.
To correctly cover these points, you need to ask the team right away where the study will be published. If we are targeting mass media, then the information within should be simple and cover a broad topic. If, on the other hand, the study will appear on a professional platform, then you can delve deeper, examine finer details, and insert a lot of tables and charts.
If we are talking about a customer that is also a user, then to make it easier for colleagues to work with the results, we present the final report not just as a set of tables. Instead, we try to highlight what is most important and remove what is most obvious — here, charts and brief tables help to highlight key numbers and describe main conclusions.
The second step is to select a hypothesis or an idea to study. For this, we can study the market: research social media discussions, similar studies, or the popular search terms on the topic. It is important to know whether we have the information needed to test the hypothesis, and if we do not, whether we can collect it ourselves. Of course, we also need to know what the purpose of the project is: is it a descriptive study, or are we looking to attract attention to new features in our product? Do we need to impress the reader with something new, or rather present something already familiar in an engaging way?
At this stage we look at similar studies, so that we can get an idea of what was done well and what was lacking. We check how popular the topic is on social media, and we can look at the number of searches on the topic.
It is also important to ensure that the information can be obtained, and moreover, obtained in the way we need. For this we look at the data carefully, and not only rely on the metrics overview. We check to see if the data adds up from the perspective of ordinary thinking.
After that preparation, we can confidently go on to formulate the final concept and goal of our study.
We still need to answer the question of how we will tackle our tasks. We need a clear methodology so that we know what our Research Team will face.
Here we chart a methodology step by step. Sometimes we prepare a separate document, other times it can be a task in our task tracker. Once we know what our subtasks are, we assign them within the team, or even outside it (for example, if certain input is required from the customer or if we need the help of another team inside the company).
For maintaining oversight, it is enough to observe the task status on our task tracker. However, if that is not enough and the project is more complex, it can be useful to set up regular short calls or meetings with the customer.
Then we hand over our result. Depending on the complexity of the project, this can be an ample report with text and conclusions, or it may be just a single table.
At the final stage we answer questions, make corrections. Here it is also very important to collect feedback. Often this feedback is clear from the customer’s questions, for example, some conclusions cannot be seen from the visualizations, or the information is not presented clearly enough. If we take these things into account, we can save everyone a great deal of time. On the basis of the experience that we have accumulated from all our tasks, we have come up with a list of risks and guidelines that we try to keep in mind when starting each new project.
Over the course of four years, our focus has changed and flexibly adjusted to the needs of the company. However, we have found that working with data is useful in a very wide variety of areas, and we believe that the tasks described in this article are only the beginning.
Now our studies play a role in Marketing departments’ campaigns, and the research is regularly published on our blog and also by specialized media sources. In addition, the team creates and supports internal tools for working with product data, as well as monitors metrics related to marketing activities’ performance.
And by the way, searches on topics relating to data analytics worldwide in 2022 have risen by 45% over 2020. Isn’t this an indication that now is the best time to get involved in this field, if you haven’t done so yet?