Last week we removed the ‘Ask Twitter’ feature from WebMynd’s interface on search results pages. The idea of the feature was that, when you search, as well as being shown results from your favourite sources on the right-hand side of the results page, you could also ask your Twitter followers for help with your search.

It seemed to be an exciting idea when we first thought of it. By putting the ability to post to Twitter right into your search workflow, we thought to improve Twitter’s utility for you and make your search more social. Early beta testing by the WebMynd team seemed to validate this – I found a great fish and chip shop in Covent Garden, London by asking my Twitter network after I had searched unsuccessfully. It would never have occurred to me to ask had the interface not been right there on the search page.

We removed this feature last week and I’ll cover the reasons. It also got me thinking in more general terms about the circumstances under which you would consider removing a feature, since this isn’t the first time we’ve done it: in early versions of WebMynd last summer had a feature allowing offline access of your web history, and a feature which let you publish parts of your web history for the world to see as you browsed.

The data looked bad

We looked at how many unique users were posting to Twitter from our interface. For a start, this number was at most 10% of the total number of new signups that day. And while we did not break down our retention stats down to that level of granularity, the numbers were low enough that we could simply observe by the individual posts that very rarely did any user post more than once from the interface. The low proportion of new users trying it could be explained by the feature as it stood only being applicable to the overlap between Twitter users and WebMynd users. But the terrible retention was more worrying.

The fact that the data looked bad after a couple of months of tracking it was a great big warning light. When thinking about the causes of the poor data, I asked myself the following questions:

Did the feature satisfy the intended use case?

While it allowed users to post a question to Twitter about their search query we had no built-in way to collate the answers. We assumed that the user, if already also a Twitter user, would have their own way to keep alerted on @replies or posts from people they followed, by for example, using TwitterFox. So we didn’t want to replicate those features in WebMynd and we had no automatic way to correlate replies to the question with the question itself.

Also, we observed that many of the users who did try out the feature appeared to be brand new Twitter users by the fact that their post from WebMynd was their first post. These users did not have many, if any followers, and probably had not yet figured out what set of habits and applications allowed them to keep track of replies.

So it seemed quite likely that the feature as it stood was not a complete enough solution.

Did we promote the benefits sufficiently?

We thought that putting the ‘Ask Twitter’ box onto our interface on Google and other major search engines would make it sufficiently high profile for a lot of our users to try out – our users saw that interface many times per day on average. But the data showed this wasn’t the case. We did not make a big splash on our website about the feature since we did not want to distract from our main use case with an experiment. We hoped that people would click on the ‘Ask Twitter’ link to open up the posting interface and so discover the feature and its usefulness.

It became clear to me that in order to really to a good test of the feature and use case, we would have to work much harder on promoting the benefits as well as making our implementation much more complete.

Could a different feature satisfy the use case better?

But that changed when we launched the WebMynd ‘Dock’ earlier this month. This allowed users to share links and post to Twitter as well as several other tools (Facebook, Delicious, Digg, Reddit, Hacker News) from a sidebar in the browser.

Within days of launching that feature, we had thousands of people actively using it, on average, 4 times per day. And while we had launched it for the purpose of sharing links as you browsed, it seemed so easy to also be able to make general posts to Twitter, including asking questions right from that interface.

For me that made the decision – we had another feature which satisfied the use case and more, where the data was showing great up-take by users. There seemed no sense in investing the effort to make the ‘Ask Twitter’ feature more complete and to promote that when the alternative feature was taking off so well.

Did the intended use case exist?

Ultimately it is possible that the ‘Ask Twitter’ feature was not well used simply because the use case simply didn’t exist. Maybe people don’t find the ability to easily ask their Twitter network questions related to their search useful. I don’t think the experiment we ran was sufficient to conclude that but it obviously was not working in the form we had originally tried.

What now?

It’s been about a week since we removed the feature, we have yet to receive a comment or complaint, and there have been no repercussions in terms of the usage of other features. So I think it is fair to conclude that it was the right choice. We should of course ask the 5 whys in our next product roadmap iteration to see what we can learn from the feature and how we might improve our decision making process.

We now have two interfaces into Twitter – the search widget that WebMynd puts on the right-hand side of Google and other search engines when the user selects it. And the Dock which allows users to post to Twitter and share links easily. I think those two features are just asking to be combined in new and interesting ways, and I very much doubt that the ‘Ask Twitter’ feature will our last experiment in this area.

In general, I don’t think we should be afraid of removing features, because if we cannot do that then we will become more reluctant to try out new ideas in the future and run the risk of them not working out. WebMynd did not start out embedding search results on the right-hand side of search engines. But when, almost in a whim, we just stuck some web history results up there to see what would happen, we discovered the most popular feature in our product. It changed our direction as a company.