Managing the long tail

My team and I deal with inventory sets in the hundreds of thousands. Such large product catalogs cause logistical pains, but in many cases, the larger issue is getting to profitability when the aggregate cost of that "long tail" of low click, non-revene generating products outweighs the revenue generated by the head of the distribution. Sometimes, this is the case whith feeds that are smaller in size.

Below is an example that shows the order distribution by sku of one of our merchants over a 30 day period. This data is for a single CSE, but the total distribution looks very similar. The x axis is the number of products in the feed, and the y axis is the number of orders each of those products generated. The most important thing to note here is that though the x axis ends at 4000, the total number of offers is actually around 20,000, which means the long tail goes well off your monitor.

Take a close look at the percentages in each of those areas. Over half the cost lives in that long tail of products, the majority of which have incurred just a few clicks. This means the individual product cost is almost invisible, but in aggregate, this poses a serious threat to profitability. The gut reaction for many is pretty simple. Chop off the long tail and leave in the feed only the products that have generated revenuve (ROAS will double!). 

This is where that middle yellow range comes in to the conversation. That range represents products that generated exactly one order over this time frame. If no action is taken, the next 30 day distribution will likely look similar to this, but the products that appear in that middle area will not be the same products next time around. So if you blindly chop off the entire long tail, you're likely cutting off a big portion of your future revenue. You then wind up with a similar distribution that does have a shorter tail, but also a much smaller head.

The ideal solution is to remove only the correct products from that tail. The question is how do you define correct?

There is no way to get it right every time. As soon as you remove a product, you risk losing revenue that could have come on the next click. But most retailers will find there are products that just aren't worth including. Here are a few ideas on how to identify those. Please note that all of these can be loosened or tightened based on your business's tolerance for risk.

• Reverse-engineer your target conversion rate: Look back at the equation in my last post. Drop in the product cost, the CPC you are paying and your target ROAS, then solve for conversion rate. Divide 1 by the result and you have your click bogey. If the product gets that many clicks and no sales, it is officially in a hole. Unfortunately, the above distribution is the result AFTER applying this rule regularly. It also is only addressing the head of the cost tail, not the really long part of the tail. This approach can help cut an unusually large portion of your cost that comes from a relatively small number of products. You could also occasionally widen the time frame on the data set used in this analysis to catch the second tier of products that are not meeting that target conversion rate, but taking longer to reach the click tolerance level. 
• Reverse-engineer a price filter: Try the same exercise as above but solve for AOV. Use your current conversion rate on the CSE in question. The result is the theoretical inflection point of product level profitability. Products under that price are less likely to work in the long, assuming they convert at or below the standard rate used in the equation. If you have a lot of products at a low price point, you may wind up with a much smaller feed, but hopefully a higher ROAS. 
• Use data from outside your CSE campaign: If you have 100,000 products, odds are some subset of those products (possibly larger than you'd care to admit) have never sold on your website. Well, if that long sought after first sale does come some day, it is as likely to come via a CSE as it is to come from any other marketing initiative. But if you are fighting this problem, it may not make sense to include that initial marketing cost here. Since CSE CPCs are pretty much flat, advertising these self ascribed long tail products is a risk. These products are probably either incredibly niche, or there is a problem with the offer itself (probably the price). If they do generate traffic, it will probably only be a few clicks, but that is exactly what we're targeting here. 

written by Mark Vandegrift