A fellow Hacker News member, Jonathan Pagel, recently made an important observation: Google search engine result pages (SERPs) are now surfacing Hacker News discussions at the top of certain searches, pushing down SEO spam. This is apparently the result of the so-called “hidden gem” update Google announced last year. Here’s how Search Engine Land explains it:
When Google initially announced the hidden gems ranking topic, it mentioned that it would be part of a future helpful content update. Instead, it ended up as part of Google’s core ranking system designed to promote more “authentic” content. This content can come from forum posts, social media posts, blog posts, or other web pages. As long as the content is authentic and shares personal insights and experiences, and if the content is found to be helpful, it might appear. It is not explicitly classifying something as a “gem,” but instead the idea is that this type of content can be perceived as especially helpful and in the past, might have been harder to appear in the results.
Hacker News vs SEO spam
HN discussions are generally very high quality, with some genuine experts freely sharing their experience and insights. I actually learned about Pagel’s blog post on this HN thread. While the answers may not be perfect or “right” all the time, they are generally far better than the SEO garbage that has dominated Google for more than a decade. It’s also a sign that the recent flood of supercharged AI-generated SEO spam is a real problem for search engines, in addition to corrupting Amazon (see AI gives new life to Amazon low content book scams).
It’s not just Hacker News that is showing up prominently. Reddit, Quora, and other discussion forums are now highly ranked. Here’s what happened when I searched for “indexing 3 fund portfolio.” The top search result now points to a wiki article from Bogleheads, another high-quality discussion forum for people who live and breathe Jack Bogle’s indexing philosophy:
At one time, the top results may have an personal finance search term that might have once returned a bunch of content farm garbage, clickbait videos, financial adviser blog posts of varying quality, and corporate landing pages from Fidelity or Vanguard. Indeed, if you scroll down below the Bogleheads wiki explanation, the SEO spam is still there:
Clearly, SEO spam articles, whether generated by AI or humans, is being pushed down in SERPs … but it’s not going away. And the outcome of Google tweaking its algorithm to favor human discussions may have some unfortunate consequences, such as the AI spam generators being turned loose on Hacker News and Bogleheads. As Pagel notes:
Many sites that benefited from the google gem update experienced a huge increase in spam. SEOs can now buy Reddit accounts, comments and upvotes to promote their sites. Google groups that reappeared through the update mostly have articles about free spins now and also Linkedin other social platforms are under attack by SEO spammers. So we can only hope that the new visibility won’t affect the wonderful HN too much.
Regardless, I hope that Google’s flight to quality may extend to the many talented specialty bloggers that never went away, but were overlooked or forgotten by Google’s algorithms. This includes bloggers like Pagel, as well as the Lean Media blog you are reading right now.