Why is S&P 500 hard to beat?

Because it purchased every stock when it becomes large (for only 0.01% position) and held on to it forever. Very few people purchased Nvidia in 2001 despite dotcom bubble and never bought or sold any shares. The so-called "rebalance" doesn't really buy any existing shares unless certain mid-cap stocks become large-cap, and it doesn't sell any existing shares unless there is a M&A, or stock being taken private, or simply the stock market cap shrank and therefore being replaced by upcoming stocks. The bottom line is that S&P 500 never bought stocks at large positions and never sold stocks voluntarily, well, of course unless a new sector of stocks are added, such as the Great 1976 Reshuffle of adding financial stocks.