How the search engine manages RSS feeds to update its index.

Originally the plan was to put a simple RSS feed in the site inspection view, though it turns out feed data is very useful for a search engine in discovering new links. The feed view is still a nice addition and a good way of ensuring that the feed discovery and retrieval is actually working well.