Automate a website change monitor with a weekly digest

A page you rely on can change at any time: terms, a pricing table, an API doc, a competitor's landing page. Re-reading from memory misses small edits, and tools that diff raw HTML flag every rotating banner and timestamp. What you want is the wording that changed, in plain text, once a week. Readable text means the words a visitor sees: scripts, styles, navigation and cookie banners are dropped, and spacing is normalised so a layout reflow is not a change. If you name a part of the page, such as the pricing table, that section alone is compared.

You list the page addresses and, optionally, the part of each page to watch. Each week the scheduled agent fetches every page. Where a server supports conditional requests with an ETag, an unchanged page is skipped quickly; otherwise the agent compares the readable text with the previous run's copy. If a site publishes a sitemap, a lastmod date there is a hint about which pages to look at, not proof of a change. The ignore field takes one phrase per line, such as Last updated or a visitor counter, and those lines are removed from both copies before comparing. A moved page is reported with its new address and compared there, so it is not mistaken for a deleted one. It reads public pages.

Before the digest reaches you, the department head agent checks it: is each quoted change really present in both versions, is a layout shuffle reported as a content change, and is a failed fetch labelled as a failure rather than no change. Then you approve, edit or skip it, and nothing is delivered until you do.

Act on the changes that matter to a decision: update your own copy, answer a rival's new offer, or re-check a terms change with your adviser. Tighten the ignore list when the same noise appears twice, and add the next page that cost you a surprise.