11 Jun, 2013 von Michael Müller 2
Simple cURL based activity monitor for websites
I wrote a simple activity monitor to notify me about changes on certain websites. The script is executed as a cronjob on a dedicated server and regularly fetches certain URIs. It then executes diff on the last file fetched from this URI. If there is a difference, it is mailed to me. To describe some use cases:
- Product in Online-Shop is out of stock, I want to be notified when it is available again.
- University lecture. Want to be notified when news/exercises/etc are put on the lecture website.
- Usenet Search. Want to be notified when certain files are available and can be found via a Usenet search site.
- Event registration. I monitored a Barcamp website to get notified, once the registration was available.
- Monitor changes on certain Wiki pages.
- Regularly check if something can be ordered now.
- Monitor changes to a certain Etherpad instance.
- Monitor changes on databases. Some projects offer a HTTP API (e.g. OpenStreetMap). Regularly exporting the database file and running diffs against it shows changes.
- Monitor newly created mailinglists via the lists interface offered by our university.
- Monitor the delivery status of packages, by fetching the transport providers tracking page.
Generally the idea is simply to monitor content changes on HTTP resources and thus the script can easily be used for anything that supports HTTP. A simple monitor is set-up like this:
./check-for-changes.sh "alice@mail.de" "http://some-resource.net/"
If you want to monitor only a certain part of a website you can use a preprocess file to filter out content you don’t want to monitor:
./check-for-changes.sh "alice@mail.de" "http://some-resource.net/" "./preprocess/only-body.sh"
I have released the project under a MIT license via GitHub.
