The Wayback Machine

Recently Amelia Kassel, a highly respected Information Professional in the USA and member of the Association of Independent Information Professionals, posted this to the AIIP discussion list.  She has given us permission to post it at our website and to circulate it to interested information professionals in Australia.

The Wayback Machine can be an incredibly useful resource in corporate research and includes Australian websites, so this summary may be a useful reminder of its applications.

The National Library of Australia is also archiving Australian websites at http://pandora.nla.gov.au/ and also provides links to other archiving sites around the world.

Amelia’s contact details are at the bottom of this posting.

Elizabeth Swan


I’ve used the Internet Archives’ Wayback Machine a number of times for research questions.  Great news a few days ago:  The Wayback Machine will be updated.  A key word search for homepages, will be included when the update is completed in 2017.  More details here: Grant to Develop the Next Generation Wayback Machine, Posted on October 21, 2015 by Wendy Hanamura
https://blog.archive.org/2015/10/21/grant-to-develop-the-next-generation-wayback-machine/

If anyone on AIIP-L is new to the Wayback Machine, here are a few applications.  I hope others will share your success stories and other comments about your experiences.

  • At one time, I used the Wayback Machine to find names and email addresses of a defunct organization’s members to invite those members to join AIIP.
  • A colleague used the Wayback Machine to find an item she remembered seeing on an Amazon but had disappeared.  She wanted to make a point for an article she was writing and found just what she needed.
  • Business researchers, competitive intelligence professionals, and prospect researchers can use older versions of a website for company research.  Older sites provide insights about a company’s growth and development, marketing strategies and more, and provide names of potential contacts for interviews and fundraising.
  • Legal researchers use old versions of a site to find information for litigation.
  • During the US government shutdown in 2013, many government sites were not updated or maintained and some shut their doors completely.  I was able to assist colleagues find answers to business and legal questions because the Wayback Machine had taken precautions, making sure to index key government sites with the most current version and keeping these sites available when they were shut down – a few details here:  Blacked Out Government Websites Available Through Wayback Machine Posted on October 2, 2013 by Brewster https://blog.archive.org/2013/10/02/governmentblackout/
  • Use the Wayback Machine to view old websites to see how a company’s web pages and level of sophistication has evolved.  School librarians I’ve worked with especially have found it useful to look at old web pages.

wayback-machine-logo

Other Comments and a Tip:

  • Although the Wayback Machine can prove a great resource at times, it doesn’t capture everything because:
  • The automated crawlers are not aware of a site’s existence at the time of the crawl.
  • Sites are password protected
  • Sites are blocked by robots.txt, a special file that prevents crawlers from accessing the site; or, otherwise inaccessible to automated systems used.  A robots.txt query exclusion error message is shown.
  • Site owners can request that their sites be excluded from the Wayback Machine. You’ll see this error message:  “blocked site error”.
  • At one time, the date lag was many months. You won’t always find the most recent information and it could disappear from current websites.
  • Tip:  It’s important to always save anything related to your research immediately.  It may not be available at a later time. In one instance, a client didn’t want to expend funds to obtain many freely available files from a competitor’s website.  The client later decided they wanted the files but when I returned to the site, it was password protected and only available to the website owner’s authorized clients.  The files were not available on the Wayback Machine because they were password protected.

For more about the Wayback machine, see https://archive.org/about/ and http://www.archive.org/about/faqs.php


Amelia Kassel | MarketingBase |  Research is your first step

Business, Market, CI, and Due Diligence Research

amelia@marketingbase.com | 707 829-9421

Research Consulting and Training 

  • Amelia’s Mentor Program – Develop, Grow, and Expand!
  • US Private Company Research Expertise


Contact Details

P.O. Box 364, Rose Bay, 2029

  • Tel. 61 2 9371 0300
  • Fax. 61 2 8569 0121
  • Mobile: 0412 734 403
  • ABN 13 003 256 380

Copyright © 2017 Web Search Pacific Pty Ltd, All Rights Reserved.

Web Search Pacific Pty Ltd is a registered company (ABN 13 003 256 380).