Misplaced Pages

Latest revision as of 08:33, 22 October 2024

• WP:WEBARCHIVES

This is an engineering document for those building bots needing help with URL formats, etc. See Help:Archiving a source for info about using these archive services.

List of known web archive services in-use on English Misplaced Pages. Sorted roughly by number of uses from most to least. The Wayback Machine is about 80% of the total. Data initially compiled by User:GreenC as of March 2017. Updates and corrections welcome.

Archive services

Internet Archive Wayback Machine

• Article: Wayback Machine
• Domain: archive.org, waybackmachine.org
• Launched: 2001
• Date range: 1996-
• Hostname: <none>, web, wayback, liveweb, www, www.web, classic-web, web-beta, replay, replay.web, web.wayback
• Path: <none>, web
• Timestamp: Number 1 digit; 4–14 digits. Or "*". Or "?". Or combination. May also contain trailing chars like "re_" for (?), "if_" for frames and "im_" for images. If timestamp missing returns best available page.
• Examples:

• http://www.web.archive.org/{14_digits}/http..
• http://web.archive.org/web/{14_digits}/http..
• http://wayback.archive.org/http..
• http://web.waybackmachine.org/20081212010700/http..

• Oldest:

• http://web.archive.org/web/0/http..
• http://web.archive.org/web/1/http..

• Newest:

• http://web.archive.org/web/2/http..

• Index:

• http://web.archive.org/web/*/http..

• Submit page:

• https://web.archive.org/save/
• http://web.archive.org/record/

• Interval between manual captures: 1 hour.
• Prefix searching (except after the question mark in query strings)
• Case-insensitive lookup

Archive.Today

• Article: archive.today
• Domain: .today, .is, .fo, .li, .vn, .md, .ph
• Launched: 2012
• Hostname: <none>, www
• Path: <none>
• Timestamp: 4–14 digits; or digits + characters (see example)
• Examples:

• http://archive.today/20130101/http..
• http://archive.today/2013.04.17-12:08:20/http..
• http://archive.today/http.. (index page)

• Submit page: http://archive.today/?run=1&url=... (requires JavaScript)
• Interval between manual captures: 1 week.
• Prefix searching with preview screenshots
• Page history with preview screenshots
• Case-sensitive lookup

Archive.Today represents captured pages as a static snapshot, rendered by the Archive.Today server, and uses a fixed-width layout. Page resources such as JavaScript and CSS files are not retained separately. For example, styling from a separate CSS file is converted to inline CSS styling, embedded in the HTML source code.

Archived pages are initially served through their short URL format, an identifier with five case-sensitive alphanumerical characters and four characters on early captures from 2012. To obtain the long URL format with time stamp and the source URL, click "share" in the top menu or append "/share" to the URL. The full URL is listed in the window.

If a redirect page is saved, Archive.Today stores both the URL of the redirect page and the URL of the redirect target. The archived page can be found by entering either URL.

Additional restrictions

As of 2023, copies of Archive.org pages can only be saved once. This restriction applies as well to the digital library (archive.org/details/) which is subject to change, not only the Wayback Machine where pages are, besides infrequent exclusions, not subject to change anyway.

If a "Welcome to nginx!" page appears, it apparently either means the user has hit a rate limit or the site is doing maintenance work.

Web Citation (WebCite) (deprecated)

• Article: WebCite
• Domain: webcitation.org

Deprecated—no longer accepting new archive requests. Site generally unstable, abandoned, and features not working

• Hostname: <none>, www
• Path: base62ID, query, cache, getfile.php, <number>
• Timestamp: None. Uses &date=2012-06-01+21:40:03 in query?url ; the short ID is base62 which converts to unix time
• Examples:

• http://www.webcitation.org/gT64fd Archived 2020-10-25 at the Wayback Machine
• https://web.archive.org/web/20120806175004/http://www.ariacharts.com.au/pages/charts_display_album.asp?chart=1G50
• http://www.webcitation.org/query?id=1138911916587475 Archived 2019-04-03 at the Wayback Machine
• http://www.webcitation.org/query?url=http..&date=2012-06-01+21:40:03
• http://www.webcitation.org/1138911916587475 Archived 2019-04-03 at the Wayback Machine
• http://www.webcitation.org/cache/73e53dd1f16cf8c5da298418d2a6e452870cf50e Archived 2021-08-19 at the Wayback Machine
• http://www.webcitation.org/getfile.php?fileid=1c46e791d68e89e12d0c2532cc3cf629b8bc8c8e

National Archives UK

• Domain: nationalarchives.gov.uk
• Hostname: webarchive, yourarchives
• Path: <none>
• Timestamp: 4–14 digits
• Examples:

• http://webarchive.nationalarchives.gov.uk/20110311030350/http://www.abilityvability.co.uk/
• http://yourarchives.nationalarchives.gov.uk/index.php?title=Backbone_radio_link_and_radio_standby_to_line_links_for_safeguarding_vital_communications

Australian Web Archive

• Article: Australian Web Archive (Trove)
• Domain: nla.gov.au
• Hostname: webarchive
• Path: see examples. First-level path can contain "awa" or "wayback". Second-level path might contain a pandora.nla.gov.au URL with a third-level destination URL which may or may not have a scheme (http://). Or the second-level path might be the final destination URL. A special archive index page URL is also available as "/tep/"
• Timestamp: Two types (20120727-0512, 20120326012340)
• Examples:

• https://webarchive.nla.gov.au/awa/20120726200849/http://pandora.nla.gov.au/pan/14231/20120727-0512/www.howlspace.com.au/en2/inxs/inxs.htm
• https://webarchive.nla.gov.au/awa/20110824211656/http://pandora.nla.gov.au/pan/128344/20110810-1451/www.theaureview.com/guide/festivals/bam-festival-2010-ivorys-rock-qld.html
• https://webarchive.nla.gov.au/awa/20010328130000/http://www.howlspace.com.au/en2/arenatina/arenatina.htm
• https://webarchive.nla.gov.au/wayback/20120326012340/http://news.defence.gov.au/2011/09/09/army-airborne-insertion-capability/
• https://webarchive.nla.gov.au/gov/20070831165847/http://www.defence.gov.au/opacolyte/default.cfm
• https://webarchive.nla.gov.au/tep/23790 (redirect from http://pandora.nla.gov.au/tep/23790 ) .. index page

Note: The Australian Web Archive incorporates the Pandora archive as well as the Australian Government Web Archive and the National Library of Australia's archive of the .au domain.

Note: No memento access

NLA Australia (deprecated)

Deprecated. Integrated into Australian Web Archive (Trove) above.

• Domain: nla.gov.au
• Hostname: pandora, trove, tep, webarchive, content.webarchive
• Path: see examples. The /pan/ regex should be /pan/{4,7}/
• Timestamp: Three types (20120727-0512, S2000-Dec-5, 20120326012340)
• Examples:

• https://pandora.nla.gov.au/pan/14231/20120727-0512/www.howlspace.com.au/en2/inxs/inxs.htm
• https://pandora.nla.gov.au/pan/128344/20110810-1451/www.theaureview.com/guide/festivals/bam-festival-2010-ivorys-rock-qld.html
• https://pandora.nla.gov.au/nph-wb/20010328130000/http://www.howlspace.com.au/en2/arenatina/arenatina.htm
• https://pandora.nla.gov.au/nph-arch/2000/S2000-Dec-5/http://www.paralympic.org.au/athletes/athleteprofile60da.html
• http://pandora.nla.gov.au/tep/23790
• https://content.webarchive.nla.gov.au/gov/wayback/20120326012340/http://news.defence.gov.au/2011/09/09/army-airborne-insertion-capability::*

• Note: Not to be confused with non-webarchive URLs that appear similar:

• http://pandora.nla.gov.au/pan/23790/20080220-0000/issue935.pdf
• http://nla.gov.au/nla.obj-291093018/view?partId=nla.obj-291093220
• http://trove.nla.gov.au/ndp/del/article/33084123

Note: No memento access

Ghost Archive

• Domain: ghostarchive.org
• Launched: ~2021
• Hostname:
• Path: archive, varchive/<YouTube_video_ID>, iarchive/<Instagram_username>
• Timestamp: 4 to 14 digits
• Examples:

• https://ghostarchive.org/archive/20210922180933/https://gstreamer.freedesktop.org/download/ (short-form)
• https://ghostarchive.org/archive/20210728022510/https://rms-support-letter.github.io/
• https://ghostarchive.org/varchive/UhCiGY75wVw (short form)
• https://ghostarchive.org/varchive/youtube/20100711020000/UhCiGY75wVw
• https://ghostarchive.org/iarchive/instagram/georgemofficial/1374848874216391600
• https://ghostarchive.org/iarchive/s/instagram/BMUccRPg4Ow
• https://ghostarchive.org/archive/20210728022510/https://www.instagram.com/p/BMUccRPg4Ow/

Note: to convert short-form to long:

For regular web pages:

• https://ghostarchive.org/longurl/o9UcZ -> https://ghostarchive.org/archive/20210728022510/https://rms-support-letter.github.io/

For video pages, e.g. YouTube:

• http://ghostarchive.org/vlongurl/UhCiGY75wVw -> https://ghostarchive.org/varchive/youtube/20100711020000/UhCiGY75wVw

To find the earliest and latest archive available use a timestamp of "1990" or "3000" e.g.

• https://web.archive.org/web/20210325005113/https://rms-support-letter.github.io/will find the earliest archived copy of that webpage, while
• will find the latest.

The /archive/ path long form will work for all types of archives, for example https://ghostarchive.org/archive/20210728022510/https://youtube.com/watch?v=UhCiGY75wVw will redirect to the video, and https://ghostarchive.org/archive/20210728022510/https://www.instagram.com/p/BMUccRPg4Ow/ will redirect to the image.

• Prefix searching with preview screenshots
• Page history with preview screenshots

Ghost Archive uses the WARC ("webarchive") format to store saved pages, meaning the verbatim content of the page resources can be recreated. When opened, Ghost Archive uses Webrecorder's ReplayWeb.page software to render the archived page as accurately as possible. Alternatively, the page can be viewed in "noscript", meaning as static HTML in its finished rendered state. This mode does not require JavaScript and is compatible with older browsers and loads faster, however some page features that rely on JavaScript such as pagination and collapsible menus are not available.

Due to Instagram's strict rate limiting, archival of Instagram profiles might fail and result in a blank page. If the archival of a YouTube video failed, an "Archiving error" page is displayed and the archival of the same video can not be retried.

Along with YouTube videos, their metadata is saved: publication date, description, and the URL of the channel in the /@ or /c format, whichever is available.

If the archived page redirects to a different URL, only the target URL is displayed. This means the archived page can not be opened by entering the URL of the redirecting page.

Megalodon.jp

• Site name: web gyotaku
• Date range: 2007-
• Article: Megalodon (website)
• Domain: megalodon.jp
• Examples: https://megalodon.jp/2023-0522-0234-30/https://gstreamer.freedesktop.org:443/download/
• No prefix searching
• Case-sensitive lookup

Similar to Archive.Today, Megalodon.jp represents archived pages as a static HTML snapshot. However, pictures are converted into BASE64 data: URLs inside the resulting HTML data, and there is no fixed width like Archive.Today .

Megalodon lets the user decide whether to save the desktop or mobile version of a page, meaning the version that appears to desktop computer and laptop users, or to smartphone users.

Using https://megalodon.jp/(full URL) (example: https://megalodon.jp/https://gstreamer.freedesktop.org:443/download/ ) can check if Megalodon archived any copy of a particular URL. http and https are treated separately.

If the archived page is a redirect to a different URL, only the URL prior to the redirect is saved. In that case, the archived page can not be opened by entering the target URL (example).

FreezePage

• Domain: freezepage.com
• Hostname: <none>, www
• Path: <none>
• Timestamp: <none> (only available via web scrape)
• Examples:

• http://www.freezepage.com/1338238555ICJBKARMZN
• http://www.freezepage.com/1343081512QUPLJKJOYU?url=http://www.telegraph.co.uk/..

Note: If the account ID which created the snapshot expires for lack of activity (no login to freezepage), the snapshot is deleted from freezepage.com

Note: No memento access

Library of Congress

• Domain: loc.gov
• Hostname: webarchive
• Path: all, lcwa####
• Timestamp: 4–14 digits
• Examples:

• http://webarchive.loc.gov/all/20160110110238/https://www.whitehouse.gov/
• http://webarchive.loc.gov/all/20111109051100/http

Arquivo.pt (Portugal)

• Domain: arquivo.pt
• Hostname: <none>
• Path: wayback, wayback/wayback, noFrame/replay
• Timestamp: 4–14 digits ... might contain "mp_" see example
• Examples:

• http://arquivo.pt/wayback/19980205082901/http://www.caleida.pt/saramago/
• http://arquivo.pt/wayback/wayback/20091010102944/http..
• http://arquivo.pt/noFrame/replay/20091010102944mp_/http..

Stanford University web archive

• Domain: stanford.edu
• Hostname: swap
• Path: was (optional but standard)
• Timestamp: 4–14 digits
• Examples:

• https://swap.stanford.edu/was/19940102000000/http://slacvm.slac.stanford.edu/FIND/slac.html::* https://swap.stanford.edu/19940102000000/http://slacvm.slac.stanford.edu/FIND/slac.html
• Archived 2024-03-03 at the Wayback Machine deprecated

Archive-It

• Domain: archive-it.org
• Hostname: wayback
• Path: "all", a 3–5 digit number; "org-" followed by a 3–4 digit number
• Timestamp: 4–14 digits; "0" or "1" for oldest; "2" for newest; "*" for index
• Examples:

• https://wayback.archive-it.org/all/20190621232545/http://example.com/
• https://wayback.archive-it.org/3348/20151201214156/https://www.heritagepreservation.org/
• https://wayback.archive-it.org/org-467/20191016094633/http://quartos.org/

• Oldest:

• https://wayback.archive-it.org/all/1/http://example.com/
• https://wayback.archive-it.org/3348/1/https://www.heritagepreservation.org/
• https://wayback.archive-it.org/org-467/1/http://quartos.org/

• Newest:

• https://wayback.archive-it.org/all/2/http://example.com/
• https://wayback.archive-it.org/3348/2/https://www.heritagepreservation.org/
• https://wayback.archive-it.org/org-467/2/http://quartos.org/

• Index:

• https://wayback.archive-it.org/all/*/http://example.com/
• https://wayback.archive-it.org/3348/*/https://www.heritagepreservation.org/
• https://wayback.archive-it.org/org-467/*/http://quartos.org/

BibAlex

• Domain: bibalex.org:80
• Hostname: web.archive, web.petabox
• Path: web
• Timestamp: 4–14 digits
• Examples:

• http://web.archive.bibalex.org/web/20051231070651/http://www.heimskringla.no/original/heimskringla/ynglingasaga.php
• http://web.archive.bibalex.org/web/20060521125008/http://developmentgap.org/rmalenvi.html

• Portal entry: https://www.bibalex.org/isis/frontend/archive/archive_web.aspx
• Example URLs above are down as of March 2024. Might be temporary.

WikiWix

• Domain: wikiwix.com
• Hostname: archive
• Path: cache
• Timestamp: 4–14 digits
• Examples:

• https://archive.wikiwix.com/cache/20180329074145/http://www.linterweb.fr
• https://archive.wikiwix.com/cache/?url=http://www.linterweb.fr

Note: Does not support Memento

Note: API access added in March 2018. By appending &apiresponse=1 to the end of the URL. (https://archive.wikiwix.com/cache/?url=http://www.linterweb.fr&apiresponse=1). This may require encoding of any other & in the url= section

Note: Supports &title argument at end of URL not part of the source URL (similar to &apiresponse). Gives the name of the Misplaced Pages article the link is being used in (optional).

National Archives US

• Domain: webharvest.gov
• Hostname: <none>
• Path: <variable>
• Timestamp: 4–14 digits
• Examples:

• https://www.webharvest.gov/peth04/20041022004143/http://www.ftc.gov/os/statutes/textile/alerts/dryclean

National Archives Iceland

• Domain: vefsafn.is
• Hostname: wayback
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• http://wayback.vefsafn.is/wayback/20110318105639/http://www.twitter.com/yagirldwoods

Europa Archives, Ireland (deprecated)

Deceased. In May 2018 all archives were moved to collections.internetmemory.org then as of September 2018, all archives were moved again to Archive-It

• Domain: europarchive.org
• Hostname: collection
• Path: nli
• Timestamp: 4–14 digits
• Move example:

• Original: https://wayback.archive-it.org/10702/20141013204117/http://www.defense.gov/:*Move 1: http://collections.internetmemory.org/nli/20141013204117/http://www.defense.gov/
• Move 2:

Perma CC

• Domain: perma-archives.org, perma.cc
• Hostname: <none>
• Path: <none>, warc
• Timestamp: 4–14 digits for perma-archives.org, or snapshot ID
• Examples:

• http://perma-archives.org/warc/20140729143852/http
• http://perma.cc/F9NT-22AK
• https://perma-archives.org/warc/F9NT-22AK/http://www.goduke.com/ViewArticle.dbml?SPSID=25943&SPID=2027&DB_LANG=C&DB_OEM_ID=4200&ATCLID=152476

Proni Web Archives (deprecated)

Deceased. As of October 2018, all archives moved to Archive-It

• Domain: proni.gov.uk
• Hostname: webarchive
• Path: <none>
• Timestamp: 4–14 digits
• Examples:

• http://webarchive.proni.gov.uk/20111213123846/http

• Move example:

• Original: http://webarchive.proni.gov.uk/20100218151844/http://www.berr.gov.uk/
• New: https://wayback.archive-it.org/11112/20100218151844/http://www.berr.gov.uk/

Parliament UK

• Domain: parliament.uk
• Hostname: webarchive
• Path: <none>
• Timestamp: 4–14 digits
• Examples:

• http://webarchive.parliament.uk/20160204060058tf_/http://www.parliament.uk/about/living-heritage/building/palace/big-ben/

UK Web Archive (British Library)

• Domain: webarchive.org.uk
• Hostname: www
• Path: wayback/archive
• Timestamp: 4–14 digits with possibility of "mp_" at end
• Examples:

• https://www.webarchive.org.uk/wayback/archive/20100602000217/www.westsussex.gov.uk/ccm/navigation/your-council/election
• https://www.webarchive.org.uk/wayback/archive/20151128210021mp_/http://newsroom.herefordshire.gov.uk/2006/november/new%2Dsculpture%2Dto%2Dbe%2Dhanded%2Dover.aspx

Libraries and Archives Canada (deprecated)

Deceased. As of May 2018 all archives moved to webarchive.bac-lac.gc.ca

• Domain: collectionscanada.gc.ca
• Hostname: www
• Path: archivesweb, webarchives
• Timestamp: 4–14 digits
• Examples:

• http://www.collectionscanada.gc.ca/webarchives/20061104084225/http://broadband.gc.ca/maps/province.html?prov=48
• http://www.collectionscanada.gc.ca/archivesweb/20060209004933/http

• Note: Not to be confused with other close URL variants. Only capture "/webarchives/" or "/archivesweb/"
• Move example:

• Original: http://www.collectionscanada.gc.ca/webarchives/20061104084225/http://broadband.gc.ca/maps/province.html?prov=48
• New: http://webarchive.bac-lac.gc.ca:8080/wayback/20061104084225/http://broadband.gc.ca/maps/province.html?prov=48

Libraries and Archives Canada (www.bac-lac.gc.ca)

As of September 2022 the new website is https://library-archives.canada.ca/eng - most of the www.bac-lac.gc.ca may no longer be available but some links still work see:

• Domain: bac-lac.gc.ca:8080
• Hostname: webarchive, www
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• http://webarchive.bac-lac.gc.ca:8080/wayback/20051228174058/http://nationalatlas.gov/

• Note: Formerly collectionscanada.gc.ca see above. As of September 2022, links from collectioncanada.gc.ca were entirely removed by LAC.

Catalonian Archive

• Domain: padi.cat(:8080)?
• Hostname: www, (none)
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• http://www.padi.cat:8080/wayback/20140404212712/http
• http://www.padi.cat/wayback/20140404212712/http

Web Archives Singapore

• Domain: nlb.gov.sg
• Hostname: eresources
• Path: webarchives/wayback
• Timestamp: 4–14 digits
• Examples:

• https://eresources.nlb.gov.sg/webarchives/2016-04-25%2019:07:06.000/wp/details/http://www.lta.gov.sg/apps/news/page.aspx?c=2&id=2dzk9l67sx9j40a1rhgdw3hvhrnxgq3zh34l77r37dj4w72jf1
• https://eresources.nlb.gov.sg/webarchives/wayback/20160425174854/https://www.lta.gov.sg/apps/news/page.aspx?c=2&id=2dzk9l67sx9j40a1rhgdw3hvhrnxgq3zh34l77r37dj4w72jf1

• Note: Not to be confused with other close URL variants. Only capture "/webarchives/wayback/"

Slovenian Archives (Spletni)

• Domain: nuk.uni-lj.si:8080
• Hostname: nukrobi2 (may change)
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• http://nukrobi2.nuk.uni-lj.si:8080/wayback/20160203130917/http

Estonia Archives

• Domain: digar.ee
• Hostname: veebiarhiiv
• Path: a
• Timestamp: 4–14 digits
• Examples:

• http://veebiarhiiv.digar.ee/a/20131014091520/http://rakvere.kovtp.ee/en_GB/twin-cities

Bavarian Archives

• Domain: bib-bvb.de
• Hostname: langzeitarchivierung
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• http://langzeitarchivierung.bib-bvb.de/wayback/20121004142737/http://www.schwabenkrieg.historicum-archiv.net/

York University Digital Library

• Domain: yorku.ca
• Hostname: digital.library
• Path: wayback
• Timestamp: 4–14 digits
• Examples:

• https://digital.library.yorku.ca/wayback/20160129214328/http://en.cijnews.com/?p%3D10033

National Library of Israel

• Domain: wayback.nli.org.il
• Path: <none>
• Timestamp: 14 digits
• Format: http://wayback.nli.org.il/{14_digits}/{URL}

Appears to have poor coverage.

Other

Memento Web

• https://web.archive.org/web/20081211202744/http://www.muslimdirectory.co.uk/displayresults.php?PHPSESSID=f0fb8b41d8758983e7d43cddb556b9df&businesstype=1&orgtype=&country=UK&city=Cardiff

Note: Redirects to an external archive service based on cached data in the Memento database which can fluctuate and/or be inaccurate due to the cache going out of sync with the client service.

Google Cache (ephemeral)

• https://web.archive.org/web/20160817214930/http://www.gapan.org/ruth-documents/Masters%20Medal%20%20Press%20Release.pdf

Note: Links quickly expire.

Note: Cannot be accessed with Memento.

Bing Cache (ephemeral)

Only accessible through search results, not manually through an URL or search prefix.

Undocumented

• https://cachedview.nl/
• https://cachedview.com/
• http://www.cachedpages.com/
• https://commoncrawl.org/
• https://www.bravenewtech.org/ "Video Vault"
• https://conifer.rhizome.org/
• https://archive.st/

See also

• List of Web archiving initiatives
• Help:Archiving a source

References

1. "Wayback Archives". Tracetools. Retrieved 27 June 2022.
2. ^ Bederov, Igor S. (8 May 2022). "Web Archive as an OSINT Tool". Medium. Retrieved 27 June 2022.
3. "Examples using Common Crawl Data". Common Crawl. Retrieved 27 June 2022.
4. "RightsLab". rightslab.org. Retrieved 27 June 2022.
5. "rightslab". Twitter. Retrieved 27 June 2022.
6. "Webrecorder". webrecorder.net. Retrieved 27 June 2022.
7. "wiki.Archiveteam". archiveteam.org. Retrieved 27 June 2022.

• Misplaced Pages archives