Link rot (or linkrot) is the process by which links on a website gradually become irrelevant or broken as time goes on, because websites that they link to disappear, change their content, or move to new locations.
The phrase also describes the effects of failing to update web pages so that they become out-of-date, containing information that is old and useless, and that clutters up search engine results.
Prevalence:
The 404 "not found" response is familiar to even the occasional Web user. A number of studies have examined the prevalence of link rot on the Web, in academic literature, and in digital libraries. In a 2003 experiment, Fetterly et al. (2003) discovered that about one link out of every 200 disappeared each week from the internet. McCown et al. (2005) discovered that half of the URLs cited in D-Lib Magazine articles were no longer accessible 10 years after publication, and other studies have shown link rot in academic literature to be even worse (Spinellis, 2003, Lawrence et al., 2001). Nelson and Allen (2002) examined link rot in digital libraries and found that about 3% of the objects were no longer accessible after one year.
Some news sites contribute to the link rot problem by keeping only recent news articles online where they are freely accessible at their original URLs, then removing them or moving them to a paid subscription area. This causes a heavy loss of supporting links in sites discussing newsworthy events and using news sites as references.
Discovering:
Detecting link rot for a given URL is difficult using automated methods. If a URL is accessed and returns back an HTTP 200 (OK) response, it may be considered accessible, but the contents of the page may have changed and may no longer be relevant. Some web servers also return a soft 404, a page returned with a 200 (OK) response (instead of a 404) that indicates the URL is no longer accessible. Bar-Yossef et al. (2004) developed a heuristic for automatically discovering soft 404s.
Combating:
There are several tools that have been developed to help combat link rot.
* WordPress guards against link rot by replacing non-canonical URLs with their canonical versions.[1]
* When a user receives a 404 response, the Google Toolbar for Internet Explorer attempts to assist the user in finding the missing page.[2]
* IBM's Peridot attempts to automatically fix broken links.
Web archiving
To combat link rot, web archivists are actively engaged in collecting the Web or particular portions of the Web and ensuring the collection is preserved in an archive, such as an archive site, for future researchers, historians, and the public. The largest web archiving organization is the Internet Archive Wayback Machine, which strives to maintain an archive of the entire Web, taking periodic snapshots of pages that can then be accessed for free and without registration many years later simply by typing in the URL. National libraries, national archives and various consortia of organizations are also involved in archiving culturally important Web content.
Individuals may also use a number of tools that allow them to archive web resources that may go missing in the future:
* WebCite, a tool specifically for scholarly authors, journal editors and publishers to permanently archive "on-demand" and retrieve cited Internet references (Eysenbach and Trudel, 2005).
* Archive-It, a subscription service that allows institutions to build, manage and search their own web archive
* Some social bookmarking websites, such as Furl, make private copies of web pages bookmarked by their users.
Webmasters
Webmasters have developed a number of best practices for combating link rot:
* Avoiding unmanaged hyperlink collections
* Avoiding links to pages deep in a website ("deep linking")
* Using hyperlink checking software or a Content Management System (CMS) that automatically checks links
* Using permalinks
* Using redirection mechanisms (e.g. "301: Moved Permanently") to automatically refer browsers and crawlers to the new location of a URL
Authors citing URLs
A number of studies have shown how widespread link rot is in academic literature (see below). Authors of scholarly publications have also developed best-practices for combating link rot in their work:
* Avoiding URL citations that point to resources on a researcher's personal home page (McCown et al., 2005)
* Using Persistent Uniform Resource Locators (PURLs) and digital object identifiers (DOIs) whenever possible
* Using web archiving services (e.g. WebCite) to permanently archive and retrieve cited Internet references (Eysenbach and Trudel, 2005).
Tuesday, August 11, 2009
Broken Link OR Dead link (404 Error)
A dead link (also called a broken link or dangling link) is a link on the World Wide Web that points to a web page or server that is permanently unavailable. The most common result of a dead link is a 404 error, which indicates that the web server responded, but the specific page could not be found. The browser may also return a DNS error indicating that a web server could not be found at that domain name. A link might also be dead because of some form of blocking such as content filters or firewalls.
Another type of dead link is a URL that points to a site unrelated to the content sought. This can sometimes occur when a domain name is allowed to lapse, and is subsequently reregistered by another party. Domain names acquired in this manner are attractive to those who wish to take advantage of the stream of unsuspecting surfers that will inflate hit counters and PageRanking.
Link rot is the process by which links on a website gradually become irrelevant or broken over time as sites they link to disappear, change content, or redirect to new locations.
Links specially crafted to not resolve, as a type of meme, are known as Zangelding, which roughly translated from German means tangle thing. A zangelding is basically a list of self referencing broken links.
Dead links commonplace on the Internet can also occur on the authoring side, when website content is assembled, copied, or deployed without properly verifying the targets, or simply not kept up to date. Because broken links are to some very annoying, generally disruptive to the user experience, and can live on for many years, sites containing them are regarded as unprofessional.
Solutions to broken links:
Due to the unprofessional image that dead links bring to both sites linking and linked to, there are multiple solutions that are available to tackle them - some working to prevent them in the first place, and others trying to resolve then when they have occurred.
* The most obvious form of link management, is employing link checking software that test each link on a website for its validity. An example of, and one of the most widely used link checkers is Xenu's Link Sleuth.
* Content Managment Systems often offer inbuilt solutions to the management of links, eg. links are updated when content is changed or moved on the site.
* Permalinking stops broken links by guaranteeing that the content will never move. Another form of permalinking is linking to a permalink that then redirects to the actual content, ensuring that even though the real content may be moved etc..., links pointing to the resources stay intact.
* The Wayback Machine, operated by the Internet Archive, keeps historical snapshots of websites. If a dead link is found, a search for the webpage on the Wayback Machine may yield a past version, which may be used to replace the dead link with an archived version.
When a broken link remains however, as is very common on the internet (especially between websites due to the difficulty of updating other websites linking to one's own), the visitor that clicks the dead link will get an HTTP 404 error, indicating the resource could not be found. The following tools try to rectify this error:
* The Linkgraph widget get the URL of the correct page based upon the old broken URL by using historical location information.
* The Google 404 Widget employs Google technology to 'guess' the correct URL, and also provides the user a Google search box to find the correct page.
* DeadURL.com gathers and ranks alternate urls for a broken link using Google Cache, the Internet Archive, and user submissions. Typing deadurl.com/ left of a broken link in the browser's address bar and pressing enter loads a ranked list of alternate urls, or (depending on user preference) immediately forwards to the best one.
Another type of dead link is a URL that points to a site unrelated to the content sought. This can sometimes occur when a domain name is allowed to lapse, and is subsequently reregistered by another party. Domain names acquired in this manner are attractive to those who wish to take advantage of the stream of unsuspecting surfers that will inflate hit counters and PageRanking.
Link rot is the process by which links on a website gradually become irrelevant or broken over time as sites they link to disappear, change content, or redirect to new locations.
Links specially crafted to not resolve, as a type of meme, are known as Zangelding, which roughly translated from German means tangle thing. A zangelding is basically a list of self referencing broken links.
Dead links commonplace on the Internet can also occur on the authoring side, when website content is assembled, copied, or deployed without properly verifying the targets, or simply not kept up to date. Because broken links are to some very annoying, generally disruptive to the user experience, and can live on for many years, sites containing them are regarded as unprofessional.
Solutions to broken links:
Due to the unprofessional image that dead links bring to both sites linking and linked to, there are multiple solutions that are available to tackle them - some working to prevent them in the first place, and others trying to resolve then when they have occurred.
* The most obvious form of link management, is employing link checking software that test each link on a website for its validity. An example of, and one of the most widely used link checkers is Xenu's Link Sleuth.
* Content Managment Systems often offer inbuilt solutions to the management of links, eg. links are updated when content is changed or moved on the site.
* Permalinking stops broken links by guaranteeing that the content will never move. Another form of permalinking is linking to a permalink that then redirects to the actual content, ensuring that even though the real content may be moved etc..., links pointing to the resources stay intact.
* The Wayback Machine, operated by the Internet Archive, keeps historical snapshots of websites. If a dead link is found, a search for the webpage on the Wayback Machine may yield a past version, which may be used to replace the dead link with an archived version.
When a broken link remains however, as is very common on the internet (especially between websites due to the difficulty of updating other websites linking to one's own), the visitor that clicks the dead link will get an HTTP 404 error, indicating the resource could not be found. The following tools try to rectify this error:
* The Linkgraph widget get the URL of the correct page based upon the old broken URL by using historical location information.
* The Google 404 Widget employs Google technology to 'guess' the correct URL, and also provides the user a Google search box to find the correct page.
* DeadURL.com gathers and ranks alternate urls for a broken link using Google Cache, the Internet Archive, and user submissions. Typing deadurl.com/ left of a broken link in the browser's address bar and pressing enter loads a ranked list of alternate urls, or (depending on user preference) immediately forwards to the best one.
Subscribe to:
Posts (Atom)
