Link rot (or linkrot) is the process by which links on a website gradually become irrelevant or broken as time goes on, because websites that they link to disappear, change their content, or move to new locations.
The phrase also describes the effects of failing to update web pages so that they become out-of-date, containing information that is old and useless, and that clutters up search engine results.
Prevalence:
The 404 "not found" response is familiar to even the occasional Web user. A number of studies have examined the prevalence of link rot on the Web, in academic literature, and in digital libraries. In a 2003 experiment, Fetterly et al. (2003) discovered that about one link out of every 200 disappeared each week from the internet. McCown et al. (2005) discovered that half of the URLs cited in D-Lib Magazine articles were no longer accessible 10 years after publication, and other studies have shown link rot in academic literature to be even worse (Spinellis, 2003, Lawrence et al., 2001). Nelson and Allen (2002) examined link rot in digital libraries and found that about 3% of the objects were no longer accessible after one year.
Some news sites contribute to the link rot problem by keeping only recent news articles online where they are freely accessible at their original URLs, then removing them or moving them to a paid subscription area. This causes a heavy loss of supporting links in sites discussing newsworthy events and using news sites as references.
Discovering:
Detecting link rot for a given URL is difficult using automated methods. If a URL is accessed and returns back an HTTP 200 (OK) response, it may be considered accessible, but the contents of the page may have changed and may no longer be relevant. Some web servers also return a soft 404, a page returned with a 200 (OK) response (instead of a 404) that indicates the URL is no longer accessible. Bar-Yossef et al. (2004) developed a heuristic for automatically discovering soft 404s.
Combating:
There are several tools that have been developed to help combat link rot.
* WordPress guards against link rot by replacing non-canonical URLs with their canonical versions.[1]
* When a user receives a 404 response, the Google Toolbar for Internet Explorer attempts to assist the user in finding the missing page.[2]
* IBM's Peridot attempts to automatically fix broken links.
Web archiving
To combat link rot, web archivists are actively engaged in collecting the Web or particular portions of the Web and ensuring the collection is preserved in an archive, such as an archive site, for future researchers, historians, and the public. The largest web archiving organization is the Internet Archive Wayback Machine, which strives to maintain an archive of the entire Web, taking periodic snapshots of pages that can then be accessed for free and without registration many years later simply by typing in the URL. National libraries, national archives and various consortia of organizations are also involved in archiving culturally important Web content.
Individuals may also use a number of tools that allow them to archive web resources that may go missing in the future:
* WebCite, a tool specifically for scholarly authors, journal editors and publishers to permanently archive "on-demand" and retrieve cited Internet references (Eysenbach and Trudel, 2005).
* Archive-It, a subscription service that allows institutions to build, manage and search their own web archive
* Some social bookmarking websites, such as Furl, make private copies of web pages bookmarked by their users.
Webmasters
Webmasters have developed a number of best practices for combating link rot:
* Avoiding unmanaged hyperlink collections
* Avoiding links to pages deep in a website ("deep linking")
* Using hyperlink checking software or a Content Management System (CMS) that automatically checks links
* Using permalinks
* Using redirection mechanisms (e.g. "301: Moved Permanently") to automatically refer browsers and crawlers to the new location of a URL
Authors citing URLs
A number of studies have shown how widespread link rot is in academic literature (see below). Authors of scholarly publications have also developed best-practices for combating link rot in their work:
* Avoiding URL citations that point to resources on a researcher's personal home page (McCown et al., 2005)
* Using Persistent Uniform Resource Locators (PURLs) and digital object identifiers (DOIs) whenever possible
* Using web archiving services (e.g. WebCite) to permanently archive and retrieve cited Internet references (Eysenbach and Trudel, 2005).
Tuesday, August 11, 2009
Broken Link OR Dead link (404 Error)
A dead link (also called a broken link or dangling link) is a link on the World Wide Web that points to a web page or server that is permanently unavailable. The most common result of a dead link is a 404 error, which indicates that the web server responded, but the specific page could not be found. The browser may also return a DNS error indicating that a web server could not be found at that domain name. A link might also be dead because of some form of blocking such as content filters or firewalls.
Another type of dead link is a URL that points to a site unrelated to the content sought. This can sometimes occur when a domain name is allowed to lapse, and is subsequently reregistered by another party. Domain names acquired in this manner are attractive to those who wish to take advantage of the stream of unsuspecting surfers that will inflate hit counters and PageRanking.
Link rot is the process by which links on a website gradually become irrelevant or broken over time as sites they link to disappear, change content, or redirect to new locations.
Links specially crafted to not resolve, as a type of meme, are known as Zangelding, which roughly translated from German means tangle thing. A zangelding is basically a list of self referencing broken links.
Dead links commonplace on the Internet can also occur on the authoring side, when website content is assembled, copied, or deployed without properly verifying the targets, or simply not kept up to date. Because broken links are to some very annoying, generally disruptive to the user experience, and can live on for many years, sites containing them are regarded as unprofessional.
Solutions to broken links:
Due to the unprofessional image that dead links bring to both sites linking and linked to, there are multiple solutions that are available to tackle them - some working to prevent them in the first place, and others trying to resolve then when they have occurred.
* The most obvious form of link management, is employing link checking software that test each link on a website for its validity. An example of, and one of the most widely used link checkers is Xenu's Link Sleuth.
* Content Managment Systems often offer inbuilt solutions to the management of links, eg. links are updated when content is changed or moved on the site.
* Permalinking stops broken links by guaranteeing that the content will never move. Another form of permalinking is linking to a permalink that then redirects to the actual content, ensuring that even though the real content may be moved etc..., links pointing to the resources stay intact.
* The Wayback Machine, operated by the Internet Archive, keeps historical snapshots of websites. If a dead link is found, a search for the webpage on the Wayback Machine may yield a past version, which may be used to replace the dead link with an archived version.
When a broken link remains however, as is very common on the internet (especially between websites due to the difficulty of updating other websites linking to one's own), the visitor that clicks the dead link will get an HTTP 404 error, indicating the resource could not be found. The following tools try to rectify this error:
* The Linkgraph widget get the URL of the correct page based upon the old broken URL by using historical location information.
* The Google 404 Widget employs Google technology to 'guess' the correct URL, and also provides the user a Google search box to find the correct page.
* DeadURL.com gathers and ranks alternate urls for a broken link using Google Cache, the Internet Archive, and user submissions. Typing deadurl.com/ left of a broken link in the browser's address bar and pressing enter loads a ranked list of alternate urls, or (depending on user preference) immediately forwards to the best one.
Another type of dead link is a URL that points to a site unrelated to the content sought. This can sometimes occur when a domain name is allowed to lapse, and is subsequently reregistered by another party. Domain names acquired in this manner are attractive to those who wish to take advantage of the stream of unsuspecting surfers that will inflate hit counters and PageRanking.
Link rot is the process by which links on a website gradually become irrelevant or broken over time as sites they link to disappear, change content, or redirect to new locations.
Links specially crafted to not resolve, as a type of meme, are known as Zangelding, which roughly translated from German means tangle thing. A zangelding is basically a list of self referencing broken links.
Dead links commonplace on the Internet can also occur on the authoring side, when website content is assembled, copied, or deployed without properly verifying the targets, or simply not kept up to date. Because broken links are to some very annoying, generally disruptive to the user experience, and can live on for many years, sites containing them are regarded as unprofessional.
Solutions to broken links:
Due to the unprofessional image that dead links bring to both sites linking and linked to, there are multiple solutions that are available to tackle them - some working to prevent them in the first place, and others trying to resolve then when they have occurred.
* The most obvious form of link management, is employing link checking software that test each link on a website for its validity. An example of, and one of the most widely used link checkers is Xenu's Link Sleuth.
* Content Managment Systems often offer inbuilt solutions to the management of links, eg. links are updated when content is changed or moved on the site.
* Permalinking stops broken links by guaranteeing that the content will never move. Another form of permalinking is linking to a permalink that then redirects to the actual content, ensuring that even though the real content may be moved etc..., links pointing to the resources stay intact.
* The Wayback Machine, operated by the Internet Archive, keeps historical snapshots of websites. If a dead link is found, a search for the webpage on the Wayback Machine may yield a past version, which may be used to replace the dead link with an archived version.
When a broken link remains however, as is very common on the internet (especially between websites due to the difficulty of updating other websites linking to one's own), the visitor that clicks the dead link will get an HTTP 404 error, indicating the resource could not be found. The following tools try to rectify this error:
* The Linkgraph widget get the URL of the correct page based upon the old broken URL by using historical location information.
* The Google 404 Widget employs Google technology to 'guess' the correct URL, and also provides the user a Google search box to find the correct page.
* DeadURL.com gathers and ranks alternate urls for a broken link using Google Cache, the Internet Archive, and user submissions. Typing deadurl.com/ left of a broken link in the browser's address bar and pressing enter loads a ranked list of alternate urls, or (depending on user preference) immediately forwards to the best one.
Tuesday, July 21, 2009
Client-server
Client-server computing or networking is a distributed application architecture that partitions tasks or work loads between service providers (servers) and service requesters, called clients. Often clients and servers operate over a computer network on separate hardware. A server machine is a high-performance host that is running one or more server programs which share its resources with clients. A client does not share any of its resources, but requests a server's content or service function. Clients therefore initiate communication sessions with servers which await (listen to) incoming requests.
Description :
Client-server describes the relationship between two computer programs in which one program, the client program, makes a service request to another, the server program. Standard networked functions such as email exchange, web access and database access, are based on the client-server model. For example, a web browser is a client program at the user computer that may access information at any web server in the world. To check your bank account from your computer, a web browser client program in your computer forwards your request to a web server program at the bank. That program may in turn forward the request to its own database client program that sends a request to a database server at another bank computer to retrieve your account balance. The balance is returned to the bank database client, which in turn serves it back to the web browser client in your personal computer, which displays the information for you.
The client-server model has become one of the central ideas of network computing. Many business applications being written today use the client-server model. So do the Internet's main application protocols, such as HTTP, SMTP, Telnet, DNS. In marketing, the term has been used to distinguish distributed computing by smaller dispersed computers from the "monolithic" centralized computing of mainframe computers. But this distinction has largely disappeared as mainframes and their applications have also turned to the client-server model and become part of network computing.
Each instance of the client software can send data requests to one or more connected servers. In turn, the servers can accept these requests, process them, and return the requested information to the client. Although this concept can be applied for a variety of reasons to many different kinds of applications, the architecture remains fundamentally the same.
The most basic type of client-server architecture employs only two types of hosts: clients and servers. This type of architecture is sometimes referred to as two-tier. It allows devices to share files and resources. The two tier architecture means that the client acts as one tier and application in combination with server acts as another tier.
The interaction between client and server is often described using sequence diagrams. Sequence diagrams are standardized in the Unified Modeling Language.
Specific types of clients include web browsers, email clients, and online chat clients.
Specific types of servers include web servers, ftp servers, application servers, database servers, name servers, mail servers, file servers, print servers, and terminal servers. Most web services are also types of servers.
Comparison to peer-to-peer architecture :
In peer-to-peer[disambiguation needed] architectures, each host or instance of the program can simultaneously act as both a client and a server, and each has equivalent responsibilities and status.
Both client-server and peer-to-peer architectures are in wide usage today. Details may be found in Comparison of Centralized (Client-Server) and Decentralized (Peer-to-Peer) Networking.
Comparison to client-queue-client architecture :
While classic client-server architecture requires one of the communication endpoints to act as a server, which is much harder to implement,[citation needed] Client-Queue-Client allows all endpoints to be simple clients, while the server consists of some external software, which also acts as passive queue (one software instance passes its query to another instance to queue, e.g. database, and then this other instance pulls it from database, makes a response, passes it to database etc.). This architecture allows greatly simplified software implementation. Peer-to-peer architecture was originally based on the Client-Queue-Client concept.
Advantages :
* In most cases, a client-server architecture enables the roles and responsibilities of a computing system to be distributed among several independent computers that are known to each other only through a network. This creates an additional advantage to this architecture: greater ease of maintenance. For example, it is possible to replace, repair, upgrade, or even relocate a server while its clients remain both unaware and unaffected by that change.
* All the data is stored on the servers, which generally have far greater security controls than most clients. Servers can better control access and resources, to guarantee that only those clients with the appropriate permissions may access and change data.
* Since data storage is centralized, updates to that data are far easier to administer than what would be possible under a P2P paradigm. Under a P2P architecture, data updates may need to be distributed and applied to each peer in the network, which is both time-consuming and error-prone, as there can be thousands or even millions of peers.
* Many mature client-server technologies are already available which were designed to ensure security, friendliness of the user interface, and ease of use.
* It functions with multiple different clients of different capabilities.
Disadvantages :
* Traffic congestion on the network has been an issue since the inception of the client-server paradigm. As the number of simultaneous client requests to a given server increases, the server can become overloaded. Contrast that to a P2P network, where its aggregated bandwidth actually increases as nodes are added, since the P2P network's overall bandwidth can be roughly computed as the sum of the bandwidths of every node in that network.
* The client-server paradigm lacks the robustness of a good P2P network. Under client-server, should a critical server fail, clients’ requests cannot be fulfilled. In P2P networks, resources are usually distributed among many nodes. Even if one or more nodes depart and abandon a downloading file, for example, the remaining nodes should still have the data needed to complete the download.
Description :
Client-server describes the relationship between two computer programs in which one program, the client program, makes a service request to another, the server program. Standard networked functions such as email exchange, web access and database access, are based on the client-server model. For example, a web browser is a client program at the user computer that may access information at any web server in the world. To check your bank account from your computer, a web browser client program in your computer forwards your request to a web server program at the bank. That program may in turn forward the request to its own database client program that sends a request to a database server at another bank computer to retrieve your account balance. The balance is returned to the bank database client, which in turn serves it back to the web browser client in your personal computer, which displays the information for you.
The client-server model has become one of the central ideas of network computing. Many business applications being written today use the client-server model. So do the Internet's main application protocols, such as HTTP, SMTP, Telnet, DNS. In marketing, the term has been used to distinguish distributed computing by smaller dispersed computers from the "monolithic" centralized computing of mainframe computers. But this distinction has largely disappeared as mainframes and their applications have also turned to the client-server model and become part of network computing.
Each instance of the client software can send data requests to one or more connected servers. In turn, the servers can accept these requests, process them, and return the requested information to the client. Although this concept can be applied for a variety of reasons to many different kinds of applications, the architecture remains fundamentally the same.
The most basic type of client-server architecture employs only two types of hosts: clients and servers. This type of architecture is sometimes referred to as two-tier. It allows devices to share files and resources. The two tier architecture means that the client acts as one tier and application in combination with server acts as another tier.
The interaction between client and server is often described using sequence diagrams. Sequence diagrams are standardized in the Unified Modeling Language.
Specific types of clients include web browsers, email clients, and online chat clients.
Specific types of servers include web servers, ftp servers, application servers, database servers, name servers, mail servers, file servers, print servers, and terminal servers. Most web services are also types of servers.
Comparison to peer-to-peer architecture :
In peer-to-peer[disambiguation needed] architectures, each host or instance of the program can simultaneously act as both a client and a server, and each has equivalent responsibilities and status.
Both client-server and peer-to-peer architectures are in wide usage today. Details may be found in Comparison of Centralized (Client-Server) and Decentralized (Peer-to-Peer) Networking.
Comparison to client-queue-client architecture :
While classic client-server architecture requires one of the communication endpoints to act as a server, which is much harder to implement,[citation needed] Client-Queue-Client allows all endpoints to be simple clients, while the server consists of some external software, which also acts as passive queue (one software instance passes its query to another instance to queue, e.g. database, and then this other instance pulls it from database, makes a response, passes it to database etc.). This architecture allows greatly simplified software implementation. Peer-to-peer architecture was originally based on the Client-Queue-Client concept.
Advantages :
* In most cases, a client-server architecture enables the roles and responsibilities of a computing system to be distributed among several independent computers that are known to each other only through a network. This creates an additional advantage to this architecture: greater ease of maintenance. For example, it is possible to replace, repair, upgrade, or even relocate a server while its clients remain both unaware and unaffected by that change.
* All the data is stored on the servers, which generally have far greater security controls than most clients. Servers can better control access and resources, to guarantee that only those clients with the appropriate permissions may access and change data.
* Since data storage is centralized, updates to that data are far easier to administer than what would be possible under a P2P paradigm. Under a P2P architecture, data updates may need to be distributed and applied to each peer in the network, which is both time-consuming and error-prone, as there can be thousands or even millions of peers.
* Many mature client-server technologies are already available which were designed to ensure security, friendliness of the user interface, and ease of use.
* It functions with multiple different clients of different capabilities.
Disadvantages :
* Traffic congestion on the network has been an issue since the inception of the client-server paradigm. As the number of simultaneous client requests to a given server increases, the server can become overloaded. Contrast that to a P2P network, where its aggregated bandwidth actually increases as nodes are added, since the P2P network's overall bandwidth can be roughly computed as the sum of the bandwidths of every node in that network.
* The client-server paradigm lacks the robustness of a good P2P network. Under client-server, should a critical server fail, clients’ requests cannot be fulfilled. In P2P networks, resources are usually distributed among many nodes. Even if one or more nodes depart and abandon a downloading file, for example, the remaining nodes should still have the data needed to complete the download.
Application Server
Definition:
An application server, in an n-tier software architecture, serves an API to expose business logic and business processes for use by third-party applications.
The term can refer to:
1. the services that a server makes available
2. the computer hardware on which the services run
3. the software framework used to host the services (such as JBoss application server or Oracle Application Server)
Java application servers :
Following the success of the Java platform, the term application server sometimes refers to a J2EE or Java EE 5 application server. Some of the better-known Java Enterprise Edition application servers include:
* WebSphere Application Server and WebSphere Application Server Community Edition (IBM)
* Sybase Enterprise Application Server (Sybase Inc)
* WebLogic Server (Oracle)
* JBoss (Red Hat)
* JRun (Adobe Systems)
* Apache Geronimo (Apache Software Foundation)
* Oracle OC4J (Oracle)
* Sun Java System Application Server (based on GlassFish Application Server)(Sun Microsystems)
* SAP Netweaver AS (ABAP/Java) (SAP)
* Glassfish Application Server (open source)
The web modules include servlets and JavaServer Pages. Business logic resides in Enterprise JavaBeans (EJB-3 and later). The Hibernate project offers an EJB-3 container implementation for the JBoss application server. Tomcat from Apache and JOnAS from ObjectWeb exemplify typical containers which can store these modules.
A Java Server Page (JSP) (a servlet from Java) executes in a web container — the Java equivalent of CGI scripts. JSPs provide a way to create HTML pages by embedding references to the server logic within the page. HTML coders and Java programmers can work side by side by referencing each other's code from within their own. JavaBeans are the independent class components of the Java architecture from Sun Microsystems.
The application servers mentioned above mainly serve web applications. Some application servers target networks other than web-based ones: Session Initiation Protocol servers, for instance, target telephony networks.
Microsoft platform :
Microsoft has contributed the .NET Framework to the world of application servers. .NET technology includes the Windows Communication Foundation, .NET Remoting, ADO.NET, and ASP.NET among several other components. It works with (or depends upon) other Microsoft products, such as Microsoft Message Queuing and Internet Information Services.
Zend platform :
Zend offers an application server called Zend Server — used for running and managing PHP applications.
Other platforms :
Open-source application servers also come from other vendors. Examples include Appaserver, Base4, Zope and Spring Framework.
Non-Java offerings have no formal interoperability specifications on a par with the Java Specification Request. As a result, interoperability between non-Java products is poor compared to that of Java EE based products. To address these shortcomings, specifications for enterprise application integration and service-oriented architecture were designed[by whom?] to connect the many different products. These specifications include Business Application Programming Interface, Web Services Interoperability, and Java EE Connector Architecture.
Advantages of application servers :
Data and code integrity
By centralizing business logic on an individual server or on a small number of server machines, updates and upgrades to the application for all users can be guaranteed. There is no risk of old versions of the application accessing or manipulating data in an older, incompatible manner.
Centralized configuration
Changes to the application configuration, such as a move of database server, or system settings, can take place centrally.
Security
A central point through which service-providers can manage access to data and portions of the application itself counts as a security benefit, devolving responsibility for authentication away from the potentially insecure client layer without exposing the database layer.
Performance
By limiting the network traffic to performance-tier traffic the client-server model improves the performance of large applications in heavy usage environments.[citation needed]
Total Cost of Ownership (TCO)
In combination, the benefits above may result in cost savings to an organization developing enterprise applications. In practice, however, the technical challenges of writing software that conforms to that paradigm, combined with the need for software distribution to distribute client code, somewhat negate these benefits.
Transaction Support
A transaction represents a unit of activity in which many updates to resources (on the same or distributed data sources) can be made atomic (as an indivisible unit of work). End-users can benefit from a system-wide standard behaviour, from reduced time to develop, and from reduced costs. As the server does a lot of the tedious code-generation, developers can focus on business logic.
An application server, in an n-tier software architecture, serves an API to expose business logic and business processes for use by third-party applications.
The term can refer to:
1. the services that a server makes available
2. the computer hardware on which the services run
3. the software framework used to host the services (such as JBoss application server or Oracle Application Server)
Java application servers :
Following the success of the Java platform, the term application server sometimes refers to a J2EE or Java EE 5 application server. Some of the better-known Java Enterprise Edition application servers include:
* WebSphere Application Server and WebSphere Application Server Community Edition (IBM)
* Sybase Enterprise Application Server (Sybase Inc)
* WebLogic Server (Oracle)
* JBoss (Red Hat)
* JRun (Adobe Systems)
* Apache Geronimo (Apache Software Foundation)
* Oracle OC4J (Oracle)
* Sun Java System Application Server (based on GlassFish Application Server)(Sun Microsystems)
* SAP Netweaver AS (ABAP/Java) (SAP)
* Glassfish Application Server (open source)
The web modules include servlets and JavaServer Pages. Business logic resides in Enterprise JavaBeans (EJB-3 and later). The Hibernate project offers an EJB-3 container implementation for the JBoss application server. Tomcat from Apache and JOnAS from ObjectWeb exemplify typical containers which can store these modules.
A Java Server Page (JSP) (a servlet from Java) executes in a web container — the Java equivalent of CGI scripts. JSPs provide a way to create HTML pages by embedding references to the server logic within the page. HTML coders and Java programmers can work side by side by referencing each other's code from within their own. JavaBeans are the independent class components of the Java architecture from Sun Microsystems.
The application servers mentioned above mainly serve web applications. Some application servers target networks other than web-based ones: Session Initiation Protocol servers, for instance, target telephony networks.
Microsoft platform :
Microsoft has contributed the .NET Framework to the world of application servers. .NET technology includes the Windows Communication Foundation, .NET Remoting, ADO.NET, and ASP.NET among several other components. It works with (or depends upon) other Microsoft products, such as Microsoft Message Queuing and Internet Information Services.
Zend platform :
Zend offers an application server called Zend Server — used for running and managing PHP applications.
Other platforms :
Open-source application servers also come from other vendors. Examples include Appaserver, Base4, Zope and Spring Framework.
Non-Java offerings have no formal interoperability specifications on a par with the Java Specification Request. As a result, interoperability between non-Java products is poor compared to that of Java EE based products. To address these shortcomings, specifications for enterprise application integration and service-oriented architecture were designed[by whom?] to connect the many different products. These specifications include Business Application Programming Interface, Web Services Interoperability, and Java EE Connector Architecture.
Advantages of application servers :
Data and code integrity
By centralizing business logic on an individual server or on a small number of server machines, updates and upgrades to the application for all users can be guaranteed. There is no risk of old versions of the application accessing or manipulating data in an older, incompatible manner.
Centralized configuration
Changes to the application configuration, such as a move of database server, or system settings, can take place centrally.
Security
A central point through which service-providers can manage access to data and portions of the application itself counts as a security benefit, devolving responsibility for authentication away from the potentially insecure client layer without exposing the database layer.
Performance
By limiting the network traffic to performance-tier traffic the client-server model improves the performance of large applications in heavy usage environments.[citation needed]
Total Cost of Ownership (TCO)
In combination, the benefits above may result in cost savings to an organization developing enterprise applications. In practice, however, the technical challenges of writing software that conforms to that paradigm, combined with the need for software distribution to distribute client code, somewhat negate these benefits.
Transaction Support
A transaction represents a unit of activity in which many updates to resources (on the same or distributed data sources) can be made atomic (as an indivisible unit of work). End-users can benefit from a system-wide standard behaviour, from reduced time to develop, and from reduced costs. As the server does a lot of the tedious code-generation, developers can focus on business logic.
Thursday, July 2, 2009
Web Server
Definition:
The term web server or webserver can mean one of two things:
1. A computer program that is responsible for accepting HTTP requests from clients (user agents such as web browsers), and serving them HTTP responses along with optional data contents, which usually are web pages such as HTML documents and linked objects (images, etc.).
2. A computer that runs a computer program as described above.
Common features:
Although web server programs differ in detail, they all share some basic common features.
1. HTTP: every web server program operates by accepting HTTP requests from the client, and providing an HTTP response to the client. The HTTP response usually consists of an HTML or XHTML document, but can also be a raw file, an image, or some other type of document (defined by MIME-types). If some error is found in client request or while trying to serve it, a web server has to send an error response which may include some custom HTML or text messages to better explain the problem to end users.
2. Logging: usually web servers have also the capability of logging some detailed information, about client requests and server responses, to log files; this allows the webmaster to collect statistics by running log analyzers on these files.
In practice many web servers also implement the following features:
1. Authentication, optional authorization request (request of user name and password) before allowing access to some or all kind of resources.
2. Handling of static content (file content recorded in server's filesystem(s)) and dynamic content by supporting one or more related interfaces (SSI, CGI, SCGI, FastCGI, JSP,ColdFusion, PHP, ASP, ASP.NET, Server API such as NSAPI, ISAPI, etc.).
3. HTTPS support (by SSL or TLS) to allow secure (encrypted) connections to the server on the standard port 443 instead of usual port 80.
4. Content compression (i.e. by gzip encoding) to reduce the size of the responses (to lower bandwidth usage, etc.).
5. Virtual hosting to serve many web sites using one IP address.
6. Large file support to be able to serve files whose size is greater than 2 GB on 32 bit OS.
7. Bandwidth throttling to limit the speed of responses in order to not saturate the network and to be able to serve more clients.
Origin of returned content:
The origin of the content sent by server is called:
•static if it comes from an existing file lying on a filesystem;
•dynamic if it is dynamically generated by some other program or script or application programming interface (API) called by the web server.
Serving static content is usually much faster (from 2 to 100 times) than serving dynamic content, especially if the latter involves data pulled from a database.
Path translation:
Web servers are able to map the path component of a Uniform Resource Locator (URL) into:
• a local file system resource (for static requests);
• an internal or external program name (for dynamic requests).
For a static request the URL path specified by the client is relative to the Web server's root directory.
Consider the following URL as it would be requested by a client:
---------------------------------------------------------------
http://www.example.com/path/file.html
---------------------------------------------------------------
The client's web browser will translate it into a connection to www.example.com with the following HTTP 1.1 request:
-----------------------------------------
GET /path/file.html HTTP/1.1
Host: www.example.com
-----------------------------------------
The web server on www.example.com will append the given path to the path of its root directory. On Unix machines, this is commonly /var/www. The result is the local file system resource:
-----------------------------------------
/var/www/path/file.html
-----------------------------------------
The web server will then read the file, if it exists, and send a response to the client's web browser. The response will describe the content of the file and contain the file itself.
Load limits:
A web server (program) has defined load limits, because it can handle only a limited number of concurrent client connections (usually between 2 and 60,000, by default between 500 and 1,000) per IP address (and TCP port) and it can serve only a certain maximum number of requests per second depending on:
• its own settings;
• the HTTP request type;
• content origin (static or dynamic);
• the fact that the served content is or is not cached;
• the hardware and software limits of the OS where it is working.
When a web server is near to or over its limits, it becomes overloaded and thus unresponsive.
Overload causes:
At any time web servers can be overloaded because of:
• Too much legitimate web traffic (i.e. thousands or even millions of clients hitting the web site in a short interval of time. e.g. Slashdot effect);
• DDoS (Distributed Denial of Service) attacks;
• Computer worms that sometimes cause abnormal traffic because of millions of infected computers (not coordinated among them);
• XSS viruses can cause high traffic because of millions of infected browsers and/or web servers;
• Internet web robots traffic not filtered/limited on large web sites with very few resources (bandwidth, etc.);
• Internet (network) slowdowns, so that client requests are served more slowly and the number of connections increases so much that server limits are reached;
• Web servers (computers) partial unavailability, this can happen because of required or urgent maintenance or upgrade, HW or SW failures, back-end (i.e. DB) failures, etc.; in these cases the remaining web servers get too much traffic and become overloaded.
Overload symptoms:
The symptoms of an overloaded web server are:
• requests are served with (possibly long) delays (from 1 second to a few hundred seconds);
• 500, 502, 503, 504 HTTP errors are returned to clients (sometimes also unrelated 404 error or even 408 error may be returned);
• TCP connections are refused or reset (interrupted) before any content is sent to clients;
• in very rare cases, only partial contents are sent (but this behavior may well be considered a bug, even if it usually depends on unavailable system resources).
Anti-overload techniques:
To partially overcome above load limits and to prevent overload, most popular web sites use common techniques like:
• managing network traffic, by using:
o Firewalls to block unwanted traffic coming from bad IP sources or having bad patterns;
o HTTP traffic managers to drop, redirect or rewrite requests having bad HTTP patterns;
o Bandwidth management and traffic shaping, in order to smooth down peaks in network usage;
• deploying web cache techniques;
• using different domain names to serve different (static and dynamic) content by separate Web servers, i.e.:
http://images.example.com
http://www.example.com
• using different domain names and/or computers to separate big files from small and medium sized files; the idea is to be able to fully cache small and medium sized files and to efficiently serve big or huge (over 10 - 1000 MB) files by using different settings;
• using many Web servers (programs) per computer, each one bound to its own network card and IP address;
• using many Web servers (computers) that are grouped together so that they act or are seen as one big Web server, see also: Load balancer;
• adding more hardware resources (i.e. RAM, disks) to each computer;
• tuning OS parameters for hardware capabilities and usage;
• using more efficient computer programs for web servers, etc.;
• using other workarounds, especially if dynamic content is involved.
The term web server or webserver can mean one of two things:
1. A computer program that is responsible for accepting HTTP requests from clients (user agents such as web browsers), and serving them HTTP responses along with optional data contents, which usually are web pages such as HTML documents and linked objects (images, etc.).
2. A computer that runs a computer program as described above.
Common features:
Although web server programs differ in detail, they all share some basic common features.
1. HTTP: every web server program operates by accepting HTTP requests from the client, and providing an HTTP response to the client. The HTTP response usually consists of an HTML or XHTML document, but can also be a raw file, an image, or some other type of document (defined by MIME-types). If some error is found in client request or while trying to serve it, a web server has to send an error response which may include some custom HTML or text messages to better explain the problem to end users.
2. Logging: usually web servers have also the capability of logging some detailed information, about client requests and server responses, to log files; this allows the webmaster to collect statistics by running log analyzers on these files.
In practice many web servers also implement the following features:
1. Authentication, optional authorization request (request of user name and password) before allowing access to some or all kind of resources.
2. Handling of static content (file content recorded in server's filesystem(s)) and dynamic content by supporting one or more related interfaces (SSI, CGI, SCGI, FastCGI, JSP,ColdFusion, PHP, ASP, ASP.NET, Server API such as NSAPI, ISAPI, etc.).
3. HTTPS support (by SSL or TLS) to allow secure (encrypted) connections to the server on the standard port 443 instead of usual port 80.
4. Content compression (i.e. by gzip encoding) to reduce the size of the responses (to lower bandwidth usage, etc.).
5. Virtual hosting to serve many web sites using one IP address.
6. Large file support to be able to serve files whose size is greater than 2 GB on 32 bit OS.
7. Bandwidth throttling to limit the speed of responses in order to not saturate the network and to be able to serve more clients.
Origin of returned content:
The origin of the content sent by server is called:
•static if it comes from an existing file lying on a filesystem;
•dynamic if it is dynamically generated by some other program or script or application programming interface (API) called by the web server.
Serving static content is usually much faster (from 2 to 100 times) than serving dynamic content, especially if the latter involves data pulled from a database.
Path translation:
Web servers are able to map the path component of a Uniform Resource Locator (URL) into:
• a local file system resource (for static requests);
• an internal or external program name (for dynamic requests).
For a static request the URL path specified by the client is relative to the Web server's root directory.
Consider the following URL as it would be requested by a client:
---------------------------------------------------------------
http://www.example.com/path/file.html
---------------------------------------------------------------
The client's web browser will translate it into a connection to www.example.com with the following HTTP 1.1 request:
-----------------------------------------
GET /path/file.html HTTP/1.1
Host: www.example.com
-----------------------------------------
The web server on www.example.com will append the given path to the path of its root directory. On Unix machines, this is commonly /var/www. The result is the local file system resource:
-----------------------------------------
/var/www/path/file.html
-----------------------------------------
The web server will then read the file, if it exists, and send a response to the client's web browser. The response will describe the content of the file and contain the file itself.
Load limits:
A web server (program) has defined load limits, because it can handle only a limited number of concurrent client connections (usually between 2 and 60,000, by default between 500 and 1,000) per IP address (and TCP port) and it can serve only a certain maximum number of requests per second depending on:
• its own settings;
• the HTTP request type;
• content origin (static or dynamic);
• the fact that the served content is or is not cached;
• the hardware and software limits of the OS where it is working.
When a web server is near to or over its limits, it becomes overloaded and thus unresponsive.
Overload causes:
At any time web servers can be overloaded because of:
• Too much legitimate web traffic (i.e. thousands or even millions of clients hitting the web site in a short interval of time. e.g. Slashdot effect);
• DDoS (Distributed Denial of Service) attacks;
• Computer worms that sometimes cause abnormal traffic because of millions of infected computers (not coordinated among them);
• XSS viruses can cause high traffic because of millions of infected browsers and/or web servers;
• Internet web robots traffic not filtered/limited on large web sites with very few resources (bandwidth, etc.);
• Internet (network) slowdowns, so that client requests are served more slowly and the number of connections increases so much that server limits are reached;
• Web servers (computers) partial unavailability, this can happen because of required or urgent maintenance or upgrade, HW or SW failures, back-end (i.e. DB) failures, etc.; in these cases the remaining web servers get too much traffic and become overloaded.
Overload symptoms:
The symptoms of an overloaded web server are:
• requests are served with (possibly long) delays (from 1 second to a few hundred seconds);
• 500, 502, 503, 504 HTTP errors are returned to clients (sometimes also unrelated 404 error or even 408 error may be returned);
• TCP connections are refused or reset (interrupted) before any content is sent to clients;
• in very rare cases, only partial contents are sent (but this behavior may well be considered a bug, even if it usually depends on unavailable system resources).
Anti-overload techniques:
To partially overcome above load limits and to prevent overload, most popular web sites use common techniques like:
• managing network traffic, by using:
o Firewalls to block unwanted traffic coming from bad IP sources or having bad patterns;
o HTTP traffic managers to drop, redirect or rewrite requests having bad HTTP patterns;
o Bandwidth management and traffic shaping, in order to smooth down peaks in network usage;
• deploying web cache techniques;
• using different domain names to serve different (static and dynamic) content by separate Web servers, i.e.:
http://images.example.com
http://www.example.com
• using different domain names and/or computers to separate big files from small and medium sized files; the idea is to be able to fully cache small and medium sized files and to efficiently serve big or huge (over 10 - 1000 MB) files by using different settings;
• using many Web servers (programs) per computer, each one bound to its own network card and IP address;
• using many Web servers (computers) that are grouped together so that they act or are seen as one big Web server, see also: Load balancer;
• adding more hardware resources (i.e. RAM, disks) to each computer;
• tuning OS parameters for hardware capabilities and usage;
• using more efficient computer programs for web servers, etc.;
• using other workarounds, especially if dynamic content is involved.
Tuesday, June 30, 2009
Paypal
PayPal is an e-commerce business allowing payments and money transfers to be made through the Internet. PayPal serves as an electronic alternative to traditional paper methods such as checks and money orders.
A PayPal account can be funded with an electronic debit from a bank account or by a credit card. The recipient of a PayPal transfer can either request a check from PayPal, establish their own PayPal deposit account or request a transfer to their bank account. PayPal is an example of a payment intermediary service that facilitates worldwide e-commerce.
PayPal performs payment processing for online vendors, auction sites, and other commercial users, for which it charges a fee. It sometimes also charges a transaction fee for receiving money (a percentage of the amount sent plus an additional fixed amount). The fees charged depend on the currency used, the payment option used, the country of the sender, the country of the recipient, the amount sent and the recipient's account type.In addition, eBay purchases made by credit card through PayPal may incur a "foreign transaction fee" if the seller is located in another country, as credit card issuers are automatically informed of the seller's country of origin.
Safety and protection policies
The PayPal Buyer Protection Policy states that customers may file a buyer complaint within 45 days if they did not receive an item or if the item they purchased was significantly not as described. If the buyer used a credit card, they might get a refund via chargeback from their credit card company.
According to PayPal, it protects sellers in a limited fashion via the Seller Protection Policy.In general the Seller Protection Policy is intended to protect the seller from certain kinds of chargebacks or complaints if seller meets certain conditions including proof of delivery to the buyer. PayPal states the Seller Protection Policy is "designed to protect sellers against claims by buyers of unauthorized payments and against claims of non-receipt of any merchandise". Note that this contrasts with the consumer protection they claim to offer. This policy should be read carefully before assuming protection. In particular the Seller Protection Policy includes a list of "Exclusions" which itself includes "Intangible goods", "Claims for receipt of goods 'not as described'" and "Total reversals over the annual limit". There are also other restrictions in terms of the sale itself, the payment method and the destination country the item is shipped to (simply having a tracking mechanism is not sufficient to guarantee the Seller Protection Policy is in effect). A class-action lawsuit was filed against PayPal, days after the company's successful initial public offering.
A PayPal account can be funded with an electronic debit from a bank account or by a credit card. The recipient of a PayPal transfer can either request a check from PayPal, establish their own PayPal deposit account or request a transfer to their bank account. PayPal is an example of a payment intermediary service that facilitates worldwide e-commerce.
PayPal performs payment processing for online vendors, auction sites, and other commercial users, for which it charges a fee. It sometimes also charges a transaction fee for receiving money (a percentage of the amount sent plus an additional fixed amount). The fees charged depend on the currency used, the payment option used, the country of the sender, the country of the recipient, the amount sent and the recipient's account type.In addition, eBay purchases made by credit card through PayPal may incur a "foreign transaction fee" if the seller is located in another country, as credit card issuers are automatically informed of the seller's country of origin.
Safety and protection policies
The PayPal Buyer Protection Policy states that customers may file a buyer complaint within 45 days if they did not receive an item or if the item they purchased was significantly not as described. If the buyer used a credit card, they might get a refund via chargeback from their credit card company.
According to PayPal, it protects sellers in a limited fashion via the Seller Protection Policy.In general the Seller Protection Policy is intended to protect the seller from certain kinds of chargebacks or complaints if seller meets certain conditions including proof of delivery to the buyer. PayPal states the Seller Protection Policy is "designed to protect sellers against claims by buyers of unauthorized payments and against claims of non-receipt of any merchandise". Note that this contrasts with the consumer protection they claim to offer. This policy should be read carefully before assuming protection. In particular the Seller Protection Policy includes a list of "Exclusions" which itself includes "Intangible goods", "Claims for receipt of goods 'not as described'" and "Total reversals over the annual limit". There are also other restrictions in terms of the sale itself, the payment method and the destination country the item is shipped to (simply having a tracking mechanism is not sufficient to guarantee the Seller Protection Policy is in effect). A class-action lawsuit was filed against PayPal, days after the company's successful initial public offering.
Wednesday, May 6, 2009
Difference Between http and https
Hypertext Transfer Protocol (http) is a system for transmitting and receiving information across the Internet. Http serves as a request and response procedure that all agents on the Internet follow so that information can be rapidly, easily, and accurately disseminated between servers, which hold information, and clients, who are trying to access it. Http is commonly used to access html pages, but other resources can be utilized as well through http. In many cases, clients may be exchanging confidential information with a server, which needs to be secured in order to prevent unauthorized access. For this reason, https, or secure http, was developed by Netscape corporation to allow authorization and secured transactions.
In many ways, https is identical to http, because it follows the same basic protocols. The http or https client, such as a Web browser, establishes a connection to a server on a standard port. When a server receives a request, it returns a status and a message, which may contain the requested information or indicate an error if part of the process malfunctioned. Both systems use the same Uniform Resource Identifier (URI) scheme, so that resources can be universally identified. Use of https in a URI scheme rather than http indicates that an encrypted connection is desired.
There are some primary differences between http and https, however, beginning with the default port, which is 80 for http and 443 for https. Https works by transmitting normal http interactions through an encrypted system, so that in theory, the information cannot be accessed by any party other than the client and end server. There are two common types of encryption layers: Transport Layer Security (TLS) and Secure Sockets Layer (SSL), both of which encode the data records being exchanged.
When using an https connection, the server responds to the initial connection by offering a list of encryption methods it supports. In response, the client selects a connection method, and the client and server exchange certificates to authenticate their identities. After this is done, both parties exchange the encrypted information after ensuring that both are using the same key, and the connection is closed. In order to host https connections, a server must have a public key certificate, which embeds key information with a verification of the key owner's identity. Most certificates are verified by a third party so that clients are assured that the key is secure.
Https is used in many situations, such as log-in pages for banking, forms, corporate log ons, and other applications in which data needs to be secure. However, if not implemented properly, https is not infallible, and therefore it is extremely important for end users to be wary about accepting questionable certificates and cautious with their personal information while using the Internet.
In many ways, https is identical to http, because it follows the same basic protocols. The http or https client, such as a Web browser, establishes a connection to a server on a standard port. When a server receives a request, it returns a status and a message, which may contain the requested information or indicate an error if part of the process malfunctioned. Both systems use the same Uniform Resource Identifier (URI) scheme, so that resources can be universally identified. Use of https in a URI scheme rather than http indicates that an encrypted connection is desired.
There are some primary differences between http and https, however, beginning with the default port, which is 80 for http and 443 for https. Https works by transmitting normal http interactions through an encrypted system, so that in theory, the information cannot be accessed by any party other than the client and end server. There are two common types of encryption layers: Transport Layer Security (TLS) and Secure Sockets Layer (SSL), both of which encode the data records being exchanged.
When using an https connection, the server responds to the initial connection by offering a list of encryption methods it supports. In response, the client selects a connection method, and the client and server exchange certificates to authenticate their identities. After this is done, both parties exchange the encrypted information after ensuring that both are using the same key, and the connection is closed. In order to host https connections, a server must have a public key certificate, which embeds key information with a verification of the key owner's identity. Most certificates are verified by a third party so that clients are assured that the key is secure.
Https is used in many situations, such as log-in pages for banking, forms, corporate log ons, and other applications in which data needs to be secure. However, if not implemented properly, https is not infallible, and therefore it is extremely important for end users to be wary about accepting questionable certificates and cautious with their personal information while using the Internet.
HTTP Secure
Hypertext Transfer Protocol Secure (HTTPS) is a combination of the Hypertext Transfer Protocol and a cryptographic protocol. HTTPS connections are often used for payment transactions on the World Wide Web and for sensitive transactions in corporate information systems.
HTTP operates at the highest layer of the TCP/IP model, the Application layer; but the security protocol operates at a lower sublayer, encrypting an HTTP message prior to transmission and decrypting a message upon arrival.
Netscape Communications created HTTPS in 1994 for its Netscape Navigator web browser.Originally, HTTPS was only used with SSL encryption, but this has been made obsolete by Transport Layer Security. HTTPS was adopted as a web standard with the publication of RFC 2818 in May 2000.
Function
Strictly speaking, HTTPS is not a separate protocol, but refers to the combination of a normal HTTP interaction over an encrypted Secure Sockets Layer (SSL) or Transport Layer Security (TLS) connection. This ensures reasonable protection from eavesdroppers and man-in-the-middle attacks, provided that adequate cipher suites are used and that the server certificate is verified and trusted.
As opposed to HTTP URLs that begin with "http://", HTTPS URLs begin with "https://". If no port is explicitly specified in the URL, an HTTPS connection is opened on port 443, as opposed to unsecured HTTP which typically uses port 80.
To prepare a Web server to accept HTTPS connections, the administrator must create a public-key certificate for the Web server. These certificates can be created for Unix-based servers with tools such as OpenSSL's ssl-ca or SuSE's gensslcert. This certificate must be signed by a certificate authority of one kind or another. The authority certifies that the certificate holder is indeed the entity it claims to be. Certificates that are registered in the root chains cost between US$13 and $1,500 per year. Web browsers are generally distributed with the signing certificates of major certificate authorities so that they can verify certificates signed by them.
Organizations may also run their own certificate authority, particularly if they are responsible for setting up browsers to access their own sites (for example, sites on a company intranet). They can easily add copies of their own signing certificate to those shipped with the browser.
Some sites, especially those operated by hobbyists, use self-signed certificates on public sites. Using such certificates provides protection against simple eavesdropping, but the prevention of man-in-the-middle attacks entails the use of some other secure method of verifying the certificate.
The system can also be used for client authentication in order to limit access to a web server to authorized users. To do this, the site administrator typically creates a certificate for each user, a certificate that is loaded into his/her browser. Normally, that contains the name and e-mail address of the authorized user and is automatically checked by the server on each reconnect to verify the user's identity, potentially without even entering a password.
Browser integration
With the newer Internet Explorer 7 browser, Microsoft has increased the warnings sent when certificates are not registered. Whereas previously only a "security advice" pop-up appeared (which differentiated between name, source, and run time of the certificate), now a warning is displayed across the entire window, which recommends not using the website. Therefore, a certificate that is not registered in the browser is not usable for mass applications.
Mozilla Firefox 3.0 introduced Secure Website Certificate Verification. When you visit a secure website, Firefox will check with the certificate provider to validate that website's certificate. Firefox sends only the certificate identification to the certificate provider, not the exact URL you are visiting. If the certificate is not valid, you will receive an error page that states the certificate was revoked and you will not be able to access that website. The technical name for this process is OCSP or On-line Certificate Status Protocol. You may completely turn off the secure website certificate verification feature in Firefox's preferences under the encryption tab.
Limitations
The level of protection depends on the correctness of the implementation by the Web browser and the server software and the actual cryptographic algorithms supported.
Also, HTTPS is vulnerable when applied to publicly-available static content. The entire site can be indexed using a web crawler, and the URI of the encrypted resource can be inferred by knowing only the intercepted request/response size.This allows an attacker to have access to the plaintext (the publicly-available static content), and the encrypted text (the encrypted version of the static content).
Because SSL operates below HTTP and has no knowledge of higher-level protocols, SSL servers can only strictly present one certificate for a particular IP/port combination. This means that, in most cases, it is not feasible to use name-based virtual hosting with HTTPS. RFC-3546 TLS Extensions describes a solution called Server Name Indication (SNI), although many older browsers don't support this extension. Support for SNI is available since Firefox 2.0, Opera 8, Mozilla 1.8, and Internet Explorer 7 on Windows Vista.
Most web browsers use the address bar to tell the user that their connection is secure.
Most web browsers also stop the user from visiting sites that have invalid security certificates.
HTTP operates at the highest layer of the TCP/IP model, the Application layer; but the security protocol operates at a lower sublayer, encrypting an HTTP message prior to transmission and decrypting a message upon arrival.
Netscape Communications created HTTPS in 1994 for its Netscape Navigator web browser.Originally, HTTPS was only used with SSL encryption, but this has been made obsolete by Transport Layer Security. HTTPS was adopted as a web standard with the publication of RFC 2818 in May 2000.
Function
Strictly speaking, HTTPS is not a separate protocol, but refers to the combination of a normal HTTP interaction over an encrypted Secure Sockets Layer (SSL) or Transport Layer Security (TLS) connection. This ensures reasonable protection from eavesdroppers and man-in-the-middle attacks, provided that adequate cipher suites are used and that the server certificate is verified and trusted.
As opposed to HTTP URLs that begin with "http://", HTTPS URLs begin with "https://". If no port is explicitly specified in the URL, an HTTPS connection is opened on port 443, as opposed to unsecured HTTP which typically uses port 80.
To prepare a Web server to accept HTTPS connections, the administrator must create a public-key certificate for the Web server. These certificates can be created for Unix-based servers with tools such as OpenSSL's ssl-ca or SuSE's gensslcert. This certificate must be signed by a certificate authority of one kind or another. The authority certifies that the certificate holder is indeed the entity it claims to be. Certificates that are registered in the root chains cost between US$13 and $1,500 per year. Web browsers are generally distributed with the signing certificates of major certificate authorities so that they can verify certificates signed by them.
Organizations may also run their own certificate authority, particularly if they are responsible for setting up browsers to access their own sites (for example, sites on a company intranet). They can easily add copies of their own signing certificate to those shipped with the browser.
Some sites, especially those operated by hobbyists, use self-signed certificates on public sites. Using such certificates provides protection against simple eavesdropping, but the prevention of man-in-the-middle attacks entails the use of some other secure method of verifying the certificate.
The system can also be used for client authentication in order to limit access to a web server to authorized users. To do this, the site administrator typically creates a certificate for each user, a certificate that is loaded into his/her browser. Normally, that contains the name and e-mail address of the authorized user and is automatically checked by the server on each reconnect to verify the user's identity, potentially without even entering a password.
Browser integration
With the newer Internet Explorer 7 browser, Microsoft has increased the warnings sent when certificates are not registered. Whereas previously only a "security advice" pop-up appeared (which differentiated between name, source, and run time of the certificate), now a warning is displayed across the entire window, which recommends not using the website. Therefore, a certificate that is not registered in the browser is not usable for mass applications.
Mozilla Firefox 3.0 introduced Secure Website Certificate Verification. When you visit a secure website, Firefox will check with the certificate provider to validate that website's certificate. Firefox sends only the certificate identification to the certificate provider, not the exact URL you are visiting. If the certificate is not valid, you will receive an error page that states the certificate was revoked and you will not be able to access that website. The technical name for this process is OCSP or On-line Certificate Status Protocol. You may completely turn off the secure website certificate verification feature in Firefox's preferences under the encryption tab.
Limitations
The level of protection depends on the correctness of the implementation by the Web browser and the server software and the actual cryptographic algorithms supported.
Also, HTTPS is vulnerable when applied to publicly-available static content. The entire site can be indexed using a web crawler, and the URI of the encrypted resource can be inferred by knowing only the intercepted request/response size.This allows an attacker to have access to the plaintext (the publicly-available static content), and the encrypted text (the encrypted version of the static content).
Because SSL operates below HTTP and has no knowledge of higher-level protocols, SSL servers can only strictly present one certificate for a particular IP/port combination. This means that, in most cases, it is not feasible to use name-based virtual hosting with HTTPS. RFC-3546 TLS Extensions describes a solution called Server Name Indication (SNI), although many older browsers don't support this extension. Support for SNI is available since Firefox 2.0, Opera 8, Mozilla 1.8, and Internet Explorer 7 on Windows Vista.
Most web browsers use the address bar to tell the user that their connection is secure.
Most web browsers also stop the user from visiting sites that have invalid security certificates.
Thursday, April 23, 2009
HTTP status codes
The following is a list of HyperText Transfer Protocol (HTTP) response status codes. This includes codes from IETF internet standards as well as unstandardised RFCs, other specifications and some additional commonly used codes. The first digit of the status code specifies one of five classes of response; the bare minimum for an HTTP client is that it recognises these five classes. Microsoft IIS may use additional decimal sub-codes to provide more specific information, but these are not listed here. The phrases used are the standard examples, but any human-readable alternative can be provided. Unless otherwise stated, the status code is part of the HTTP/1.1 standard.
1xx Informational:
Request received, continuing process.
This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. Since HTTP/1.0 did not define any 1xx status codes, servers must not send a 1xx response to an HTTP/1.0 client except under experimental conditions.
100 Continue
This means that the server has received the request headers, and that the client should proceed to send the request body (in the case of a request for which a body needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the request's headers alone, a client must send Expect: 100-continue as a header in its initial request and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).
101 Switching Protocols
102 Processing (WebDAV)
2xx Success:
The action was successfully received, understood, and accepted.
This class of status code indicates that the client's request was successfully received, understood, and accepted.
200 OK
Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.
201 Created
The request has been fulfilled and resulted in a new resource being created.
202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.
203 Non-Authoritative Information (since HTTP/1.1)
204 No Content
205 Reset Content
206 Partial Content
The server is serving only part of the resource due to a range header sent by the client. This is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.
207 Multi-Status (WebDAV)
The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.
3xx Redirection:
The client must take additional action to complete the request.
This class of status code indicates that further action needs to be taken by the user agent in order to fulfil the request. The action required may be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than five times, since such redirections usually indicate an infinite loop.
300 Multiple Choices
Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.
301 Moved Permanently
This and all future requests should be directed to the given URI.
302 Found
This is the most popular redirect code[citation needed], but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945) required the client to perform a temporary redirect (the original describing phrase was "Moved Temporarily"), but popular browsers implemented it as a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to disambiguate between the two behaviours. However, the majority of Web applications and frameworks still use the 302 status code as if it were the 303.
303 See Other (since HTTP/1.1)
The response to the request can be found under another URI using a GET method. When received in response to a PUT, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.
304 Not Modified
Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Utilizing this saves bandwidth and reprocessing on both the server and client[citation needed].
305 Use Proxy (since HTTP/1.1)
Many HTTP clients (such as Mozilla[3] and Internet Explorer) do not correctly handle responses with this status code, primarily for security reasons.
306 Switch Proxy
No longer used.
307 Temporary Redirect (since HTTP/1.1)
In this occasion, the request should be repeated with another URI, but future requests can still use the original URI. In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.
4xx Client Error:
The request contains bad syntax or cannot be fulfilled.
The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents should display any included entity to the user. These are typically the most common error codes encountered while online.
400 Bad Request
The request contains bad syntax or cannot be fulfilled.
401 Unauthorized
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. See Basic access authentication and Digest access authentication.
402 Payment Required
The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code has never been used.
403 Forbidden
The request was a legal request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference.
404 Not Found
The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
406 Not Acceptable
The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
407 Proxy Authentication Required
408 Request Timeout
Client failed to continue the request
409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
410 Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.
411 Length Required
The request did not specify the length of its content, which is required by the requested resource.
412 Precondition Failed
413 Request Entity Too Large
The resource that was requested is too large to transmit using the current protocol.
414 Request-URI Too Long
The URI provided was too long for the server to process.
415 Unsupported Media Type
The request did not specify any media types that the server or resource supports. For example the client specified that an image resource should be served as image/svg+xml, but the server cannot find a matching version of the image.
416 Requested Range Not Satisfiable
The client has asked for a portion of the file, but the server cannot supply that portion (for example, if the client asked for a part of the file that lies beyond the end of the file).
417 Expectation Failed
418 I'm a teapot
The HTCPCP server is a teapot. The responding entity MAY be short and stout. Defined by the April Fools' specification RFC 2324. See Hyper Text Coffee Pot Control Protocol for more information.
422 Unprocessable Entity (WebDAV) (RFC 4918)
The request was well-formed but was unable to be followed due to semantic errors.
423 Locked (WebDAV) (RFC 4918)
The resource that is being accessed is locked
424 Failed Dependency (WebDAV) (RFC 4918)
The request failed due to failure of a previous request (e.g. a PROPPATCH).
425 Unordered Collection
Defined in drafts of WebDav Advanced Collections, but not present in "Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol" (RFC 3648).
426 Upgrade Required (RFC 2817)
The client should switch to TLS/1.0.
449 Retry With
A Microsoft extension. The request should be retried after doing the appropriate action.
450 Blocked
A Microsoft extension. Used for blocking sites with Windows Parental Controls.
5xx Server Error:
The server failed to fulfil an apparently valid request.
Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has encountered an error or is otherwise incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and indicate whether it is a temporary or permanent condition. Likewise, user agents should display any included entity to the user. These response codes are applicable to any request method.
500 Internal Server Error
A generic error message, given when no more specific message is suitable.
501 Not Implemented
The server either does not recognise the request method, or it lacks the ability to fulfil the request.
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
506 Variant Also Negotiates (RFC 2295)
507 Insufficient Storage (WebDAV) (RFC 4918)
509 Bandwidth Limit Exceeded (Apache bw/limited extension)
This status code, while used by many servers, is not specified in any RFCs.
510 Not Extended (RFC 2774)
Further extensions to the request are required for the server to fulfil it.
1xx Informational:
Request received, continuing process.
This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. Since HTTP/1.0 did not define any 1xx status codes, servers must not send a 1xx response to an HTTP/1.0 client except under experimental conditions.
100 Continue
This means that the server has received the request headers, and that the client should proceed to send the request body (in the case of a request for which a body needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the request's headers alone, a client must send Expect: 100-continue as a header in its initial request and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).
101 Switching Protocols
102 Processing (WebDAV)
2xx Success:
The action was successfully received, understood, and accepted.
This class of status code indicates that the client's request was successfully received, understood, and accepted.
200 OK
Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.
201 Created
The request has been fulfilled and resulted in a new resource being created.
202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.
203 Non-Authoritative Information (since HTTP/1.1)
204 No Content
205 Reset Content
206 Partial Content
The server is serving only part of the resource due to a range header sent by the client. This is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.
207 Multi-Status (WebDAV)
The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.
3xx Redirection:
The client must take additional action to complete the request.
This class of status code indicates that further action needs to be taken by the user agent in order to fulfil the request. The action required may be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than five times, since such redirections usually indicate an infinite loop.
300 Multiple Choices
Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.
301 Moved Permanently
This and all future requests should be directed to the given URI.
302 Found
This is the most popular redirect code[citation needed], but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945) required the client to perform a temporary redirect (the original describing phrase was "Moved Temporarily"), but popular browsers implemented it as a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to disambiguate between the two behaviours. However, the majority of Web applications and frameworks still use the 302 status code as if it were the 303.
303 See Other (since HTTP/1.1)
The response to the request can be found under another URI using a GET method. When received in response to a PUT, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.
304 Not Modified
Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Utilizing this saves bandwidth and reprocessing on both the server and client[citation needed].
305 Use Proxy (since HTTP/1.1)
Many HTTP clients (such as Mozilla[3] and Internet Explorer) do not correctly handle responses with this status code, primarily for security reasons.
306 Switch Proxy
No longer used.
307 Temporary Redirect (since HTTP/1.1)
In this occasion, the request should be repeated with another URI, but future requests can still use the original URI. In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.
4xx Client Error:
The request contains bad syntax or cannot be fulfilled.
The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents should display any included entity to the user. These are typically the most common error codes encountered while online.
400 Bad Request
The request contains bad syntax or cannot be fulfilled.
401 Unauthorized
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. See Basic access authentication and Digest access authentication.
402 Payment Required
The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code has never been used.
403 Forbidden
The request was a legal request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference.
404 Not Found
The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
406 Not Acceptable
The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
407 Proxy Authentication Required
408 Request Timeout
Client failed to continue the request
409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
410 Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.
411 Length Required
The request did not specify the length of its content, which is required by the requested resource.
412 Precondition Failed
413 Request Entity Too Large
The resource that was requested is too large to transmit using the current protocol.
414 Request-URI Too Long
The URI provided was too long for the server to process.
415 Unsupported Media Type
The request did not specify any media types that the server or resource supports. For example the client specified that an image resource should be served as image/svg+xml, but the server cannot find a matching version of the image.
416 Requested Range Not Satisfiable
The client has asked for a portion of the file, but the server cannot supply that portion (for example, if the client asked for a part of the file that lies beyond the end of the file).
417 Expectation Failed
418 I'm a teapot
The HTCPCP server is a teapot. The responding entity MAY be short and stout. Defined by the April Fools' specification RFC 2324. See Hyper Text Coffee Pot Control Protocol for more information.
422 Unprocessable Entity (WebDAV) (RFC 4918)
The request was well-formed but was unable to be followed due to semantic errors.
423 Locked (WebDAV) (RFC 4918)
The resource that is being accessed is locked
424 Failed Dependency (WebDAV) (RFC 4918)
The request failed due to failure of a previous request (e.g. a PROPPATCH).
425 Unordered Collection
Defined in drafts of WebDav Advanced Collections, but not present in "Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol" (RFC 3648).
426 Upgrade Required (RFC 2817)
The client should switch to TLS/1.0.
449 Retry With
A Microsoft extension. The request should be retried after doing the appropriate action.
450 Blocked
A Microsoft extension. Used for blocking sites with Windows Parental Controls.
5xx Server Error:
The server failed to fulfil an apparently valid request.
Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has encountered an error or is otherwise incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and indicate whether it is a temporary or permanent condition. Likewise, user agents should display any included entity to the user. These response codes are applicable to any request method.
500 Internal Server Error
A generic error message, given when no more specific message is suitable.
501 Not Implemented
The server either does not recognise the request method, or it lacks the ability to fulfil the request.
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
506 Variant Also Negotiates (RFC 2295)
507 Insufficient Storage (WebDAV) (RFC 4918)
509 Bandwidth Limit Exceeded (Apache bw/limited extension)
This status code, while used by many servers, is not specified in any RFCs.
510 Not Extended (RFC 2774)
Further extensions to the request are required for the server to fulfil it.
Wednesday, April 22, 2009
Hypertext Transfer Protocol (http)
Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. Its use for retrieving inter-linked resources led to the establishment of the World Wide Web.
HTTP development was coordinated by the World Wide Web Consortium and the Internet Engineering Task Force (IETF), culminating in the publication of a series of Requests for Comments (RFCs), most notably RFC 2616 (June 1999), which defines HTTP/1.1, the version of HTTP in common use.
HTTP is a request/response standard of a client and a server. A client is the end-user, the server is the web site. The client making a HTTP request—using a web browser, spider, or other end-user tool—is referred to as the user agent. The responding server—which stores or creates resources such as HTML files and images—is called the origin server. In between the user agent and origin server may be several intermediaries, such as proxies, gateways, and tunnels. HTTP is not constrained to using TCP/IP and its supporting layers, although this is its most popular application on the Internet. Indeed HTTP can be "implemented on top of any other protocol on the Internet, or on other networks." HTTP only presumes a reliable transport; any protocol that provides such guarantees can be used."
Typically, an HTTP client initiates a request. It establishes a Transmission Control Protocol (TCP) connection to a particular port on a host (port 80 by default; see List of TCP and UDP port numbers). An HTTP server listening on that port waits for the client to send a request message. Upon receiving the request, the server sends back a status line, such as "HTTP/1.1 200 OK", and a message of its own, the body of which is perhaps the requested resource, an error message, or some other information.
Resources to be accessed by HTTP are identified using Uniform Resource Identifiers (URIs) (or, more specifically, Uniform Resource Locators (URLs)) using the http: or https URI schemes.
Request message:
uest message consists of the following:
* Request line, such as GET /images/logo.gif HTTP/1.1, which requests a resource called /images/logo.gif from server
* Headers, such as Accept-Language: en
* An empty line
* An optional message body
The request line and headers must all end with (that is, a carriage return followed by a line feed). The empty line must consist of only and no other whitespace. In the HTTP/1.1 protocol, all headers except Host are optional.
A request line containing only the path name is accepted by servers to maintain compatibility with HTTP clients before the HTTP/1.0 specification.
Request methods:
HTTP defines eight methods (sometimes referred to as "verbs") indicating the desired action to be performed on the identified resource. What this resource represents, whether pre-existing data or data that is generated dynamically, depends on the implementation of the server. Often, the resource corresponds to a file or the output of an executable residing on the server.
HEAD
Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
GET
Requests a representation of the specified resource. Note that GET should not be used for operations that cause side-effects, such as using it for taking actions in web applications. One reason for this is that GET may be used arbitrarily by robots or crawlers, which should not need to consider the side effects that a request should cause. See safe methods below.
POST
Submits data to be processed (e.g., from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both.
PUT
Uploads a representation of the specified resource.
DELETE
Deletes the specified resource.
TRACE
Echoes back the received request, so that a client can see what intermediate servers are adding or changing in the request.
OPTIONS
Returns the HTTP methods that the server supports for specified URL. This can be used to check the functionality of a web server by requesting '*' instead of a specific resource.
CONNECT
Converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
HTTP servers are required to implement at least the GET and HEAD methods and, whenever possible, also the OPTIONS method.
Safe methods
Some methods (for example, HEAD, GET, OPTIONS and TRACE) are defined as safe, which means they are intended only for information retrieval and should not change the state of the server. In other words, they should not have side effects, beyond relatively harmless effects such as logging, caching, the serving of banner advertisements or incrementing a web counter. Making arbitrary GET requests without regard to the context of the application's state should therefore be considered safe.
By contrast, methods such as POST, PUT and DELETE are intended for actions which may cause side effects either on the server, or external side effects such as financial transactions or transmission of email. Such methods are therefore not usually used by conforming web robots or web crawlers, which tend to make requests without regard to context or consequences.
Despite the prescribed safety of GET requests, in practice their handling by the server is not technically limited in any way, and careless or deliberate programming can just as easily (or more easily, due to lack of user agent precautions) cause non-trivial changes on the server. This is discouraged, because it can cause problems for Web caching, search engines and other automated agents, which can make unintended changes on the server.
Idempotent methods and web applications
Methods PUT and DELETE are defined to be idempotent, meaning that multiple identical requests should have the same effect as a single request. Methods GET, HEAD, OPTIONS and TRACE, being prescribed as safe, should also be idempotent, as HTTP is a stateless protocol.
By contrast, the POST method is not necessarily idempotent, and therefore sending an identical POST request multiple times may further affect state or cause further side effects (such as financial transactions). In some cases this may be desirable, but in other cases this could be due to an accident, such as when a user does not realize that their action will result in sending another request, or they did not receive adequate feedback that their first request was successful. While web browsers may show alert dialog boxes to warn users in some cases where reloading a page may re-submit a POST request, it is generally up to the web application to handle cases where a POST request should not be submitted more than once.
Note that whether a method is idempotent is not enforced by the protocol or web server. It is perfectly possible to write a web application in which (for example) a database insert or other non-idempotent action is triggered by a GET or other request. Ignoring this recommendation, however, may result in undesirable consequences if a user agent assumes that repeating the same request is safe when it isn't.
Status codes:
n HTTP/1.0 and since, the first line of the HTTP response is called the status line and includes a numeric status code (such as "404") and a textual reason phrase (such as "Not Found"). The way the user agent handles the response primarily depends on the code and secondarily on the response headers. Custom status codes can be used since, if the user agent encounters a code it does not recognize, it can use the first digit of the code to determine the general class of the response.
Also, the standard reason phrases are only recommendations and can be replaced with "local equivalents" at the web developer's discretion. If the status code indicated a problem, the user agent might display the reason phrase to the user to provide further information about the nature of the problem. The standard also allows the user agent to attempt to interpret the reason phrase, though this might be unwise since the standard explicitly specifies that status codes are machine-readable and reason phrases are human-readable.
List of HTTP status codes
1. 1xx Informational
2. 2xx Success
3. 3xx Redirection
4. 4xx Client Error
5. 5xx Server Error
Persistent connections:
In HTTP/0.9 and 1.0, the connection is closed after a single request/response pair. In HTTP/1.1 a keep-alive-mechanism was introduced, where a connection could be reused for more than one request.
Such persistent connections reduce lag perceptibly, because the client does not need to re-negotiate the TCP connection after the first request has been sent.
Version 1.1 of the protocol made bandwidth optimization improvements to HTTP/1.0. For example, HTTP/1.1 introduced chunked transfer encoding to allow content on persistent connections to be streamed, rather than buffered. HTTP pipelining further reduces lag time, allowing clients to send multiple requests before a previous response has been received to the first one. Another improvement to the protocol was byte serving, which is when a server transmits just the portion of a resource explicitly requested by a client.
HTTP session state:
HTTP is a stateless protocol. The advantage of a stateless protocol is that hosts do not need to retain information about users between requests, but this forces web developers to use alternative methods for maintaining users' states. For example, when a host needs to customize the content of a website for a user, the web application must be written to track the user's progress from page to page. A common method for solving this problem involves sending and receiving cookies. Other methods include server side sessions, hidden variables (when the current page is a form), and URL encoded parameters (such as /index.php?session_id=some_unique_session_code).
Secure HTTP:
There are currently two methods of establishing a secure HTTP connection: the HTTPS URI scheme and the HTTP 1.1 Upgrade header, introduced by RFC 2817. Browser support for the Upgrade header is, however, nearly non-existent, hence the HTTPS URI scheme is still the dominant method of establishing a secure HTTP connection. Secure HTTP is notated by the prefix HTTPS:// instead of HTTP://
HTTPS URI scheme
HTTPS: is a URI scheme syntactically identical to the http: scheme used for normal HTTP connections, but which signals the browser to use an added encryption layer of SSL/TLS to protect the traffic. SSL is especially suited for HTTP since it can provide some protection even if only one side of the communication is authenticated. This is the case with HTTP transactions over the Internet, where typically only the server is authenticated (by the client examining the server's certificate).
HTTP 1.1 Upgrade header
HTTP 1.1 introduced support for the Upgrade header. In the exchange, the client begins by making a clear-text request, which is later upgraded to TLS. Either the client or the server may request (or demand) that the connection be upgraded. The most common usage is a clear-text request by the client followed by a server demand to upgrade the connection, which looks like this:
Client:
GET /encrypted-area HTTP/1.1
Host: www.example.com
Server:
HTTP/1.1 426 Upgrade Required
Upgrade: TLS/1.0, HTTP/1.1
Connection: Upgrade
The server returns a 426 status-code because 400 level codes indicate a client failure (see List of HTTP status codes), which correctly alerts legacy clients that the failure was client-related.
The benefits of using this method for establishing a secure connection are:
* that it removes messy and problematic redirection and URL rewriting on the server side,
* it allows virtual hosting of secured websites (although HTTPS also allows this using Server Name Indication), and
* it reduces user confusion by providing a single way to access a particular resource.
A weakness with this method is that the requirement for a secure HTTP cannot be specified in the URI. In practice, the (untrusted) server will thus be responsible for enabling secure HTTP, not the (trusted) client.
HTTP development was coordinated by the World Wide Web Consortium and the Internet Engineering Task Force (IETF), culminating in the publication of a series of Requests for Comments (RFCs), most notably RFC 2616 (June 1999), which defines HTTP/1.1, the version of HTTP in common use.
HTTP is a request/response standard of a client and a server. A client is the end-user, the server is the web site. The client making a HTTP request—using a web browser, spider, or other end-user tool—is referred to as the user agent. The responding server—which stores or creates resources such as HTML files and images—is called the origin server. In between the user agent and origin server may be several intermediaries, such as proxies, gateways, and tunnels. HTTP is not constrained to using TCP/IP and its supporting layers, although this is its most popular application on the Internet. Indeed HTTP can be "implemented on top of any other protocol on the Internet, or on other networks." HTTP only presumes a reliable transport; any protocol that provides such guarantees can be used."
Typically, an HTTP client initiates a request. It establishes a Transmission Control Protocol (TCP) connection to a particular port on a host (port 80 by default; see List of TCP and UDP port numbers). An HTTP server listening on that port waits for the client to send a request message. Upon receiving the request, the server sends back a status line, such as "HTTP/1.1 200 OK", and a message of its own, the body of which is perhaps the requested resource, an error message, or some other information.
Resources to be accessed by HTTP are identified using Uniform Resource Identifiers (URIs) (or, more specifically, Uniform Resource Locators (URLs)) using the http: or https URI schemes.
Request message:
uest message consists of the following:
* Request line, such as GET /images/logo.gif HTTP/1.1, which requests a resource called /images/logo.gif from server
* Headers, such as Accept-Language: en
* An empty line
* An optional message body
The request line and headers must all end with
A request line containing only the path name is accepted by servers to maintain compatibility with HTTP clients before the HTTP/1.0 specification.
Request methods:
HTTP defines eight methods (sometimes referred to as "verbs") indicating the desired action to be performed on the identified resource. What this resource represents, whether pre-existing data or data that is generated dynamically, depends on the implementation of the server. Often, the resource corresponds to a file or the output of an executable residing on the server.
HEAD
Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
GET
Requests a representation of the specified resource. Note that GET should not be used for operations that cause side-effects, such as using it for taking actions in web applications. One reason for this is that GET may be used arbitrarily by robots or crawlers, which should not need to consider the side effects that a request should cause. See safe methods below.
POST
Submits data to be processed (e.g., from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both.
PUT
Uploads a representation of the specified resource.
DELETE
Deletes the specified resource.
TRACE
Echoes back the received request, so that a client can see what intermediate servers are adding or changing in the request.
OPTIONS
Returns the HTTP methods that the server supports for specified URL. This can be used to check the functionality of a web server by requesting '*' instead of a specific resource.
CONNECT
Converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
HTTP servers are required to implement at least the GET and HEAD methods and, whenever possible, also the OPTIONS method.
Safe methods
Some methods (for example, HEAD, GET, OPTIONS and TRACE) are defined as safe, which means they are intended only for information retrieval and should not change the state of the server. In other words, they should not have side effects, beyond relatively harmless effects such as logging, caching, the serving of banner advertisements or incrementing a web counter. Making arbitrary GET requests without regard to the context of the application's state should therefore be considered safe.
By contrast, methods such as POST, PUT and DELETE are intended for actions which may cause side effects either on the server, or external side effects such as financial transactions or transmission of email. Such methods are therefore not usually used by conforming web robots or web crawlers, which tend to make requests without regard to context or consequences.
Despite the prescribed safety of GET requests, in practice their handling by the server is not technically limited in any way, and careless or deliberate programming can just as easily (or more easily, due to lack of user agent precautions) cause non-trivial changes on the server. This is discouraged, because it can cause problems for Web caching, search engines and other automated agents, which can make unintended changes on the server.
Idempotent methods and web applications
Methods PUT and DELETE are defined to be idempotent, meaning that multiple identical requests should have the same effect as a single request. Methods GET, HEAD, OPTIONS and TRACE, being prescribed as safe, should also be idempotent, as HTTP is a stateless protocol.
By contrast, the POST method is not necessarily idempotent, and therefore sending an identical POST request multiple times may further affect state or cause further side effects (such as financial transactions). In some cases this may be desirable, but in other cases this could be due to an accident, such as when a user does not realize that their action will result in sending another request, or they did not receive adequate feedback that their first request was successful. While web browsers may show alert dialog boxes to warn users in some cases where reloading a page may re-submit a POST request, it is generally up to the web application to handle cases where a POST request should not be submitted more than once.
Note that whether a method is idempotent is not enforced by the protocol or web server. It is perfectly possible to write a web application in which (for example) a database insert or other non-idempotent action is triggered by a GET or other request. Ignoring this recommendation, however, may result in undesirable consequences if a user agent assumes that repeating the same request is safe when it isn't.
Status codes:
n HTTP/1.0 and since, the first line of the HTTP response is called the status line and includes a numeric status code (such as "404") and a textual reason phrase (such as "Not Found"). The way the user agent handles the response primarily depends on the code and secondarily on the response headers. Custom status codes can be used since, if the user agent encounters a code it does not recognize, it can use the first digit of the code to determine the general class of the response.
Also, the standard reason phrases are only recommendations and can be replaced with "local equivalents" at the web developer's discretion. If the status code indicated a problem, the user agent might display the reason phrase to the user to provide further information about the nature of the problem. The standard also allows the user agent to attempt to interpret the reason phrase, though this might be unwise since the standard explicitly specifies that status codes are machine-readable and reason phrases are human-readable.
List of HTTP status codes
1. 1xx Informational
2. 2xx Success
3. 3xx Redirection
4. 4xx Client Error
5. 5xx Server Error
Persistent connections:
In HTTP/0.9 and 1.0, the connection is closed after a single request/response pair. In HTTP/1.1 a keep-alive-mechanism was introduced, where a connection could be reused for more than one request.
Such persistent connections reduce lag perceptibly, because the client does not need to re-negotiate the TCP connection after the first request has been sent.
Version 1.1 of the protocol made bandwidth optimization improvements to HTTP/1.0. For example, HTTP/1.1 introduced chunked transfer encoding to allow content on persistent connections to be streamed, rather than buffered. HTTP pipelining further reduces lag time, allowing clients to send multiple requests before a previous response has been received to the first one. Another improvement to the protocol was byte serving, which is when a server transmits just the portion of a resource explicitly requested by a client.
HTTP session state:
HTTP is a stateless protocol. The advantage of a stateless protocol is that hosts do not need to retain information about users between requests, but this forces web developers to use alternative methods for maintaining users' states. For example, when a host needs to customize the content of a website for a user, the web application must be written to track the user's progress from page to page. A common method for solving this problem involves sending and receiving cookies. Other methods include server side sessions, hidden variables (when the current page is a form), and URL encoded parameters (such as /index.php?session_id=some_unique_session_code).
Secure HTTP:
There are currently two methods of establishing a secure HTTP connection: the HTTPS URI scheme and the HTTP 1.1 Upgrade header, introduced by RFC 2817. Browser support for the Upgrade header is, however, nearly non-existent, hence the HTTPS URI scheme is still the dominant method of establishing a secure HTTP connection. Secure HTTP is notated by the prefix HTTPS:// instead of HTTP://
HTTPS URI scheme
HTTPS: is a URI scheme syntactically identical to the http: scheme used for normal HTTP connections, but which signals the browser to use an added encryption layer of SSL/TLS to protect the traffic. SSL is especially suited for HTTP since it can provide some protection even if only one side of the communication is authenticated. This is the case with HTTP transactions over the Internet, where typically only the server is authenticated (by the client examining the server's certificate).
HTTP 1.1 Upgrade header
HTTP 1.1 introduced support for the Upgrade header. In the exchange, the client begins by making a clear-text request, which is later upgraded to TLS. Either the client or the server may request (or demand) that the connection be upgraded. The most common usage is a clear-text request by the client followed by a server demand to upgrade the connection, which looks like this:
Client:
GET /encrypted-area HTTP/1.1
Host: www.example.com
Server:
HTTP/1.1 426 Upgrade Required
Upgrade: TLS/1.0, HTTP/1.1
Connection: Upgrade
The server returns a 426 status-code because 400 level codes indicate a client failure (see List of HTTP status codes), which correctly alerts legacy clients that the failure was client-related.
The benefits of using this method for establishing a secure connection are:
* that it removes messy and problematic redirection and URL rewriting on the server side,
* it allows virtual hosting of secured websites (although HTTPS also allows this using Server Name Indication), and
* it reduces user confusion by providing a single way to access a particular resource.
A weakness with this method is that the requirement for a secure HTTP cannot be specified in the URI. In practice, the (untrusted) server will thus be responsible for enabling secure HTTP, not the (trusted) client.
Subscribe to:
Posts (Atom)
