Handling User access (Authentication,Authorization,Accounting,Access Control,Session Management)

                                        Crimson Security Group
                                                              We yearn Security.....

Handling User Access

A central security requirement that virtually any application needs to meet  is controlling users access to its data and functionality. A typical situation has several different categories of user, such as anonymous users, ordinary authenticated users, and administrative users. Furthermore, in many situations different users are permitted to access a different set of data.

 Authentication

Authentication is a process by which you verify that someone is who they claim they are.

This usually involves asking the user for a user name and a password, but it can include any other method of demonstrating identity, for example a smart card, a PIN number, a secret code sent in a letter in the post, a fingerprint scan, and so on.In order to perform authentication, a user must already have an account created in a system which can be interrogated by the authentication mechanism, or an account must be created as part of the process of the very first authentication.

Without this facility, the application would need to treat all users as anonymous

The majority of today’s web applications employ the conventional authentication model, in which the user submits a user name and password, which the application checks for validity 

 

 Authentication mechanisms suffer from a wide range of defects in both design and implementation. Common problems may enable an attacker to identify other users’ user names, guess their passwords, or bypass the login function by exploiting defects in its logic. When you are attacking a web application you should invest a significant amount of attention to the various authentication-related functions it contains. Surprisingly frequently, defects in this functionality enable you to gain unauthorized access to sensitive data and functionality.

Once you have authenticated a user, they may be authorized for different types of access or activity.

Authorization

Authorization is the process of establishing if the user (who is already authenticated), is permitted to have access to a resource. Authorization determines what a user is and is not allowed to do.

For example, data associated with a user may indicate if they are a member of a given group such as “Administrators” or “Customers”, or it may indicate if they have paid a subscription for some paid-for content, or it may indicate that they are still within the 90 day period of a free trial.

Authorization also includes an Authorization Management component, which is a system that provides the functionality to create the authorization rules. For example, it may allow an administrator to create a rule to allow another user to edit or publish content to a website. Authorization Management often uses Groups, Roles, Privileges and Permissions to define these rules.

Accounting 

which measures the resources a user consumes during access. This can include the amount of system time or the amount of data a user has sent and/or received during a session. Accounting is carried out by logging of session statistics and usage information and is used for authorization control, billing, trend analysis, resource utilization, and capacity planning activities.

Session Management 

   

     What is session ?  

            

            A session can be defined as a server-side storage of information that is

           desired to persist throughout the user's interaction with the web site or

           web application.  

The next logical task in the process of handling user access is to manage the

authenticated user’s session. After successfully logging in to the application, the user accesses various pages and functions, making a series of HTTP requests from his browser. At the same time, the application receives countless other requests from different users, some of whom are authenticated and some of whom are anonymous. To enforce effective access control, the application needs a way to identify and process the series of requests that originate from each unique user.

Virtually all web applications meet this requirement by creating a session for each user and issuing the user a token that identifies the session. The session  itself is a set of data structures held on the server that track the state of the user’s interaction with the application. The token is a unique string that the application maps to the session. When a user receives a token, the browser automatically submits it back to the server in each subsequent HTTP request, enabling the application to associate the request with that user. HTTP cookies are the standard method for transmitting session tokens, although many applications use hidden form fields or the URL query string for this purpose. If a user does not make a request for a certain amount of time, the session is ideally expired.

In terms of attack surface, the session management mechanism is highly dependent on the security of its tokens. The majority of attacks against it seek to compromise the tokens issued to other users. If this is possible, an attacker can masquerade as the victim user and use the application just as if he had actually authenticated as that user. The principal areas of vulnerability arise from defects in how tokens are generated, enabling an attacker to guess the tokens issued to other users, and defects in how tokens are subsequently handled, enabling an attacker to capture other users’ tokens.

A small number of applications dispense with the need for session tokens by using other means of reidentifying users across multiple requests. If HTTP’s built-in authentication mechanism is used, the browser automatically resubmits the user’s credentials with each request, enabling the application to identify the user directly from these. In other cases, the application stores the state information on the client side rather than the server, usually in encrypted form to prevent tampering

Access Control

It is the process of handling user access is to make and enforce correct decisions about whether each individual request should be permitted or denied.

Access Control is the process of enforcing the required security for a particular resource.Once we know who a user is, and we know what authorization level they have and what we should and should not give them access to, we need to physically prevent that user from accessing anything that they should not be able to. Access Control can be seen as the combination of Authentication and Authorization plus additional measures, such as clock- or IP-based restrictions.

Note that lack of adequate access control is more often the cause of security vulnerabilities in applications than faulty authentication or authorization mechanisms, simply because access control is more complex to implement and becomes more complex as the application being secured becomes more complex itself.

 

 

 

 

 

 

                                                  

HTTP Security Headers

                                       Crimson Security Group
                                                          We yearn security.....


HTTP security headers provide yet another layer of security by helping to mitigate attacks and security vulnerabilities.

A number of new HTTP headers have been introduced whose purpose is to help enhancing the security of a website. Some of these headers can be very useful protection against certain type of attacks, but their use is not widely spread in some cases.Latest attacks such as Click jacking, XSS Filter Bypassing..etc have dramatically affected the World Wide Web community.

The main idea is to specify security in an HTTP parameter that is set by the server and is sent along to the web browser as a part of the in-line response. A browser renders the content of that web page by scrutinizing(Examining) the HTTP headers and tries to invoke the specified security module in order to head off the attacks.

As the name implies, the protection parameters for a specific set of attack are declared by the developer as a part of the web server or application running on the server.

Most of these declarative security protection parameters are not the part of HTTP 1.1 specification, but are considered as vendor specific or customized security solutions related to a specific product. Usually, the declarative security in HTTP parameters is understood as the”X” factor protection. Most of the HTTP headers start with”X” in order to differentiate between standard HTTP 1.1 and normalized ones.Microsoft, Chrome and Mozilla have adopted this type of security.

Example : X-XSS-Protection, X-Content-Security-Policy,X-Frame-Options etc

Content-Security-Policy

It is a browser based mechanism that allows you to white list  locations from which your web application can load resources 

This policy helps prevent attacks such as Cross Site Scripting (XSS) and other code injection attacks by defining content sources which are approved and thus allowing the browser to load them.Involves deciding what policies you want to enforce, and then configuring them and using X-Content-Security-Policy to establish your policy.Allows the client to detect and block malicious scripts injected into the application by an attacker.

 

Example :

 Lets see the Directive keywords for Content Security Policy....

How it Helps ?

 

        

Reference Link:    

                       https://developer.mozilla.org/en-US/docs/Web/Security/CSP

                              http://www.slideshare.net/BinuRamakrishnan/content-security-policy-lessons-learned-at-yahoo-55438493

Strict-Transport-Security

HTTP Strict Transport Security (HSTS) is a security enhancement that restricts web browsers to access web servers solely over HTTPS. This ensures the connection cannot be establish through an insecure HTTP connection which could be susceptible to attacks. HSTS is defined in the response header as Strict-Transport-Security and once the supported browser receives that header it knows to deliver all information over HTTPS.

How it Works

In order to set up HTTP Strict Transport Security, it must first be enabled on your origin server. Once it is enabled on the server side, the web server will begin adding an additional response header to tell the browser to communicate solely over HTTPS

   Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

a)   max-age defines the time in seconds for which the web server should only    

      deliver through HTTPS 

b)   IncludeSubDomains is optimal. This will apply HSTS to all the site’s

      subdomains as well.

 c)  preload is also optional. The site owner can submit their website to the 

     preload list which is a list of sites hardcoded into Chrome as being HTTPS

     only. 

The following 3 points are common threats that HSTS is able to protect against. 

1) Attackers using an invalid certificate in the hopes the user will accept the

    bad cert

2) Old bookmarks that contain http:// or manually entered http:// urls that can be

    vulnerable to an attack 

3) Sites claiming to be fully HTTPS serving HTTP content

Due to HSTS’s strict rules, the above threats will no longer be relevant as it doesn’t allow the use of insecure HTTP.

 

Valid Settings :

The following values must exist over the secure connection (HTTPS) and are ineffective if accessed over HTTP.

* max-age=31536000 - Tells the user-agent to cache the domain in the STS list
   for one year.  

* max-age=31536000; includeSubDomains - Tells the user-agent to cache the
  domain in the STS list for one year and include any sub-domains. 

* max-age=0 - Tells the user-agent to remove, or not cache the host in the STS
   cache.
  
Invalid Settings :
 
 *  Setting the includeSubDomains directive on https://www.example.com where users can still  
     access the site at http://example.com. If example.com does not redirect to https://example.com
     and set the STS header, only direct requests to http://www.example.com will be automatically
     redirected to https://www.example.com by the user-agent. 

 *  max-age=60 - This only sets the domain in the STS cache for 60 seconds. This is not long
     enough to protect a user who accesses the site, goes to their local coffee shop and attempts to
     access the site over http first. 

 *  max-age=31536000 includeSubDomains - Directives must be separated by a ;. In this case
     Chrome will not add the site to the STS cache even though the max-age value is correct. 

 *  max-age=31536000, includeSubDomains - Same as above. 

 *  max-age=0 - While this is technically a valid configuration. Many sites may do this accidentally,
     thinking a value of 0 means forever.
 

 X-Frame-Options

Stop Clickjacking with one simple header

Where a website can be embedded in an IFRAME element, an attacker could socially engineer a situation where a victim is directed to a website under an attacker’s control that frames the target website. The attacker could then manipulate the victim into unknowingly performing actions on the target website. This attack is possible even with cross-site request forgery protection in place, and is known as "clickjacking", for more information please refer to https://www.owasp.org/index.php/Clickjacking. In order to avoid this, the "X-Frame-Options" header was created. This header lets the owner of the website decide which sites are allowed to frame their site.

The common recommendation is to set this header to either "SAMEORIGIN", which allows only resources which are part of the Same Origin Policy to frame the protected resource, or to "DENY", which denies any resource (local or remote) from attempting to frame the resource that also supplied the "X-Frame-Options" header. This is shown below:

 

                 X-Frame-Options: SAMEORIGIN

DENY - Denies any resource (local or remote) from attempting to frame the resource that also supplied the X-Frame-Options header 

SAMEORIGIN - Allows only resources which are apart of the same origin policy to frame the protected resource. 

ALLOW-FROM http://www.example.com - Allows a single serialized-origin (must have scheme) to frame the protected resource. This is only valid in Internet Explorer and Firefox. The default of other browsers is to allow any origin (as if X-Frame-Options was not set). 

X-XSS-Protection

This response header can be used to configure a user-agent's built in reflective XSS protection. Currently, only Microsoft's Internet Explorer, Google Chrome and Safari (WebKit) support this header.

The recommended configuration is to set this header to the following value, which will enable the XSS protection and instruct the browser to block the response in the event that a malicious script has been inserted from user input, instead of sanitizing:

                  X-XSS-Protection: 1; mode=block. 

Valid Settings :

 *  0 - Disables the XSS Protections offered by the user-agent. 

 *  1 - Enables the XSS Protections 

 * 1; mode=block - Enables XSS protections and instructs the user-agent to block

   the response in the event that script has been inserted from user input, instead

   of sanitizing. 

 * 1; report=http://site.com/report - A Chrome and WebKit only directive that 

   tells the user-agent to report potential XSS attacks to a single URL. Data will

   be POST'd to the report URL in JSON format

 Invalid Settings:

* 0; mode=block; - A common misconfiguration where the 0  value will disable

  protections even though the mode=block is defined. It should be noted that

  Chrome has been enhanced to fail closed and treat this as an invalid setting but

  still keep default XSS protections in place. 

* 1 mode=block; - All directives must be separated by a ;. Spaces and , are

   invalid separators. However, IE and Chrome will default to sanitizing the XSS 

  in this case but not enable blocking mode as everything after the 1 is 

  considered invalid.

X-Content-Type-Options

A nifty attack known as MIME type confusion was the reason this header was created. Most of the browsers employ a technique called MIME sniffing, that consists on taking an educate guess at what the content type of the server response is, instead of trusting what the header’s content type value says. Under certain circumstances, browsers can be tricked into making the incorrect decision, allowing attackers to execute malicious code on victim’s browsers. For more information please refer to https://en.wikipedia.org/wiki/Content_sniffing. 

"X-Content-Type-Options" can be used to prevent this "educated" guess from happening by setting the value of this header to "nosniff", as shown below:

                              X-Content-Type-Options: nosniff 

      nosniff - This is the only valid setting, it must match nosniff.

Please, note that Internet Explorer, Chrome and Safari have support for this header, but the Firefox team is still debating it:https://bugzilla.mozilla.org/show_bug.cgi?id=471020

CORS   ( cross-origin resource sharing )

Nowadays, websites are a rich and complex flow of information. It is common to find websites that interact with other websites, for example requesting resources such as JavaScript scripts and fonts. There is a mechanism known as cross-origin resource sharing (CORS) that makes this possible in a secure manner. As part of the CORS specification, a header known as "Access-Control-Allow-Origin" was defined. This header is used to determine which websites are allowed to access certain resources.  

As an example of this header, to permit https://www.example.com to request and integrate your site’s data and content would be the following:

       Access-Control-Allow-Origin: https://www.example.com

The use of a wildcard (*) is allowed, but not generally recommended. If a wildcard is used, anyone would be permitted to request and integrate your site’s data and content.

For a more detailed explanation of all the headers that are part of the CORS standard, and how they all work, please refer to the following resource: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS  

It is recommended that you do not set any CORS related headers unless you are sure you want to allow requests from other websites you do not control. A website that does not set any CORS headers, will not allow requests from any other sites.

 

 

 

Format of an HTTP Response

                                     Crimson Security Group
                                                             We yearn security.....

Format of an HTTP Response

Similar to an HTTP Request, an HTTP Response also has three main components, which are :

a) Protocol/Version, Status Code, and its Description - 

The very first line of a valid HTTP Response is consists of the protocol name, it's version, status code of the request, and a short description of the status code. A status code of 200 means the processing of request was successful and the description in this case will be 'OK'. Similarly, a status code of '404' means the file requested was not found at the HTTP Server at the expected location and the description in this case is 'File Not Found'.

b) HTTP Response Headers

similar to HTTP Request Headers, HTTP Response Headers also contain useful information. The only difference is that HTTP Request Headers contain information about the environment of the client machine whereas HTTP Response Headers contain information about the environment of the server machine. This is easy to understand as HTTP Requests are formed at the client machine whereas HTTP Responses are formed at the server machine. Few of these HTTP Response headers are: Server, Content-Type, Last-Modified, Content-Length, etc.

c)HTTP Response Body

This the actual response which is rendered in the client window (the browser window). The content of the body will be HTML code. Similar to HTTP Request, in this case also the Body and the Headers components are separated by a mandatory blank line (CRLF sequence). 

 

HTTP status Codes :   

Reference Links :

 

https://httpstatuses.com/ 

https://www.smartlabsoftware.com/ref/http-status-codes.htm

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

HTTP Response Headers:

The response-header fields allow the server to pass additional information about the response which cannot be placed in the Status- Line. These header fields give information about the server and about further access to the resource identified by the Request-URI

1) Accept-Ranges - The Accept-Ranges response-header field allows the   

     server to indicate its acceptance of range requests for a resource.

2) Age - The Age response-header field conveys the sender's estimate of the

     amount of time since the response (or its revalidation) was generated at 

     the origin server. A cached response is "fresh" if its age does not exceed

     its freshness lifetime.

3) ETag - The ETag response-header field provides the current value of the

     entity tag for the requested variant. 

4) Proxy-Authenticate - The Proxy-Authenticate response-header field 

     MUST be included as part of a 407 (Proxy Authentication Required)

     response. The field value consists of a challenge that indicates the

     authentication scheme and parameters applicable to the proxy for this

     Request-URI.

5)  Server - The Server response-header field contains information about

      the software used by the origin server to handle the request 

6) WWW-Authenticate - The WWW-Authenticate response-header field MUST 

    be included in 401 (Unauthorized) response messages. The field value consists

    of at least one challenge that indicates the authentication scheme(s) and

   parameters applicable to the Request-URI. 

so many response headers are there, but we are not going to discuss here.When we involve in to learning vulnerabilities one by one I can add.Below you can find links for further studying....

Reference Link:

       https://www.whitehatsec.com/blog/list-of-http-response-headers/

      

       https://www.w3.org/Protocols/rfc2616/rfc2616.html

Format of an HTTP Request

                                       Crimson Security Group
                                                                  we yearn security......

Format of an HTTP Request

It has three main components, which are:-

     1) HTTP Request Method, URI, and Protocol Version

    2) HTTP Request Headers

    3) HTTP Request Body  


HTTP Request :

A request message from a client to a server includes, within the first line of that message, the method to be applied to the resource, the identifier of the resource, and the protocol version in use.  

Request    =    Request-Line

                *(( general-header| request-header | entity-header ) CRLF) 

                  CRLF 

                 [ message-body ] 

Request-Line :

           Request-Line   = Method SP Request-URI SP HTTP-Version CRLF

                      The Request-Line begins with a method token, followed by the
                      Request-URI and the protocol version, and ending with CRLF. The
                      elements are separated by SP characters. No CR or LF is allowed
                      except in the final CRLF sequence.

Consider the following HTTP request example to understand easily... 

HTTP Methods:

The Method token indicates the method to be performed on the resource identified by the Request-URI. The method is case-sensitive.

GET :  Retrieve a Document

The GET method is used to retrieve information from the given server using a given URI. Requests using GET should only retrieve data and should have no other effect on the data.This is the main method used for retrieving html, images, JavaScript, CSS, etc. Most data that loads in your browser was requested using this method.  

POST: Send Data to the server

A POST request is used to send data to the server, for example, customer information, file upload, etc. using HTML forms. Even though you can send data to the server using GET and the query string, in many cases POST will be preferable. Sending large amounts of data using GET is not practical and has limitations.

Difference between GET and POST methods ....

HEAD: Retrieve Header Information

Same as GET, but it transfers the status line and the header section only.HEAD is identical to GET, except the server does not return the content in the HTTP response. When you send a HEAD request, it means that you are only interested in the response code and the HTTP headers, not the document itself.

With this method the browser can check if a document has been modified, for caching purposes. It can also check if the document exists at all.For example, if you have a lot of links on your website, you can periodically send HEAD requests to all of them to check for broken links. This will work much faster than using GET.

        EXAMPLE :  

                         Request :    HEAD /index.html HTTP/1.0
                                       ( blank line)

                         Response:    
                                      HTTP/1.1 200 OK
                                      Date: Sun, 18 Oct 2009 14:09:16 GMT
                                      Server: Apache/2.2.14 (Win32)
                                      Last-Modified: Sat, 20 Nov 2004 07:16:26 GMT
                                      ETag: "10000000565a5-2c-3e94b66c2e680"
                                      Accept-Ranges: bytes
                                      Content-Length: 44
                                      Connection: close
                                      Content-Type: text/html
                                      X-Pad: avoid browser bug

Notice that the response consists of the header only without the body, which contains 

the actual document.

PUT  :

A POST request is used to send data to the server, for example, customer information, file upload, etc. using HTML forms.

consider the difference between POST and PUT...

PATCH :

The HTTP methods PATCH can be used to update partial resources

Difference between PUT and PATCH ....

PUT = replace the ENTIRE RESOURCE with the new representation provided 

PATCH = replace parts of the source resource with the values provided  

Example 1: 

you have a profile settings page and in the form you want to change only your name but not your twitter account and git hub account. You'll only update part of the data. For that, you will use a PATCH request.

Example 2: 

You have a form with location settings where you have to change all of the fields: City, Country, Zip Code, Address etc. For that, you'll use a PUT request.

DELETE :

 Removes all the current representations of the target resource given by URI.

CONNECT :

Establishes a tunnel to the server identified by a given URI

HTTP CONNECT can be used to establish a network connection to a web server over HTTP. It’s primarily used in cases where a secure/encrypted HTTP connection (tunnel) needs to be established between a client and a web server such as an SSL connection.

The HTTP CONNECT request is used to ask a proxy to make a connection to anther host and simply relay the content, rather than attempting to parse or cache the message. This is often used to make a connection through a proxy.  

Simple HTTP tunnels are an unencrypted connection through an HTTP proxy to an arbitrary destination. The tunnel takes advantage of the HTTP CONNECT method normally used for HTTPS (secure web traffic) to connect to the destination server. 

OPTIONS :

Options is useful for finding out which HTTP methods are accessible by a client. Depending on how the web server you are trying to connect to is configured, the administrator may only have the POST and GET HTTP methods accessible. While other HTTP methods such as DELETE, TRACE, etc are disabled.

A client can use an OPTIONS request method to query the server which request methods are supported

  EXAMPLE:

                    

OPTIONS http://www.example.com/ HTTP/1.1           ***  REQUEST LINE ***
Host: www.example.com
Connection: Close
(blank line)

HTTP/1.1 200 OK                                    *** RESPONSE LINE ***
Date: Fri, 27 Feb 2004 09:42:46 GMT
Content-Length: 0
Connection: close
Server:Apache/1.3.6 
Allow: GET, HEAD, POST, OPTIONS, TRACE           (Allowed Methods) 
Connection: close
(blank line) 

 

TRACE:

Performs a message loop back test along with the path to the target resource.

HTTP TRACE is used to echo the contents of an HTTP Request back to the   requester (which can be useful for debugging). This however may pose a security threat because malicious code can abuse HTTP TRACE functionality to gain access to information in HTTP headers such as cookies and authentication data, if an HTTP TRACE request is sent the original request data will be returned in addition to any user specific data. 

A client can send a TRACE request to ask the server to return a diagnostic trace.

REQUEST 

TRACE http://www.example.com/ HTTP/1.1
Host: www.example.com
Connection: Close
(blank line)

RESPONSE

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Fri, 27 Feb 2004 09:44:21 GMT
Content-Type: message/http
Connection: close
Server:  Apache/1.3.6
Connection: close

   
9d
TRACE / HTTP/1.1
Connection: keep-alive
Host: www.example.com
Via: 1.1 xproxy (NetCache NetApp/5.3.1R4D5)

   

   

HTTP  HEADERS:

HTTP headers allow the client and the server to pass additional information with the request or the response. A request header consists of its case-insensitive name followed by a colon ':', then by its value (without line breaks). Leading white space before the value is ignored.Custom proprietary headers can be added using the 'X-' prefix.

Headers can be grouped according to their contexts:

a) General Headers:

Headers applying to both requests and responses but with no relation to the data eventually transmitted in the body.

             "https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.5"

b) Request Headers:

Headers containing more information about the resource to be fetched or about the client itself
    
            " https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5 "

c) Response Headers:

Headers with additional information about the response, like its location or about the server itself (name and version etc.)

            "https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6"

d) Entity  Headers:

Headers containing more information about the body of the entity, like its content length or its MIME-type  

              " https://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html#sec7 "

Reference links : 

https://www.cs.tut.fi/~jkorpela/http.html 

https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

http://geekexplains.blogspot.in/2008/06/whats-http-explain-http-request-and.html 

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
 
https://www.ntu.edu.sg/home/ehchua/programming/webprogramming/HTTP_Basics.html

Structure of HTTP Transactions

                                                Crimson Security Group
                                                                      we yearn security....


Structure of HTTP Transactions:

First we need to Understand what are resources....
  
What are  " resources " ?

HTTP is used to transmit resources, not just files. A resource is some chunk of information that can be identified by a URL (it's the R in URL). The most common kind of resource is a file, but a resource may also be a dynamically-generated query result, the output of a CGI script, a document that is available in several languages, or something else.

While learning HTTP, it may help to think of a resource as similar to a file, but more general. As a practical matter, almost all HTTP resources are currently either files or server-side script output.  

Now see how HTTP Transaction occurs....

Like most network protocols, HTTP uses the client-server model: An HTTP client opens a connection and sends a request message to an HTTP server; the server then returns a response message, usually containing the resource that was requested. After delivering the response, the server closes the connection (making HTTP a stateless protocol, i.e. not maintaining any connection information between transactions).

The format of the request and response messages are similar, and English-oriented. Both kinds of messages consist of:  

* an initial line,

* zero or more header lines,

* a blank line (i.e. a CRLF by itself), and  

* an optional message body (e.g. a file, or query data, or query output). 

The format of an HTTP message is:  

<initial line, different for request vs. response>
Header1: value1
Header2: value2
Header3: value3

<optional message body goes here, like file contents or query data;
 it can be many lines long, or even binary data $&*%@!^$@>

Initial lines and headers should end in CRLF, though you should gracefully handle lines ending in just LF. (More exactly, CR and LF here mean ASCII values 13 and 10, even though some platforms may use different characters.) 

Initial Request Line

The initial line is different for the request than for the response. A request line has three parts, separated by spaces: a method name, the local path of the requested resource, and the version of HTTP being used. A typical request line is:

  1                 2                      3

GET /path/to/file/index.html HTTP/1.0

Method names are always uppercase (GET is a method)

 

1   GET is the most common HTTP method; it says "give me this resource". We   

     will discuss more methods later...

2   The path is the part of the URL after the host name, also called the request    

     URI (a URI is like a URL, but more general)

3   The HTTP version always takes the form "HTTP/x.x", uppercase 

Initial Response Line (Status Line)

The initial response line, called the status line, also has three parts separated by spaces: the HTTP version, a response status code that gives the result of the request, and an English reason phrase describing the status code. Typical status lines are:  

HTTP/1.0 200 OK            ---> status code 200 OK  means resource available

or

HTTP/1.0 404 Not Found     

The HTTP version is in the same format as in the request line, "HTTP/x.x". 

The status code is meant to be computer-readable; the reason phrase is meant to be human-readable, and may vary.

The status code is a three-digit integer, and the first digit identifies the general category of response

      1xx indicates an informational message only
       2xx indicates success of some kind
       3xx redirects the client to another URL
       4xx indicates an error on the client's part
      5xx indicates an error on the server's part 

The most common status codes are:  

200 OK               The request succeeded, and the resulting resource (e.g. file or 
                        script output) is returned in the message body. 

404 Not Found     The requested resource doesn't exist

 

301              Moved Permanently  

 

302              Moved Temporarily  

 

303 See Other (HTTP 1.1 only)

 

The resource has moved to another URL (given by the Location: response header), and should be automatically retrieved by the client. This is often used by a CGI script to redirect the browser to an existing file

500 Server Error

An unexpected server error. The most common cause is a server-side script that has bad syntax, fails, or otherwise can't run correctly. 

Reference Link: https://httpstatuses.com/
 

Header Lines

Header lines provide information about the request or response, or about the object sent in the message body.The header lines are in the usual text header format, which is: one line per header, of the form "Header-Name: value", ending with CRLF.

*  As noted above, they should end in CRLF, but you should handle      LF correctly. 

*  The header name is not case-sensitive (though the value may be)

*  Any number of spaces or tabs may be between the ":" and the          value.

*  Header lines beginning with space or tab are actually part of the    previous header line, folded into multiple lines for easy reading.

HTTP 1.0 defines 16 headers, though none are required.          HTTP 1.1 defines 46 headers, and one (Host:) is required in requests.

The From: header gives the email address of whoever making the request, or running the program doing so. (This must be user-configurable, for privacy concerns.)  

The User-Agent: header identifies the program that's making the request, in the form "Program-name/x.xx", where x.xx is the (mostly) alphanumeric version of the program. For example, Netscape 3.0 sends the header "User-agent: Mozilla/3.0Gold". 

These headers help web masters troubleshoot problems. They also reveal information about the user. When you decide which headers to include, you must balance the web masters logging needs against your users' needs for privacy. 

The Message Body

An HTTP message may have a body of data sent after the header lines. In a response, this is where the requested resource is returned to the client (the most common use of the message body), or perhaps explanatory text if there's an error. In a request, this is where user-entered data or uploaded files are sent to the server

If an HTTP message includes a body, there are usually header lines in the message that describe the body. In particular,

The Content-Type: header gives the MIME-type of the data in the body, such as text/html or image/gif.

The Content-Length: header gives the number of bytes in the body.

Sample HTTP Exchange

To retrieve the file at the URL

http://www.somehost.com/path/file.html

first open a socket to the host www.somehost.com, port 80 (use the default port of 80 because none is specified in the URL). Then, send something like the following through the socket: 

GET /path/file.html HTTP/1.0
From: someuser@crimson.com
User-Agent: HTTPTool/1.0
[blank line here]

The server should respond with something like the following, sent back through the same socket: 

HTTP/1.0 200 OK
Date: Fri, 31 Dec 1999 23:59:59 GMT
Content-Type: text/html
Content-Length: 1354

<html>
<body>
<h1>we yearn security</h1>
(more file contents)
  .
  .
  .
</body>
</html>

 After sending the response, the server closes the socket.