Web Basics

HTTP Protocol

HTTP Request Headers

GET / HTTP/1.1
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: text/html,application/xhtml+xml,application/xml;
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close

Explanation

1. Method + Resource + Protocol

   • GET: retrieve pages, send parameters(data) in URL.

   • POST: send parameters(data) in message body like form submissions.

   • HEAD: similar to GET but without message body.

   • OPTIONS: list all the supported methods by the web server.

   • PUT: put files on the server.

   • DELETE: delete files from the server.

   • TRACE: print the request back in the response body for debugging purposes.

2. Domain name of the server like example.com or example.com:8080

3. Name of the web browser used

4. Media types that are acceptable for the response

5. List of acceptable human languages for the response

6. List of acceptable encoding for the response

7. Control options for the current connection like close or keep-alive

HTTP Response Headers

HTTP1.1 200 OK
Date: Wed, 09 Oct 2019 19:58:01 GMT
Server: Apache/2.4.41 (Debian)
Content-Type: text/html;charset=UTF-8
Content-Length: 3115
Connection: close

Explanation

1. Protocol + Status-Code

   • 1xx: Information

   • 2xx: Success

   • 3xx: Redirect

   • 4xx: Resource not found or Unauthorized

   • 5xx: Internal server error

2. The date and time that the message was sent

3. Name of the web server

4. The MIME type of this content

5. The length of the response body

6. Control options for the current connection like close or keep-alive

Reference: https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

HTTP Cookies

Explanation

• Content: PHPSESSID=7145drfkb8tmhjetbh56qgtjo3;

• Expire: 13 Mar 2019, IF NOT EXIST will expire when browser closed.

• Domain: google.com IF NOT EXIST will be the target host only (127.0.0.1).

• Path: /blog which means (/blog and /blog/folder1 and /blog/folder1/file.html)

• HTTPONLY: access through http only not javascript.

• Secure: access through https only not http.

Note about Cookies Domain:

• Cookies for y.z.com domain will be applicable to y.z.com and x.y.z.com and a.x.y.z.com.

• Domain like x.y.z.com can set cookies for itself and it's parents (x.y.z.com and y.z.com).

HTTP Authentication

Basic Authentication

Request: GET /BasicAuth HTTP/1.1

Response: WWW-Authenticate: Basic realm="Any Text"

Request: Authorization: Basic base64(username:password)
Ex:      Authorization: Basic YWRtaW46YWRtaW4=

Response: Failed or Success

Digest Authentication

Request: GET /DigestAuth HTTP/1.1

Response: WWW-Authenticate: Digest realm="Any Text",
          nonce="w4n4QvmDBQA=fc49b571115859e9a3b7ac4d9c68ec06bdf6415f", 
          algorithm=MD5,
          qop="auth"

Request: Authorization: Digest username="user1", realm="Any Text", 
         nonce="w4n4QvmDBQA=fc49b571115859e9a3b7ac4d9c68ec06bdf6415f",
         uri="/scripts/digest",
         algorithm=MD5, 
         response="2b2c8d7e6f3c495b3b7d2b963198f596",
         qop=auth, 
         nc=00000001,
         cnonce="6b15c646d6dc6d43"

Ex:      HA1 = MD5(username:realm:password)
         HA2 = MD5(method:digestURI)
         response = MD5(HA1:nonce:nc:cnonce:qop:HA2)

Response: Failed or Success

HTTPS Protocol

Explanation

1. Google buy a signed certification from Certificate Authority (CA).

2. Web browser ask google.com to identify himself.

3. Google send his certificate and public key to web browser.

4. Web browser check the certification from CA.

5. Web browser encrypt session key by the google public key and send it to google.com.

6. Google decrypt the session key with his private key, and then use that session key to encrypt and decrypt all the messages between him and web browser.

7. Now the web browser and google.com can encrypt and decrypt all the messages using that key.

# Test Using NMAP
nmap -p 443 --script ssl-enum-ciphers yahoo.com

# Test Using SSLlab
https://www.ssllabs.com/ssltest/

Same Origin Policy (SOP)

The same-origin policy is a restrictive cross-origin specification that limits the ability for a website to interact with resources outside of the source domain. The same-origin policy was defined many years ago in response to potentially malicious cross-domain interactions, such as one website stealing private data from another. It generally allows a domain to issue requests to other domains, but not to access the responses.

Must three parts match:

• Protocol: http, https

• Host: google.com, yahoo.com

• Port: 80, 443

Compare this website:

• http://www.example.com

Hidden Directories & Files

# Wordlists
wget https://raw.githubusercontent.com/danielmiessler/RobotsDisallowed/master/archive/Top100000-RobotsDisallowed.txt
wget https://gist.githubusercontent.com/jhaddix/b80ea67d85c13206125806f0828f4d10/raw/c81a34fe84731430741e0463eb6076129c20c4c0/content_discovery_all.txt
sort content_discovery_all.txt Top100000-RobotsDisallowed.txt | uniq > result.txt

# Gobuster
gobuster dir -u http://google.com -t 100 -w result.txt
gobuster dir -u http://google.com -t 100 -w result.txt -f -x sh,cgi

# dirb and dirbuster
dirb http://google.com