HTTP request and response data tool: Difference between revisions

From LemonWiki共筆
Jump to navigation Jump to search
 
(4 intermediate revisions by the same user not shown)
Line 59: Line 59:
</pre>
</pre>


=== Redirect checker ===
* [https://wheregoes.com/ Link Checker | Redirect Checker - WhereGoes] {{Gd}}
Curl command
<pre>
curl -sI -L "https://example.com/" | grep -i "^location:"
</pre>
This command '''traces the redirect chain of a URL''', printing only the destination of each redirect step. Here's the breakdown:
<code>curl -sI -L "https://example.com/"</code>
* '''<code>curl</code>''': a tool for sending HTTP requests
* '''<code>-s</code>''' (silent): quiet mode, suppresses progress bars and other status info
* '''<code>-I</code>''' (capital, <code>--head</code>): fetches only the '''HTTP headers''', not the page body — faster
* '''<code>-L</code>''' (<code>--location</code>): if the server responds with a 3xx redirect status code (like 301, 302, 307), curl will '''automatically follow''' the new URL and keep requesting until it reaches the final page (or hits the redirect limit)
* '''<code>"https://example.com/"</code>''': the target URL to query
So this sends a HEAD request for each redirect step along the way, and prints all the response headers, which might look like:
<pre>
HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/
...
HTTP/1.1 200 OK
...
</pre>
<code>| grep -i "^location:"</code>
* '''<code>|</code>''': pipes curl's output into <code>grep</code>
* '''<code>grep -i</code>''': searches text, <code>-i</code> means case-insensitive (so it matches both <code>Location:</code> and <code>location:</code>)
* '''<code>"^location:"</code>''': <code>^</code> means '''start of line''', so it only picks lines that '''begin with''' <code>location:</code> — that is, the new destination URL specified by the redirect header
This command will:
# Send a request to the target URL, fetching only headers
# Automatically follow any redirects
# Filter out every <code>Location:</code> line — i.e., the destination URL of '''each''' redirect step


=== other tools ===
=== other tools ===
Line 75: Line 114:
== HTTP & HTTPS Proxy ==
== HTTP & HTTPS Proxy ==
* [https://mitmproxy.org/ mitmproxy - an interactive HTTPS proxy] ([https://github.com/mitmproxy/mitmproxy/blob/main/LICENSE MIT license] {{Gd}}) on {{Win}}, {{Mac}} & {{Linux}}<ref>[https://docs.mitmproxy.org/stable/overview/installation/ Installation]</ref>
* [https://mitmproxy.org/ mitmproxy - an interactive HTTPS proxy] ([https://github.com/mitmproxy/mitmproxy/blob/main/LICENSE MIT license] {{Gd}}) on {{Win}}, {{Mac}} & {{Linux}}<ref>[https://docs.mitmproxy.org/stable/overview/installation/ Installation]</ref>
* [https://portswigger.net/burp Burp Suite - Application Security Testing Software - PortSwigger]
* ''$'' [https://www.charlesproxy.com/ Charles Web Debugging Proxy • HTTP Monitor / HTTP Proxy / HTTPS & SSL Proxy / Reverse Proxy] on {{Mac}}
* ''$'' [https://www.charlesproxy.com/ Charles Web Debugging Proxy • HTTP Monitor / HTTP Proxy / HTTPS & SSL Proxy / Reverse Proxy] on {{Mac}}
* ''$'' [https://www.telerik.com/fiddler Fiddler - Web Debugging Proxy - Telerik] on {{Win}}, {{Mac}} & {{Linux}}
* ''$'' [https://www.telerik.com/fiddler Fiddler - Web Debugging Proxy - Telerik] on {{Win}}, {{Mac}} & {{Linux}}
* ''$'' [https://proxyman.io/ Proxyman · Native, Modern Web Debugging Proxy · Inspect network from Mac, iOS, Android devices with ease]
* ''$'' [https://proxyman.io/ Proxyman · Native, Modern Web Debugging Proxy · Inspect network from Mac, iOS, Android devices with ease]
* [https://portswigger.net/burp Burp Suite - Application Security Testing Software - PortSwigger]


== Web page compression check ==
== Web page compression check ==
Line 95: Line 134:
<references/>
<references/>


[[Category:Design]]
[[Category: Design]]
[[Category:Programming]]
[[Category: Programming]]
[[Category:Data collecting]]
[[Category: Data collecting]]
[[Category:Security]]
[[Category: Security]]
[[Category: Revised with LLMs]]

Latest revision as of 15:25, 31 July 2026

<< Testing

HTTP request and response data tool[edit]

Testing the API or pressure test.

HTTP headers generator[edit]

  • Apache Jmeter v. 2.7[1]
    • approach 1: (1) Add config element: HTTP request defaults (2) Add Sampler: HTTP request (3) Add Listener: View results tree (4) Run the test plan
    • approach 2: (1) install unofficial jmeter-plugins (2) Add Sampler: jp@gc - HTTP Raw Request[2] (3) Add Listener: View results tree (4) Run the test plan
telnet localhost 80
"header content" Enter
Enter

Display HTTP headers of a web page[edit]

  • curl e.g. Input the command curl -L -I <URL>[Last visited: 2018-09-20]
    • Option -L, --location "If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code)." quoted from manual.
    • Option -I, --head "Fetch the headers only!" quoted from manual.
  • Wget - GNU Project - Free Software Foundation e.g. Input the command wget -S --spider <URL> [3][Last visited: 2018-09-20]
    • Option -S, --server-response "Print the headers sent by HTTP servers and responses sent by FTP servers." quoted from manual.
    • Option --spider "When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there." quoted from manual.

Example result after executed curl command: 

$ curl -L -I https://www.google.com

HTTP/2 200
date: Thu, 20 Sep 2018 02:56:29 GMT
expires: -1
cache-control: private, max-age=0
content-type: text/html; charset=ISO-8859-1
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
set-cookie: 1P_JAR=2018-09-20-02; expires=Sat, 20-Oct-2018 02:56:29 GMT; path=/; domain=.google.com
set-cookie: NID=139=DXgMIx0L06ZUBLaTUD2J_pqIvfgSEo945An0URyIwGqVf_NOxPcHcaAxhNwNforv-Lw0-m6DSKX-y1wz0EhuC-tdzLHPyWYqLVOdu7VBgjH9spnMr_2MfY79uh05aYuH; expires=Fri, 22-Mar-2019 02:56:29 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: quic=":443"; ma=2592000; v="44,43,39,35"
accept-ranges: none
vary: Accept-Encoding

Redirect checker[edit]

Curl command 

curl -sI -L "https://example.com/" | grep -i "^location:"

This command traces the redirect chain of a URL, printing only the destination of each redirect step. Here's the breakdown:

curl -sI -L "https://example.com/"

  • curl: a tool for sending HTTP requests
  • -s (silent): quiet mode, suppresses progress bars and other status info
  • -I (capital, --head): fetches only the HTTP headers, not the page body — faster
  • -L (--location): if the server responds with a 3xx redirect status code (like 301, 302, 307), curl will automatically follow the new URL and keep requesting until it reaches the final page (or hits the redirect limit)
  • "https://example.com/": the target URL to query

So this sends a HEAD request for each redirect step along the way, and prints all the response headers, which might look like: 

HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/
...

HTTP/1.1 200 OK
...

| grep -i "^location:"

  • |: pipes curl's output into grep
  • grep -i: searches text, -i means case-insensitive (so it matches both Location: and location:)
  • "^location:": ^ means start of line, so it only picks lines that begin with location: — that is, the new destination URL specified by the redirect header

This command will:

  1. Send a request to the target URL, fetching only headers
  2. Automatically follow any redirects
  3. Filter out every Location: line — i.e., the destination URL of each redirect step

other tools[edit]

echo

  • httpbin.org "A simple HTTP Request & Response Service."

web security

HTTP & HTTPS Proxy[edit]

Web page compression check[edit]

online gzip test


related article[edit]

References[edit]

  1. ↑ JMeter proxy Step-by-step
  2. ↑ RawRequest - jmeter-plugins - Every load test needs some sexy features! - Google Project Hosting
  3. ↑ Viewing HTTP Headers using Wget | Zyxware Technologies
  4. ↑ Installation
Retrieved from "https://wiki.planetoid.info/index.php?title=HTTP_request_and_response_data_tool&oldid=26421"