HTTP request and response data tool: Difference between revisions

From LemonWiki共筆
Jump to navigation Jump to search
mNo edit summary
 
(25 intermediate revisions by the same user not shown)
Line 21: Line 21:


=== Display HTTP headers of a web page ===
=== Display HTTP headers of a web page ===
* [http://getfirebug.com/ Firebug] for {{Fx}}: Menu -> Net
* {{Gd}} [https://www.getpostman.com/ Postman | API Development Environment]
* {{Chrome}}: Press {{kbd | key = F12}} to open the '''Developer Tools''' window -> Switch to ''Network'' panel -> Click the ''Headers'' to display the HTTP headers of a web page. (1)Section ''Query String Parameters'' for HTTP GET request (2)Section ''Form Data'' for HTTP POST request.  Further reading: [https://developers.google.com/chrome-developer-tools/docs/network?hl=zh-TW Chrome DevTools — Google Developers]
* [https://www.mozilla.org/zh-TW/firefox/new/ Firefox]: [https://developer.mozilla.org/en-US/docs/Tools/Web_Console/Opening_the_Web_Console Opening the Web Console - Firefox Developer Tools | MDN] --> Switch to "Network" label --> Click one of URLs --> Show the Headers. (formerly [http://getfirebug.com/ Firebug])
 
* [http://livehttpheaders.mozdev.org/ LiveHTTPHeaders] for {{Fx}}
* [http://livehttpheaders.mozdev.org/ LiveHTTPHeaders] for {{Fx}}
* [http://www.ericgiguere.com/articles/masquerading-your-browser.html Masquerading Your Browser]: also offer the tool [http://www.ericgiguere.com/tools/http-header-viewer.html HTTP Header Viewer] {{access | date=2012-05-16}}
* [http://www.ericgiguere.com/articles/masquerading-your-browser.html Masquerading Your Browser]: also offer the tool [http://www.ericgiguere.com/tools/http-header-viewer.html HTTP Header Viewer] {{access | date=2012-05-16}}
* {{Chrome}}: Press {{kbd | key = F12}} to open the '''Developer Tools''' window -> Switch to ''Network'' panel -> Click the ''Headers'' to display the HTTP headers of a web page. (1)Section ''Query String Parameters'' for HTTP GET request (2)Section ''Form Data'' for HTTP POST request. Further reading: [https://developers.google.com/chrome-developer-tools/docs/network?hl=zh-TW Chrome DevTools — Google Developers]
 
* [https://curl.haxx.se/ curl] e.g. Input the command {{kbd | key=<nowiki>curl -L -I <URL></nowiki>}}{{access | date=2018-09-20}}
** Option {{kbd | key=<nowiki>-L, --location</nowiki>}} "If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code)." quoted from [https://curl.haxx.se/docs/manpage.html manual].
** Option {{kbd | key=<nowiki>-I, --head</nowiki>}} "Fetch the headers only!" quoted from manual.
 
* [https://www.gnu.org/software/wget/ Wget - GNU Project - Free Software Foundation] e.g. Input the command {{kbd | key=<nowiki>wget -S --spider <URL></nowiki>}} <ref>[https://www.zyxware.com/articles/2402/viewing-http-headers-using-wget Viewing HTTP Headers using Wget | Zyxware Technologies]</ref>{{access | date=2018-09-20}}
** Option {{kbd | key=<nowiki>-S, --server-response</nowiki>}} "Print the headers sent by HTTP servers and responses sent by FTP servers." quoted from manual.
** Option {{kbd | key=<nowiki>--spider</nowiki>}} "When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there." quoted from manual.
 
* ''$'' [https://paw.cloud/ Paw – The most advanced API tool] for {{Mac}}
 
Example result after executed {{kbd | key=curl}} command:
<pre>
$ curl -L -I https://www.google.com
 
HTTP/2 200
date: Thu, 20 Sep 2018 02:56:29 GMT
expires: -1
cache-control: private, max-age=0
content-type: text/html; charset=ISO-8859-1
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
set-cookie: 1P_JAR=2018-09-20-02; expires=Sat, 20-Oct-2018 02:56:29 GMT; path=/; domain=.google.com
set-cookie: NID=139=DXgMIx0L06ZUBLaTUD2J_pqIvfgSEo945An0URyIwGqVf_NOxPcHcaAxhNwNforv-Lw0-m6DSKX-y1wz0EhuC-tdzLHPyWYqLVOdu7VBgjH9spnMr_2MfY79uh05aYuH; expires=Fri, 22-Mar-2019 02:56:29 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: quic=":443"; ma=2592000; v="44,43,39,35"
accept-ranges: none
vary: Accept-Encoding
</pre>
 
=== Redirect checker ===
* [https://wheregoes.com/ Link Checker | Redirect Checker - WhereGoes] {{Gd}}
 
Curl command
<pre>
curl -sI -L "https://example.com/" | grep -i "^location:"
</pre>
 
This command '''traces the redirect chain of a URL''', printing only the destination of each redirect step. Here's the breakdown:
 
<code>curl -sI -L "https://example.com/"</code>
 
* '''<code>curl</code>''': a tool for sending HTTP requests
* '''<code>-s</code>''' (silent): quiet mode, suppresses progress bars and other status info
* '''<code>-I</code>''' (capital, <code>--head</code>): fetches only the '''HTTP headers''', not the page body — faster
* '''<code>-L</code>''' (<code>--location</code>): if the server responds with a 3xx redirect status code (like 301, 302, 307), curl will '''automatically follow''' the new URL and keep requesting until it reaches the final page (or hits the redirect limit)
* '''<code>"https://example.com/"</code>''': the target URL to query
 
So this sends a HEAD request for each redirect step along the way, and prints all the response headers, which might look like:
 
<pre>
HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/
...
 
HTTP/1.1 200 OK
...
</pre>
 
<code>| grep -i "^location:"</code>
 
* '''<code>|</code>''': pipes curl's output into <code>grep</code>
* '''<code>grep -i</code>''': searches text, <code>-i</code> means case-insensitive (so it matches both <code>Location:</code> and <code>location:</code>)
* '''<code>"^location:"</code>''': <code>^</code> means '''start of line''', so it only picks lines that '''begin with''' <code>location:</code> — that is, the new destination URL specified by the redirect header
 
This command will:
# Send a request to the target URL, fetching only headers
# Automatically follow any redirects
# Filter out every <code>Location:</code> line — i.e., the destination URL of '''each''' redirect step


=== other tools ===
=== other tools ===
Line 31: Line 104:
* [http://www.httpwatch.com/ HttpWatch 9.0: HTTP Sniffer] for {{IE}}, {{Fx}} and iPhone
* [http://www.httpwatch.com/ HttpWatch 9.0: HTTP Sniffer] for {{IE}}, {{Fx}} and iPhone
** Steps to view the POST data: (1)start recording HTTP requests (2)click URL (3)switch the label to '''POST Data'''
** Steps to view the POST data: (1)start recording HTTP requests (2)click URL (3)switch the label to '''POST Data'''
echo
* [https://httpbin.org/ httpbin.org] "A simple HTTP Request & Response Service."
web security
* [https://tools.geekflare.com/tools/x-frame-options-test X-Frame-Options Header Checker Tool] {{access | date=2019-03-06}}
* [https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project OWASP Zed Attack Proxy Project - OWASP] {{access | date=2019-03-06}}
== HTTP & HTTPS Proxy ==
* [https://mitmproxy.org/ mitmproxy - an interactive HTTPS proxy] ([https://github.com/mitmproxy/mitmproxy/blob/main/LICENSE MIT license] {{Gd}}) on {{Win}}, {{Mac}} & {{Linux}}<ref>[https://docs.mitmproxy.org/stable/overview/installation/ Installation]</ref>
* [https://portswigger.net/burp Burp Suite - Application Security Testing Software - PortSwigger]
* ''$'' [https://www.charlesproxy.com/ Charles Web Debugging Proxy • HTTP Monitor / HTTP Proxy / HTTPS & SSL Proxy / Reverse Proxy] on {{Mac}}
* ''$'' [https://www.telerik.com/fiddler Fiddler - Web Debugging Proxy - Telerik] on {{Win}}, {{Mac}} & {{Linux}}
* ''$'' [https://proxyman.io/ Proxyman · Native, Modern Web Debugging Proxy · Inspect network from Mac, iOS, Android devices with ease]


== Web page compression check ==
== Web page compression check ==
Line 44: Line 131:
* [http://stackoverflow.com/questions/3390547/capturing-ajax-requests html - capturing ajax requests - Stack Overflow] {{access | date = 2015-09-01}}
* [http://stackoverflow.com/questions/3390547/capturing-ajax-requests html - capturing ajax requests - Stack Overflow] {{access | date = 2015-09-01}}


== references ==
== References ==
<references/>
<references/>


[[Category:WebDesign]]
[[Category: Design]]
[[Category:Programming]]
[[Category: Programming]]
[[Category:Data science]]
[[Category: Data collecting]]
[[Category:Data collecting]]
[[Category: Security]]
[[Category: Revised with LLMs]]

Latest revision as of 15:25, 31 July 2026

<< Testing

HTTP request and response data tool[edit]

Testing the API or pressure test.

HTTP headers generator[edit]

  • Apache Jmeter v. 2.7[1]
    • approach 1: (1) Add config element: HTTP request defaults (2) Add Sampler: HTTP request (3) Add Listener: View results tree (4) Run the test plan
    • approach 2: (1) install unofficial jmeter-plugins (2) Add Sampler: jp@gc - HTTP Raw Request[2] (3) Add Listener: View results tree (4) Run the test plan
telnet localhost 80
"header content" Enter
Enter

Display HTTP headers of a web page[edit]

  • curl e.g. Input the command curl -L -I <URL>[Last visited: 2018-09-20]
    • Option -L, --location "If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code)." quoted from manual.
    • Option -I, --head "Fetch the headers only!" quoted from manual.
  • Wget - GNU Project - Free Software Foundation e.g. Input the command wget -S --spider <URL> [3][Last visited: 2018-09-20]
    • Option -S, --server-response "Print the headers sent by HTTP servers and responses sent by FTP servers." quoted from manual.
    • Option --spider "When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there." quoted from manual.

Example result after executed curl command: 

$ curl -L -I https://www.google.com

HTTP/2 200
date: Thu, 20 Sep 2018 02:56:29 GMT
expires: -1
cache-control: private, max-age=0
content-type: text/html; charset=ISO-8859-1
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
set-cookie: 1P_JAR=2018-09-20-02; expires=Sat, 20-Oct-2018 02:56:29 GMT; path=/; domain=.google.com
set-cookie: NID=139=DXgMIx0L06ZUBLaTUD2J_pqIvfgSEo945An0URyIwGqVf_NOxPcHcaAxhNwNforv-Lw0-m6DSKX-y1wz0EhuC-tdzLHPyWYqLVOdu7VBgjH9spnMr_2MfY79uh05aYuH; expires=Fri, 22-Mar-2019 02:56:29 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: quic=":443"; ma=2592000; v="44,43,39,35"
accept-ranges: none
vary: Accept-Encoding

Redirect checker[edit]

Curl command 

curl -sI -L "https://example.com/" | grep -i "^location:"

This command traces the redirect chain of a URL, printing only the destination of each redirect step. Here's the breakdown:

curl -sI -L "https://example.com/"

  • curl: a tool for sending HTTP requests
  • -s (silent): quiet mode, suppresses progress bars and other status info
  • -I (capital, --head): fetches only the HTTP headers, not the page body — faster
  • -L (--location): if the server responds with a 3xx redirect status code (like 301, 302, 307), curl will automatically follow the new URL and keep requesting until it reaches the final page (or hits the redirect limit)
  • "https://example.com/": the target URL to query

So this sends a HEAD request for each redirect step along the way, and prints all the response headers, which might look like: 

HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/
...

HTTP/1.1 200 OK
...

| grep -i "^location:"

  • |: pipes curl's output into grep
  • grep -i: searches text, -i means case-insensitive (so it matches both Location: and location:)
  • "^location:": ^ means start of line, so it only picks lines that begin with location: — that is, the new destination URL specified by the redirect header

This command will:

  1. Send a request to the target URL, fetching only headers
  2. Automatically follow any redirects
  3. Filter out every Location: line — i.e., the destination URL of each redirect step

other tools[edit]

echo

  • httpbin.org "A simple HTTP Request & Response Service."

web security

HTTP & HTTPS Proxy[edit]

Web page compression check[edit]

online gzip test


related article[edit]

References[edit]

  1. ↑ JMeter proxy Step-by-step
  2. ↑ RawRequest - jmeter-plugins - Every load test needs some sexy features! - Google Project Hosting
  3. ↑ Viewing HTTP Headers using Wget | Zyxware Technologies
  4. ↑ Installation
Retrieved from "https://wiki.planetoid.info/index.php?title=HTTP_request_and_response_data_tool&oldid=26421"