Troubleshooting of curl errors

From LemonWiki共筆
Jump to navigation Jump to search

cUrl "is used in command lines or scripts to transfer data."[1] (alternative library to achive sampe purpose: (1) PHP naive Client URL Library (2) cURL class for PHP (3) Installing cURL on Cygwin on Windows)

🌐 Switch language: EN, 漢字


Owl icon.jpg How to debug the cUrl command: (1) Enable the verbose option e.g. "... run the command with the -v / --verbose option to get more information." or (2) Enable the trace options e.g. --trace-ascii [filename] Note: trace options will be override by verbose option if enable both options.

How to resolve Command Execution: Works in DOS/Terminal but Not in PHP[edit]

Often, commands that run perfectly when manually executed in DOS or the Windows terminal (like ConEmu) might not produce the expected outcome when run through PHP. A common issue is the command not generating the anticipated file.

Example cURL command for Windows terminal

curl "https://www.example.com/path/to/resource/" ^
-H "authority: www.example.com" ^
-H "accept: text/html" ^
--data-raw "PARAMS" --compressed > output.html

Solution

  • Single Continuous String in PHP: In PHP, when executing a shell command, avoid using the caret (^) for multi-line commands[1], which is typical in the Command Prompt. Instead, ensure the command is one continuous string.
  • Unsupported curl Options: If you encounter an error like "curl: option --compressed: the installed libcurl version doesn't support this", it's a clear indication to remove or adjust the problematic option in the curl command.

Revised cURL command for PHP

curl "https://www.example.com/path/to/resource/" -H "authority: www.example.com" -H "accept: text/html" --data-raw "PARAMS" > output.html

How to resolve cUrl error (#5): Unsupported proxy syntax[edit]

Condition: the username of socks5 proxy contains @ symbol such as [email protected]

curl_setopt($ch, CURLOPT_PROXY, 'socks5://username:password@localhost:12345');
// result is ok

curl_setopt($ch, CURLOPT_PROXY, 'socks5://[email protected]:password@localhost:12345');
// met cUrl error (#5): Unsupported proxy syntax

Solution: Escape the special symbol[2] using URL Encode and Decode - Online or urlencode function

curl_setopt($ch, CURLOPT_PROXY, 'socks5://bob%40email.com:password@localhost:12345');
// result is ok

How to resolve cUrl error (#56): OpenSSL SSL_read: No error.[edit]

Condition: The curl vesion 7.67.0 on windows met problem. Result of curl_version returned:

version: Array
(
    [version_number] => 475904
    [age] => 5
    [features] => 2953117
    [ssl_version_number] => 0
    [version] => 7.67.0
    [host] => x86_64-pc-win32
    [ssl_version] => OpenSSL/1.1.1d
    [libz_version] => 1.2.11
)

Solution: Use other version of curl command or curl on Cygwin or cURL class for PHP.

Related articles

How to resolve the error: error code: 1010[edit]

Identifying the Issue

  • While attempting to download a webpage content via the cURL command, I encountered an issue where the output displayed was "error code: 1010". To delve deeper, I reran the cURL command with verbose mode activated (using the -v flag). The headers revealed in this mode included:
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=xxx"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare

Proposed Solution

  • Instead of relying on the cURL command, switching to a JavaScript-based crawler is recommended as a solution to this problem [3].

How to resolve Error 405 Method Not Allowed[edit]

When I tried to crawl the web resource but met the error message: "Error 405 Method Not Allowed"

Solution: Make sure using the correct HTTP request method[4] e.g. GET or POST to access the web resource.

How to resolve Error 415 Unsupported Media Type[edit]

When I tried to POST JSON (as payload) but met the error message: "Error 415 Unsupported Media Type"

Solution: Set the cUrl with the header Content-Type: application/json[5] Example syntax of cUrl command as following:

curl -X POST --header "Content-Type: application/json" --data '{"user" : "your_user_name", "token" : "your_token"}' "https://example.com/"

How to rseolve cURL returns binary data instead of html or json[edit]

How to resolve the downloaded file was empty[edit]

Error condition

  • When I executed the Curl command curl -O https://path/to/plain_text.txt, but the downloaded file was empty.

Solution

  • After I enabled verbose option, I found the HTTP status code HTTP/1.1 302 Moved Temporarily. I added the location (-L) parameter to resolve the downloaded file was empty: curl -L -O https://path/to/plain_text.txt,
The truncated result after executed the command: curl -v -O https://path/to/plain_text.txt

< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Tue, 25 Aug 2020 03:45:13 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Keep-Alive: timeout=60
< Location: /path/to/plain_text.txt

How to resolve the error: no matches found[edit]

Error condition

  • When I executed the Curl command curl http://website.test/http_status.php?case=500, but I met the error message: "zsh: no matches found: http://website.test/http_status.php?case=500"

Solution[6]

  • Double quote the URL e.g. curl "http://website.test/http_status.php?case=500"

How to resolve the error: Remote file name has no length[edit]

Error message

curl -O "https://website.com/?p=123"
curl: Remote file name has no length
curl: (23) Failed writing received data to disk/application

Solution

curl "https://website.com/?p=123" -o myfile.html 

Explanation of solution

  • When you encounter an error while downloading a file with `curl`, it may be because the URL does not include a file name, causing `curl` to be unable to determine the name to save the file under. The solution is to manually specify a file name using the `-o` option, for example, `curl -o myfile.html "https://website.com/?p=123"`. Additionally, make sure you have enough disk space and are in a directory where you have write permissions.

How to resolve the error: zsh: parse error near `&'[edit]

Error condition

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today's_Biggest_Trends"}'
dquote>

Solution: Escape the single quote ' became '' (repeat the single quote twice)

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today''s_Biggest_Trends"}'

Trivial Solution Icon_exclaim.gif NOT WORK: single quote ' became \'

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today\'s_Biggest_Trends"}'
dquote>

Further reading[edit]

References[edit]


Troubleshooting of ...

Template