Troubleshooting of curl errors

From LemonWiki共筆
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

cUrl "is used in command lines or scripts to transfer data."[1] (alternative library to achive sampe purpose: (1) PHP naive Client URL Library (2) cURL class for PHP (3) Installing cURL on Cygwin on Windows)

🌐 Switch language: EN, 漢字


Owl icon.jpg How to debug the cUrl command: (1) Enable the verbose option e.g. "... run the command with the -v / --verbose option to get more information." or (2) Enable the trace options e.g. --trace-ascii [filename] Note: trace options will be override by verbose option if enable both options.

How to resolve Command Execution: Works in DOS/Terminal but Not in PHP

Often, commands that run perfectly when manually executed in DOS or the Windows terminal (like ConEmu) might not produce the expected outcome when run through PHP. A common issue is the command not generating the anticipated file.

Example cURL command for Windows terminal

curl "https://www.example.com/path/to/resource/" ^
-H "authority: www.example.com" ^
-H "accept: text/html" ^
--data-raw "PARAMS" --compressed > output.html

Solution

  • Single Continuous String in PHP: In PHP, when executing a shell command, avoid using the caret (^) for multi-line commands[1], which is typical in the Command Prompt. Instead, ensure the command is one continuous string.
  • Unsupported curl Options: If you encounter an error like "curl: option --compressed: the installed libcurl version doesn't support this", it's a clear indication to remove or adjust the problematic option in the curl command.

Revised cURL command for PHP

curl "https://www.example.com/path/to/resource/" -H "authority: www.example.com" -H "accept: text/html" --data-raw "PARAMS" > output.html

How to resolve cUrl error (#5): Unsupported proxy syntax

Condition: the username of socks5 proxy contains @ symbol such as [email protected]

curl_setopt($ch, CURLOPT_PROXY, 'socks5://username:password@localhost:12345');
// result is ok

curl_setopt($ch, CURLOPT_PROXY, 'socks5://[email protected]:password@localhost:12345');
// met cUrl error (#5): Unsupported proxy syntax

Solution: Escape the special symbol[2] using URL Encode and Decode - Online or urlencode function

curl_setopt($ch, CURLOPT_PROXY, 'socks5://bob%40email.com:password@localhost:12345');
// result is ok

How to resolve cUrl error (#56): OpenSSL SSL_read: No error.

Condition: The curl vesion 7.67.0 on windows met problem. Result of curl_version returned:

version: Array
(
    [version_number] => 475904
    [age] => 5
    [features] => 2953117
    [ssl_version_number] => 0
    [version] => 7.67.0
    [host] => x86_64-pc-win32
    [ssl_version] => OpenSSL/1.1.1d
    [libz_version] => 1.2.11
)

Solution: Use other version of curl command or curl on Cygwin or cURL class for PHP.

Related articles

How to resolve the error: error code: 1010

Identifying the Issue

  • While attempting to download a webpage content via the cURL command, I encountered an issue where the output displayed was "error code: 1010". To delve deeper, I reran the cURL command with verbose mode activated (using the -v flag). The headers revealed in this mode included:
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=xxx"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare

Proposed Solution

  • Instead of relying on the cURL command, switching to a JavaScript-based crawler is recommended as a solution to this problem [3].

How to resolve Error 405 Method Not Allowed

When I tried to crawl the web resource but met the error message: "Error 405 Method Not Allowed"

Solution: Make sure using the correct HTTP request method[4] e.g. GET or POST to access the web resource.

How to resolve Error 415 Unsupported Media Type

When I tried to POST JSON (as payload) but met the error message: "Error 415 Unsupported Media Type"

Solution: Set the cUrl with the header Content-Type: application/json[5] Example syntax of cUrl command as following:

curl -X POST --header "Content-Type: application/json" --data '{"user" : "your_user_name", "token" : "your_token"}' "https://example.com/"

How to rseolve cURL returns binary data instead of html or json

How to resolve the downloaded file was empty

Error condition

  • When I executed the Curl command curl -O https://path/to/plain_text.txt, but the downloaded file was empty.

Solution

  • After I enabled verbose option, I found the HTTP status code HTTP/1.1 302 Moved Temporarily. I added the location (-L) parameter to resolve the downloaded file was empty: curl -L -O https://path/to/plain_text.txt,
The truncated result after executed the command: curl -v -O https://path/to/plain_text.txt

< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Tue, 25 Aug 2020 03:45:13 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Keep-Alive: timeout=60
< Location: /path/to/plain_text.txt

How to resolve the error: no matches found

Error condition

  • When I executed the Curl command curl http://website.test/http_status.php?case=500, but I met the error message: "zsh: no matches found: http://website.test/http_status.php?case=500"

Solution[6]

  • Double quote the URL e.g. curl "http://website.test/http_status.php?case=500"

How to resolve the error: Remote file name has no length

Error message

curl -O "https://website.com/?p=123"
curl: Remote file name has no length
curl: (23) Failed writing received data to disk/application

Solution

curl "https://website.com/?p=123" -o myfile.html 

Explanation of solution

  • When you encounter an error while downloading a file with `curl`, it may be because the URL does not include a file name, causing `curl` to be unable to determine the name to save the file under. The solution is to manually specify a file name using the `-o` option, for example, `curl -o myfile.html "https://website.com/?p=123"`. Additionally, make sure you have enough disk space and are in a directory where you have write permissions.

How to resolve the error: zsh: parse error near `&'

Error condition

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today's_Biggest_Trends"}'
dquote>

Solution: Escape the single quote ' became '' (repeat the single quote twice)

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today''s_Biggest_Trends"}'

Trivial Solution Icon_exclaim.gif NOT WORK: single quote ' became \'

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today\'s_Biggest_Trends"}'
dquote>

Further reading

References


Troubleshooting of ...

Template