Troubleshooting of curl errors

cUrl "is used in command lines or scripts to transfer data."[1] (alternative library to achive sampe purpose: (1) PHP naive Client URL Library (2) cURL class for PHP (3) Installing cURL on Cygwin on Windows)

🌐 Switch language: EN, 漢字

How to debug the cUrl command: (1) Enable the verbose option e.g. "... run the command with the -v / --verbose option to get more information." or (2) Enable the trace options e.g. --trace-ascii [filename] Note: trace options will be override by verbose option if enable both options.

How to resolve Command Execution: Works in DOS/Terminal but Not in PHP

Often, commands that run perfectly when manually executed in DOS or the Windows terminal (like ConEmu) might not produce the expected outcome when run through PHP. A common issue is the command not generating the anticipated file.

Example cURL command for Windows terminal

curl "https://www.example.com/path/to/resource/" ^
-H "authority: www.example.com" ^
-H "accept: text/html" ^
--data-raw "PARAMS" --compressed > output.html

Solution

• Single Continuous String in PHP: In PHP, when executing a shell command, avoid using the caret (^) for multi-line commands, which is typical in the Command Prompt. Instead, ensure the command is one continuous string.

• Unsupported curl Options: If you encounter an error like "curl: option --compressed: the installed libcurl version doesn't support this", it's a clear indication to remove or adjust the problematic option in the curl command.

Revised cURL command for PHP

curl "https://www.example.com/path/to/resource/" -H "authority: www.example.com" -H "accept: text/html" --data-raw "PARAMS" > output.html

How to resolve cUrl error (#5): Unsupported proxy syntax

Condition: the username of socks5 proxy contains @ symbol such as [email protected]

curl_setopt($ch, CURLOPT_PROXY, 'socks5://username:password@localhost:12345');
// result is ok

curl_setopt($ch, CURLOPT_PROXY, 'socks5://[email protected]:password@localhost:12345');
// met cUrl error (#5): Unsupported proxy syntax

Solution: Escape the special symbol using URL Encode and Decode - Online or urlencode function

curl_setopt($ch, CURLOPT_PROXY, 'socks5://bob%40email.com:password@localhost:12345');
// result is ok

How to resolve cUrl error (#56): OpenSSL SSL_read: No error.

Condition: The curl vesion 7.67.0 on windows met problem. Result of curl_version returned:

version: Array
(
    [version_number] => 475904
    [age] => 5
    [features] => 2953117
    [ssl_version_number] => 0
    [version] => 7.67.0
    [host] => x86_64-pc-win32
    [ssl_version] => OpenSSL/1.1.1d
    [libz_version] => 1.2.11
)

Solution: Use other version of curl command or curl on Cygwin or cURL class for PHP.

Related articles

• cURL error 56: OpenSSL SSL_read: No error information · Issue #4982 · curl/curl

How to resolve Error 405 Method Not Allowed

When I tried to crawl the web resource but met the error message: "Error 405 Method Not Allowed"

Solution: Make sure using the correct HTTP request method e.g. GET or POST to access the web resource.

How to resolve Error 415 Unsupported Media Type

When I tried to POST JSON (as payload) but met the error message: "Error 415 Unsupported Media Type"

Solution: Set the cUrl with the header Content-Type: application/json Example syntax of cUrl command as following:

curl -X POST --header "Content-Type: application/json" --data '{"user" : "your_user_name", "token" : "your_token"}' "https://example.com/"

How to rseolve cURL returns binary data instead of html or json

• php - cURL returns binary data instead of html - Stack Overflow
• 解決 cUrl 下載的網頁或 JSON 看起來像是亂碼的問題

How to resolve the downloaded file was empty

Error condition

• When I executed the Curl command curl -O https://path/to/plain_text.txt, but the downloaded file was empty.

Solution

• After I enabled verbose option, I found the HTTP status code HTTP/1.1 302 Moved Temporarily. I added the location (-L) parameter to resolve the downloaded file was empty: curl -L -O https://path/to/plain_text.txt,

The truncated result after executed the command: curl -v -O https://path/to/plain_text.txt

< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Tue, 25 Aug 2020 03:45:13 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Keep-Alive: timeout=60
< Location: /path/to/plain_text.txt

How to resolve the error: no matches found

Error condition

• When I executed the Curl command curl http://website.test/http_status.php?case=500, but I met the error message: "zsh: no matches found: http://website.test/http_status.php?case=500"

Solution

• Double quote the URL e.g. curl "http://website.test/http_status.php?case=500"

How to resolve the error: Remote file name has no length

Error message

curl -O "https://website.com/?p=123"
curl: Remote file name has no length
curl: (23) Failed writing received data to disk/application

Solution

curl "https://website.com/?p=123" -o myfile.html 

Explanation of solution

• When you encounter an error while downloading a file with `curl`, it may be because the URL does not include a file name, causing `curl` to be unable to determine the name to save the file under. The solution is to manually specify a file name using the `-o` option, for example, `curl -o myfile.html "https://website.com/?p=123"`. Additionally, make sure you have enough disk space and are in a directory where you have write permissions.

How to resolve the error: zsh: parse error near `&'

Error condition

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today's_Biggest_Trends"}'
dquote>

Solution: Escape the single quote ' became '' (repeat the single quote twice)

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today''s_Biggest_Trends"}'

Trivial Solution NOT WORK: single quote ' became \'

% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json"  -d'{"url": "https://someweb.com/Today\'s_Biggest_Trends"}'
dquote>

Further reading

• Introduction - Everything curl
• curl - How To Use
• bash - CURL escape single quote - Stack Overflow

References

Troubleshooting of ...

• Troubleshooting of Excel errors

• PHP, cUrl, Python, selenium, HTTP status code errors

• Database: SQL syntax debug, MySQL errors, MySQLTuner errors or PostgreSQL errors

• Troubleshooting of regular expression

• HTML/Javascript: Troubleshooting of javascript, XPath

• Software: Mediawiki, Docker, FTP problems, online conference software

• Test connectivity for the web service, Web Ping, Network problem, Web user behavior, Web scrape troubleshooting

Template

• Bug report template