Troubleshooting of curl errors
cUrl "is used in command lines or scripts to transfer data."[1] (alternative library to achive sampe purpose: (1) PHP naive Client URL Library (2) cURL class for PHP (3) Installing cURL on Cygwin on Windows)
- How to debug the cUrl command: (1) Enable the verbose option e.g. "... run the command with the -v / --verbose option to get more information." or (2) Enable the trace options e.g. --trace-ascii [filename] Note: trace options will be override by verbose option if enable both options.
How to resolve Command Execution: Works in DOS/Terminal but Not in PHP[edit]
Often, commands that run perfectly when manually executed in DOS or the Windows terminal (like ConEmu) might not produce the expected outcome when run through PHP. A common issue is the command not generating the anticipated file.
Example cURL command for Windows terminal
curl "https://www.example.com/path/to/resource/" ^ -H "authority: www.example.com" ^ -H "accept: text/html" ^ --data-raw "PARAMS" --compressed > output.html
Solution
- Single Continuous String in PHP: In PHP, when executing a shell command, avoid using the caret (^) for multi-line commands[1], which is typical in the Command Prompt. Instead, ensure the command is one continuous string.
- Unsupported curl Options: If you encounter an error like "curl: option --compressed: the installed libcurl version doesn't support this", it's a clear indication to remove or adjust the problematic option in the curl command.
Revised cURL command for PHP
curl "https://www.example.com/path/to/resource/" -H "authority: www.example.com" -H "accept: text/html" --data-raw "PARAMS" > output.html
How to resolve cUrl error (#5): Unsupported proxy syntax[edit]
Condition: the username of socks5 proxy contains @ symbol such as [email protected]
curl_setopt($ch, CURLOPT_PROXY, 'socks5://username:password@localhost:12345'); // result is ok curl_setopt($ch, CURLOPT_PROXY, 'socks5://[email protected]:password@localhost:12345'); // met cUrl error (#5): Unsupported proxy syntax
Solution: Escape the special symbol[2] using URL Encode and Decode - Online or urlencode function
curl_setopt($ch, CURLOPT_PROXY, 'socks5://bob%40email.com:password@localhost:12345'); // result is ok
How to resolve cUrl error (#56): OpenSSL SSL_read: No error.[edit]
Condition: The curl vesion 7.67.0 on windows met problem. Result of curl_version returned:
version: Array ( [version_number] => 475904 [age] => 5 [features] => 2953117 [ssl_version_number] => 0 [version] => 7.67.0 [host] => x86_64-pc-win32 [ssl_version] => OpenSSL/1.1.1d [libz_version] => 1.2.11 )
Solution: Use other version of curl command or curl on Cygwin or cURL class for PHP.
Related articles
How to resolve the error: error code: 1010[edit]
Identifying the Issue
- While attempting to download a webpage content via the cURL command, I encountered an issue where the output displayed was "error code: 1010". To delve deeper, I reran the cURL command with verbose mode activated (using the -v flag). The headers revealed in this mode included:
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=xxx"}],"group":"cf-nel","max_age":604800} < nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800} < server: cloudflare
Proposed Solution
- Instead of relying on the cURL command, switching to a JavaScript-based crawler is recommended as a solution to this problem [3].
How to resolve Error 405 Method Not Allowed[edit]
When I tried to crawl the web resource but met the error message: "Error 405 Method Not Allowed"
Solution: Make sure using the correct HTTP request method[4] e.g. GET or POST to access the web resource.
How to resolve Error 415 Unsupported Media Type[edit]
When I tried to POST JSON (as payload) but met the error message: "Error 415 Unsupported Media Type"
Solution: Set the cUrl with the header Content-Type: application/json[5] Example syntax of cUrl command as following:
curl -X POST --header "Content-Type: application/json" --data '{"user" : "your_user_name", "token" : "your_token"}' "https://example.com/"
How to rseolve cURL returns binary data instead of html or json[edit]
How to resolve the downloaded file was empty[edit]
Error condition
- When I executed the Curl command curl -O https://path/to/plain_text.txt, but the downloaded file was empty.
Solution
- After I enabled verbose option, I found the HTTP status code HTTP/1.1 302 Moved Temporarily. I added the location (-L) parameter to resolve the downloaded file was empty: curl -L -O https://path/to/plain_text.txt,
The truncated result after executed the command: curl -v -O https://path/to/plain_text.txt < HTTP/1.1 302 Moved Temporarily < Server: nginx < Date: Tue, 25 Aug 2020 03:45:13 GMT < Content-Type: text/html < Transfer-Encoding: chunked < Connection: keep-alive < Keep-Alive: timeout=60 < Location: /path/to/plain_text.txt
How to resolve the error: no matches found[edit]
Error condition
- When I executed the Curl command curl http://website.test/http_status.php?case=500, but I met the error message: "zsh: no matches found: http://website.test/http_status.php?case=500"
Solution[6]
- Double quote the URL e.g. curl "http://website.test/http_status.php?case=500"
How to resolve the error: Remote file name has no length[edit]
Error message
curl -O "https://website.com/?p=123" curl: Remote file name has no length curl: (23) Failed writing received data to disk/application
Solution
curl "https://website.com/?p=123" -o myfile.html
Explanation of solution
- When you encounter an error while downloading a file with `curl`, it may be because the URL does not include a file name, causing `curl` to be unable to determine the name to save the file under. The solution is to manually specify a file name using the `-o` option, for example, `curl -o myfile.html "https://website.com/?p=123"`. Additionally, make sure you have enough disk space and are in a directory where you have write permissions.
How to resolve the error: zsh: parse error near `&'[edit]
Error condition
% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json" -d'{"url": "https://someweb.com/Today's_Biggest_Trends"}' dquote>
Solution: Escape the single quote ' became '' (repeat the single quote twice)
% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json" -d'{"url": "https://someweb.com/Today''s_Biggest_Trends"}'
Trivial Solution NOT WORK: single quote ' became \'
% curl -XPOST http://localhost:3000/ -H "Content-Type: application/json" -d'{"url": "https://someweb.com/Today\'s_Biggest_Trends"}' dquote>
Further reading[edit]
References[edit]
- ↑ Split long commands in multiple lines through Windows batch file - Stack Overflow
- ↑ Log in to FTP using Windows Explorer with @ in the username - Super User
- ↑ Cloudflare Error 1010: What Is It and How to Avoid - ZenRows
- ↑ HTTP/1.1: Method Definitions
- ↑ PHP cURL POST returns a 415 - Unsupported Media Type - Stack Overflow
- ↑ 為什麼 Mac terminal curl 指令的網址通常要加雙引號 - 彼得潘的 Swift iOS App 開發問題解答集 - Medium
Troubleshooting of ...
- PHP, cUrl, Python, selenium, HTTP status code errors
- Database: SQL syntax debug, MySQL errors, MySQLTuner errors or PostgreSQL errors
- HTML/Javascript: Troubleshooting of javascript, XPath
- Software: Mediawiki, Docker, FTP problems, online conference software
- Test connectivity for the web service, Web Ping, Network problem, Web user behavior, Web scrape troubleshooting
Template