Intercepting and Collecting Web Evidence in the Times of TLS1.3 and HTTP3.0

web scraping, TLS/SSL, MitM, HTTP

The end-to-end HTTPS encryption and the volatile nature of web content make any interception and collection of data on the Internet a challenge. The presentation introduces methods addressing both of these phenomena intercepting TLS/SSL connections with the help of man-in-the-middle attack employing proxy and automatically creating snapshots of problematic web pages.  Speakers outline necessary theory (including news about TLS 1.3, HSTS, HTTP3.0), well-known attacks (e.g., renegotiation, downgrade, cipherspec change,  and others), and industry-standard tools for traffic analysis (such as Wireshark, Fiddler proxy, SSL-Split) and decoding (e.g., Selenium, Scrapy). The session will include a live demo of MitM attack on HTTPS connection enhanced with covert extraction of form data, which would be later used to periodically web scrape and archive protected content.