Stealthy website scanning thanks to archive.org

Mar 6, 2020 by Thibault Debatty | 1962 views

Offensive Security

https://cylab.be/blog/62/stealthy-website-scanning-thanks-to-archiveorg

Scanning a website is an important step of the reconnaissance phase. Different tools, like BlackWidow, can automate the process. We present here another tool that allows to scan a website without leaving traces on the target servers : waybackurls.

To achieve this, waybackurls actually queries the wayback machine from Internet Archive. This project keeps backups of over 418 billion web pages and offers a nice REST API.

For example, this query allows to list known URL's from cylab.be :

http://web.archive.org/cdx/search/cdx?url=cylab.be/*&output=json
  &fl=original&collapse=urlkey

Using waybackurls with Docker

The easiest way to use waybackurls is with docker:

$ docker pull cylab/waybackurls

You can then run waybackurls by feeding a list of domains on stdin :

$ echo "cylab.be" | docker run -i cylab/waybackurls

https://cylab.be/
https://cylab.be/css/app.css
http://cylab.be/favicon.ico
https://cylab.be/fonts/et-line.eot?26ec3c7d0366e0825d705c6e22
https://cylab.be/fonts/et-line.eot?26ec3c7d0366e0825d705c6e22?
https://cylab.be/fonts/et-line.svg?569bd9082c15cc30fa6e05626a
https://cylab.be/fonts/et-line.ttf?98126e3e1238b0f3b941ad285320
https://cylab.be/fonts/et-line.woff?b01ff252761958325faab1535c9

Manual installation and usage

Waybackurls is actually written in GO. So here is the way to install it manually and run it from your host machine:

$ go get github.com/tomnomnom/waybackurls
$ echo "cylab.be" | ./go/bin/waybackurls

This blog post is licensed under CC BY-SA 4.0

How to detect filtered (and opened) outgoing ports on a network?
Sometimes you want to access services running on unusual ports, like a SSH server running on port 2222 for example. If connection fails, how can we detect the outgoing ports that are filtered or open on the network?
May 16, 2023 | 1640 views
Adv-Bot: Realistic Adversarial Botnet Attacks against Network Intrusion Detection Systems
Due to the numerous advantages of machine learning (ML) algorithms, many applications now incorporate them. However, many studies in the field of image classification have shown that MLs can be fooled by a variety of adversarial attacks. This raises many questions in the cybersecurity field, where a growing number of researchers are recently investigating the feasibility of such attacks against machine learning-based security systems, such as intrusion detection systems.
Mar 28, 2023 | 1100 views
SQLMap : additional techniques
In a previous blog post, we have explained what SQL injection is, and how to exploit it using sqlmap. In this blog post, we will show some additional techniques: how to exploit web applications that use clean URLs, how to exploit a POSTed form, how to hide traces etc.
Mar 21, 2023 | 1928 views
This website uses cookies. More information about the use of cookies is available in the cookies policy.
Accept