OSINT With Buscador

OSINT stands for Open Source Intelligence and is the practice of scraping the internet for publicly available information. This information, when examined individually, may seem relatively innocuous but when you gather enough individual pieces and can consider them together you can often build a eerily clear picture of someone or something.

As an example, if you have a Facebook account chances are your birthday is listed somewhere on it. By itself, this doesn’t seem like a big deal. We give our birthday out all the time to verify our age. Besides, all you may use Facebook for is exchanging dank memes with your best friends and the occasional family photo that your mom tagged you in.

Individually, none of this seems that important but when taken altogether it starts to sound a lot like some very common security questions that you have to answer to recover lost account passwords for emails, banks or your iCloud. Collecting this information could allow a malicious actor to take control over these accounts.

Buscador is a pre-built Linux virtual machine that can be downloaded from https://inteltechniques.com/buscador/ that comes pre-configured with dozens of tools to make gathering and correlating information on people and companies easy. While it sounds scary, and it is, it’s important to realize that there are plenty of legitimate uses for performing this kind of information gathering like finding and scrubbing your own personal data to secure your own identity, or for investigative journalists digging to bring light to the next breaking story.

Setting Up

I will be using Oracle’s VirtualBox as my hypervisor, so I downloaded the Buscador .OVA file that corresponds to VirtualBox:

Once the file is downloaded, open up VirtualBox, select the File menu and click on Import Appliance In the window that pops up, click on the little folder icon next to the input box and navigate to the .ova file you just downloaded:

Go ahead and click next then Import to import the VM. We will adjust the system settings after the import rather than tweak them here. Once the import is complete, you should see the machine on the left panel of the VirtualBox dashboard. Select the new machine, then from the Machine menu at the top open the Settings On the General tab, you can rename the machine to whatever suits you:

Next, select the Advanced tab in the General options and set Shared Clipboard to bidirectional:

Under the System option’s Motherboard tab, it is recommended you set the Base Memory to half your available total, although I am limiting mine to 8GB:

Next, navigate to the Display options and set Video Memory to 128 MB if available:

Finally, you’ll want to switch to the Shared Folders menu and choose a folder on your host system to share to this machine and select auto-mount:

With this final setting change, close the settings window and go ahead and start the VM. Upon boot, you will log in to the osint user with the password osint. Once at the desktop, go under the Devices VirutalBox menu at the top and select Insert Guest Additions CD. Allow the guest additions to install, reboot the machine and log back in:

If you have the same experience that I did with the guest additions installation, the installation will fail with an unknown error. You can download the VirtualBox Guest Additions .iso file directly from the website, being sure to grab the one for your version of VirtualBox. In my case I am running v6. When you download the file, select the option to automatically open it and allow it to auto-run upon completion. It should prompt you for a the password for your account, which is osint, and then successfully install from there. After it finishes, reboot again:

If all went well, we should be able to enter full screen display, utilize our shared folder and copy files and text in and out of our machine. The last thing we need to do to setup is to update the core system, and the OSINT scripts specific to Buscador. To update the core system, click on Activities in the top left, and search for package updater:

Download and install all updates available though Package Updater:

Afterwards, open a terminal by clicking on the black square icon on the dock to the left of your desktop. If prompted to update say yes:

At the terminal prompt, type update_scripts. NOTE: As of this writing, the repository that this script pulls seems to be missing so this script is broken. As I have come across tools that don’t work, I have been able to manually update each to get them working and this will be shown when we talk about those tools.

With our setup out of the way, we want to completely shutdown the machine and create a snapshot so we can easily restore to a fresh install. This is important for a few reasons. Firstly, a lot of the tools we will use can generate a large amount of files and data. This can quickly fill up a virtual drive, so it’s a good idea to start every investigation with a fresh system. Secondly, and possibly most important, if you are using Buscador to perform actual legal investigations that will result in submitting findings as evidence in court, it is extremely important to maintain forensic integrity meaning you don’t want any residual data from unrelated investigations that could call the integrity of your evidence in to question.

Once shut down, select the hamburger menu to the right of your Buscador VM and switch to the snapshot menu. Select Take at the top, give your snapshot an intuitive name and save it:

That concludes the initial setup of Buscador. It is a good idea to perform the updates each time you start the machine from the fresh snapshot and create a newer updated snapshot before starting an investigation. This way you will always be working with the latest packages and features.

Video Downloader

Video Downloader is a utility that takes in a URL that contains a video, and will download it locally for further assessment. Like most of the functionality in Buscador, underneath the useful GUI provided by this custom virtual machine lies an elaborate python back-end. The official site for this python utility is https://ytdl-org.github.io/youtube-dl/download.html and contains useful documentation on the script itself. To launch Video Downloader, locate and click the red rectangular icon with a white play button in the dock:

Upon launching, you will be presented with a simple window asking for the URL of the page with the video or videos you would like to download. Paste your URL in and click OK. You will be greeted by an animated progress window and then the file explorer will automatically open to the default download directory (/home/osint/Video_Utilities/Youtube-DL/):

When I tested this with the URL in the picture, it appeared to progress normally but when the directory popped open it was empty. Unfortunately with update_scripts being broken, it seems like the current version doesn’t work anymore. I was able to fix this by manually updating the youtube-dl application by opening the terminal and issuing: sudo pip install --upgrade youtube-dl :

With the update complete, launch Video Downloader again and paste your URL in. Afterwards, you should see a new file in the file explorer that pops up:

As a final note for this section, this utility will download all videos found in the URL that you submit. While we pasted a URL in that corresponded to a single YouTube video, had we submitted a URL to,say, https://www.youtube.com/user/forevertrollin421/videos it would download every video that the highly trained unprofessional John Maxwell, aka The Harley Tech, has uploaded to his YouTube channel. This can be really useful to grab a lot of videos with one easy command, but be aware that it can eat up a lot of time and disc space.

As an aside: If you prefer 2 wheels over 4, check out his channel I just linked and find him on Instagram and Facebook because he puts out some great content that constantly makes me want to abandon computers altogether and switch careers.

Video Utilities

This menu option available in the dock is an easy to use interface to a powerful video utility called FFmpeg. You can open it by clicking on the blue bubble with the white play icon on it:

This tool can play pretty much any video format, from the usual formats to corrupted files to security camera footage that has no associated video player. When you click on the icon in the dock, you will be asked to select a video file to open. Navigate to whatever video file you are interested in and click OK. Next, FFmpeg needs to know what you want to do with this file and it can do a LOT:

Play a video is relatively self explanatory, so all I will say is it will try to use every codec it knows to open and run the file. It’s probably safe to say that if FFmpeg can’t play it, it can’t be played.

The second option listed is Convert a video to mp4. Since FFmpeg can play files that can’t always be played elsewhere, this allows you to convert it to a format that is commonly playable so you can distribute it to your stakeholders, or transfer it to one of your other systems where this tool isn’t available. Note, I am not a lawyer and I have no law enforcement experience but if I had to guess I would imagine that the converted file is probably not forensically sound since it’s technically altered. Use your best judgement if you are working in that domain.

The next option, Extract video frames, will process the video and open a directory with each frame of the video exported as a still image in .bmp format. This is good for gathering important snapshots from a video to submit as evidence or include in reports. By default, the stills will output to /home/Video_Utilities/ffmpeg_out/<current_date_time>-frames/:

As you can see in the screenshots, it creates a lot of images taking up a lot of space. The video used in this example was 76 MB in size and the file output was 13.2 GB. This is why we take snapshots of this machine and start with a fresh instance for every investigation.

The next two options that FFmpeg presents us are very similar. Shorten video (low activity) and Shorten video (high activity) both do exactly what it sounds like. They will analyze the video and remove frames where there is nothing interesting happening. This can be good for long surveillance videos that need to be reviewed. The difference is in the tolerance used for determining what is interesting. the low activity version has a strict tolerance, and will keep sections that have pretty much any movement at all, where the high activity version is better suited for videos with a good deal of background movement like outdoor security cameras that might capture birds or pedestrians off in the distance.

Finally, the last option allows you to extract just the audio of the provided file out into an mp3 file. This can be situationally useful as you can imagine.

Metagoofil

Metagoofil is a command-line utility that will take a domain that you provide and go out to collect all kinds of documents found related to the site. To launch it, click on the Domains Interact option in the dock and select Metagoofil from the resulting popup:

Enter your target domain in the next window and click OK. Select the maximum number of documents you would like to collect and click OK again:

Once it completes it’s run, it will store the results in /home/Domains/Metagoofil/ but will not open the directory automatically like many of the other tools on Buscador:

As you can see in the following screenshot, choosing my own domain as the example target was a bad idea because there was nothing to find. However you can still see the general format of what it will output, including an HTML report of it’s findings:

In general there will be 4 main files that are generated by this tool. Firstly, there will be a file called Full.txt that contains the full text of all metadata found in all documents found on the domain. Second will be an Authors.csv file, which will contain a list of names of all the people that wrote the documents that were found. Companies.csv will contain only information on companies that was found in the metadata, which helps identify associations between your target and other companies that may be relevant to the investigation. Last but not least, Modified.csv lists names of anyone who has modified the files found. This further assists in building a list of names of people associated with your target.

Harvester

One important piece of information that you usually want to find when performing an OSINT investigation against an organization are employee email addresses. These can be used to for phishing campaigns, which is a common method of gaining initial access into a corporate network as long as it is in scope. One good tool to do this is Harvester. Harvester can be found under the Domain Interact menu on the dock where you previously found Metagoofil:

This time you will want to select Harvester as seen below:

It will prompt you for the domain of your target. Since I don’t have an email for my domain, I opted to use a different domain which I have blurred in all screenshots out of respect. This is all publicly available information, but I don’t see any need to put anyone specific in the spotlight here:

Harvester searches many different sources in order to find email addresses associated with your target domain that have been publicly announced on the internet. These sources include: Baidu, Google, Twitter, Bing, DuckDuckGo, VirusTotal and many more. It will generate two reports in /home/Domains/TheHarvester related to your target domain, one HTML report for easy viewing and one XML report for easy consumption by other tools:

Subdomain Enumeration Tools

There are a few tools included in Buscador that allow you to take a single domain name and quickly discover many subdomains associated with it. These include:

• Sublist3r
• Knock
• Aquatone
• Amass
• Subbrute

These can all be found under the same Domain Interact dock menu. While these tools all perform similar tasks, they do vary in the sources that they pull from so it’s a good idea to run several. With that said, I personally run only two tools for this purpose which are Amass and Subfinder. While Subfinder is not part of Buscador, these two tools will have their own dedicated post in this blog. Given how similar all these tools are, I will leave the full write up for those posts, for now it is enough to know that they will output a list of all the subdomains available for a given domain.

Social Media

Social media is a goldmine of oversharing. For whatever reason, people are comfortable blasting all kinds of personal information out on sites like Twitter and Instagram for fake internet points. Given such a trove of valuable information like birth dates, addresses, the name of their favorite security question and when they will be away from their house it’s no surprise that some decent tools exist to pull this information out of the proverbial ether(net). In Buscador, these tools can be found on the dock under the Social Networks option:

Firstly, a custom script aptly named Twitter script takes a Twitter handle and will pull all available photos, tweets and a list of friends:

The files will be stored in /home/Social/Twitter/Exporter/<handle>/:

Clearly, I need to step up my Twitter game since I only have one picture of my cat.. You get the point though.

There is one more Twitter tool that is publicly accessible outside of Buscador and that is Tinfoleak. Tinfoleak seems to be much more powerful than the Twitter script, but that comes with a layer of complexity that isn’t always necessary so both have their place. Under the same Social Networks menu, this time select Tinfoleak and click OK. Here you can select and configure many different options to customize what you want to capture. Lets collect everything I can to see what happens:

The output report will be save in /opt/tinfoleak/Output_Report/<handle>.html:

When you open the report, you will see that it gathered a LOT of information. You will see every tweet, your tweet-to-like ratio, who’s following you and who you are following along with a lot of other data that lends to a report that is too large for me to get a good screenshot of. Here is just the beginning of it:

One other social media scraping script available is instaLooter, a python based tool that works similarly to the Twitter script but scrapes Instagram account data instead of Twitter. I discovered that this script is broken in the same way that youtube-dl was broken and that we could again fix it by issuing the command: sudo pip install --upgrade instaLooter:

As I don’t have an Instagram account myself, I will once again use John Maxwell as an example. Open instaLooter from the Social Networks menu option on the dock and enter an Instagram handle:

Once it completes, you should have a directory pop open with the information that it was able to scrape from the targetted account. Unfortunately, I didn’t make the appropriate sacrifices to the Demo Gods and it is not returning results for me right now to grab a screenshot of, so I will continue to troubleshoot the script and update this post if I get it working. In the mean time, if YOU get it working, be sure to leave a comment below to tell me what you did.

Spiderfoot

The last tool that I want to cover is Spiderfoot. Spiderfoot is a tool that takes in a domain name as a target and queries over 100 sources to gather information on everything from IP addresses, names, account names, emails and websites with the same account names registered. It creates a large report with graphs and tables to visualize this data and allow you to examine the details. There is a lot of correlation and guessing involved, and can possibly have a lot of false positives if your target doesn’t have a large web presence but it provides a good starting point for large scale investigations of a persons or organizations activities online.

It is availble as a bookmark at the top of the Firefox browser in Buscador:

When you first open it, you won’t have any scans listed so click on `New Scan` at the top to configure and start your first one. There are four modes to run a scan and each one can take a relatively long time to run. For my example, I ran a Footprint scan to gather information on the network perimete, associated identities and othe rinformation that is obtained through web crawling and search engines:

Once the scan completes, which took 2 hours for my example, you can see a list of categories of information that found along with the counts of datapoints for each group. By clicking on the type, you can drill down and see the specific datapoints. For my example, almost everything ended up being false positives since I have a very common name and not a large web presence, however I did discover a couple accounts on websites from 10 or 15 years ago that I had completely forgotten about.

In the tab graph, you can see the relationships between the datapoints, which your originally seeded target highlighted in red. This is a good way to judge how closely related the data points are. If someting is directly connect to your seed node in the graph, theres a lot higher confidence of the relationship to your target than if it’s linked to a different node, three or 5 nodes away.

Conclusion

OSINT is an enormous area of study which represents a great benefit and a great threat to everyone. The people that want to use this information to harm others will continue to craft their own tools and scripts to do so, so having a publically accessible and easy to use resource like Buscador is vitally import to allow us to discover what we have exposed and take necessary steps to protect ourselves.

While this was not an exhaustive list of the tools available in or out of Buscador, the tools covered here represent a great entry point for those of us justgetting started in our OSINT journey.

If you found this information and these tools interesting, head over to IntelTechniques and show support, and definitely pick up Michael Bazzells book Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information. It’s a great read and covers this information and a lot more in a much greater depth, and I am excited to continue reading through it.

If you found this content helpful, please consider donating your favorite cryptocurrency below.

  • Bitcoin
  • Ethereum
  • Cardano
  • Xrp
  • Litecoin
  • Stellar
  • Bitcoin cash
  • Dogecoin
  • Usdcoin
  • Aave
  • Uniswap
  • Wrappedbitcoin
  • Eos
  • Cosmos
  • Synthetix
  • Dash
  • Maker
  • Dai
  • Ethereum classic
Scan to Donate Bitcoin to 3ENZABkzJ8V2u3ewaHGFmsWx3i7pdy47Nr

Donate Bitcoin to this address

Scan the QR code or copy the address below into your wallet to send some Bitcoin

Scan to Donate Ethereum to 0x0a7B929203F77353DDBF9419Fc0d50a50932010c

Donate Ethereum to this address

Scan the QR code or copy the address below into your wallet to send some Ethereum

Scan to Donate Cardano to DdzFFzCqrhsuy6avGUazSq2S5VKEBS6rTxo1Cy71DKwozED8nXFwtzAZda4u5GbE3B4oKFwKVNw3fbAfCrkRdpv8pma83rsgokDk1CTy

Donate Cardano to this address

Scan the QR code or copy the address below into your wallet to send some Cardano

Scan to Donate Xrp to rw2ciyaNshpHe7bCHo4bRWq6pqqynnWKQg:::ucl:::2447425404

Donate Xrp to this address

Scan the QR code or copy the address below into your wallet to send some Xrp

Scan to Donate Litecoin to LXQcKbXSemXnF3VNH5Ez7n4SDUFTkh1uJp

Donate Litecoin to this address

Scan the QR code or copy the address below into your wallet to send some Litecoin

Scan to Donate Stellar to GDQP2KPQGKIHYJGXNUIYOMHARUARCA7DJT5FO2FFOOKY3B2WSQHG4W37:::ucl:::639763260

Donate Stellar to this address

Scan the QR code or copy the address below into your wallet to send some Stellar

Scan to Donate Bitcoin cash to 17iu6o47bdYRbGoxV3WeWUCUBcYZsrXYeG

Donate Bitcoin cash to this address

Scan the QR code or copy the address below into your wallet to send some Bitcoin cash

Scan to Donate Dogecoin to DNserpUCc77Q9EvoU2BePPMnB2nJzJcbRW

Donate Dogecoin to this address

Scan the QR code or copy the address below into your wallet to send some Dogecoin

Scan to Donate Usdcoin to 0xC8354d47Df616EbEb9E91b65C64F352Ce2457215

Donate Usdcoin to this address

Scan the QR code or copy the address below into your wallet to send some Usdcoin

Scan to Donate Aave to 0x268a1fc2eF33c37c0f6D593C17886c2105456f7C

Donate Aave to this address

Scan the QR code or copy the address below into your wallet to send some Aave

Scan to Donate Uniswap to 0xf83FA7Ef077fDA536029EC1F6B73CC87d1D33a54

Donate Uniswap to this address

Scan the QR code or copy the address below into your wallet to send some Uniswap

Scan to Donate Wrappedbitcoin to 0xac7C2A4a700a3c6B341658960C7e99BcC7A93387

Donate Wrappedbitcoin to this address

Scan the QR code or copy the address below into your wallet to send some Wrappedbitcoin

Scan to Donate Eos to coinbasebase:::ucl:::2151882130

Donate Eos to this address

Scan the QR code or copy the address below into your wallet to send some Eos

Scan to Donate Cosmos to cosmos1scwcfre6h4c7epkyrdfegpeaz8umqldl50gn8w

Donate Cosmos to this address

Scan the QR code or copy the address below into your wallet to send some Cosmos

Tag/Note:- 102476592
Scan to Donate Synthetix to 0x89Ee88FdD6B30f72a8e6E6BF3F948Ac3B45e16A8

Donate Synthetix to this address

Scan the QR code or copy the address below into your wallet to send some Synthetix

Scan to Donate Dash to XoMHNXYDx37sDq4bij96cTEpcBznLsbaab

Donate Dash to this address

Scan the QR code or copy the address below into your wallet to send some Dash

Scan to Donate Maker to 0xF72Ace0DdCa3b8e2F691700d8a78282b5d628E12

Donate Maker to this address

Scan the QR code or copy the address below into your wallet to send some Maker

Scan to Donate Dai to 0x22675f754c285bCD7fF8D411Eca32348021a4162

Donate Dai to this address

Scan the QR code or copy the address below into your wallet to send some Dai

Scan to Donate Ethereum classic to 0x03DC1DeFBaFD17Ebf77827faEAc5E50AB90e3b52

Donate Ethereum classic to this address

Scan the QR code or copy the address below into your wallet to send some Ethereum classic

Leave a Reply