Teaching Cyber Security students to think like an attacker and attempting to inculcate in them the security mindset is a common yet important facet of security education. Most of the literature concerning this topic focuses on technical activities such as network mapping, reconnaissance, or host exploitation. In this paper, we describe our experience with a hands-on activity in which students had the chance to think like an attacker, but in the less technical, but arguably equally important, area of Open Source Intelligence gathering. Our students were given the opportunity to conduct an information gathering attack on real targets, compiling the results into a report that was delivered to the target, and receiving feedback from the target about their findings. We finished the project by surveying the students to discover what they learned and how the project changed, not only their perspective on the availability of information on the internet, but also how they intend to change their behavior as a result of the exercise.