Thinking inside the box: An evaluation of a novel search‐assisting tool for supporting (meta)cognition during exploratory search

Abstract Exploratory searches involve significant cognitively demanding aiming at learning and investigation. However, users gain little support from search engines for their cognitive and metacognitive activities (e.g., discovery, synthesis, planning, transformation, monitoring, and reflection) during exploratory searches. To better support the exploratory search process, we designed a new search assistance tool called OrgBox. OrgBox allows users to drag‐and‐drop information they find during searches into “boxes” and “items” that can be created, labeled, and rearranged on a canvas. We conducted a controlled, within‐subjects user study with 24 participants to evaluate the OrgBox versus a baseline tool called the OrgDoc that supported rich‐text features. Our findings show that participants perceived the OrgBox tool to provide more support for grouping and reorganizing information, tracking thought processes, planning and monitoring search and task processes, and gaining a visual overview of the collected information. The usability test reveals users' preferences for simplicity, familiarity, and flexibility of the design of OrgBox, along with technical problems such as delay of response and restrictions of use. Our results have implications for the design of search‐assisting systems that encourage cognitive and metacognitive activities during exploratory search processes.