ChainForge: An open-source visual programming environment for prompt engineering

Prompt engineering for large language models (LLMs) is a critical to effectively leverage their capabilities. However, due to the inherent stochastic and opaque nature of LLMs, prompt engineering is far from an exact science. Crafting prompts that elicit the desired responses still requires a lot of trial and error to gain a nuanced understanding of a model's strengths and limitations for one's specific task context and target application. To support users in sensemaking around the outputs of LLMs, we create ChainForge, an open-source visual programming environment for prompt engineering. ChainForge is publicly available, both on the web (https://chainforge.ai) and as a locally installable Python package hosted on PyPI. We detail some features of ChainForge and how we iterated the design in response to internal and external feedback.