The complex carbohydrate structures decorating human proteins and lipids, also called glycans, are abundantly present at cell surfaces and in the secretome. Glycosylation is vital for biological processes including cell-cell recognition, immune responses, and signaling pathways. Therefore, the structural and functional characterization of the human glycome is gaining more and more interest in basic biochemistry research and in the context of developing new therapies, diagnostic tools, and biotechnology applications. For glycomics to reach its full potential in these fields, it is critical to appreciate the specific factors defining the function of the human glycome. Here, we review the glycosyltransferases (the writers) that form the glycome and the glycan-binding proteins (the readers) with an essential role in decoding glycan functions. While abundantly present throughout different cells and tissues, the function of specific glycosylation features is highly dependent on their context. In this review, we highlight the relevance of studying the glycome in the context of specific carrier proteins, cell types, and subcellular locations. With this, we hope to contribute to a richer understanding of the glycome and a more systematic approach to identifying the roles of glycosylation in human physiology.