In this study, we explore the efficacy of Generative AI and Large Language Models (LLMs) in the tasks of constructing and completing Educational Knowledge Graphs (EduKGs). Knowledge Graphs (KGs) help represent real-world relationships. This can take the form of modeling course domains and student progression in educational settings. Through this work, we leverage GPT-4 to aid KG construction and align it with predefined learning objectives, course structure, and human interaction in validating and refining the generated KGs. The methodology employed utilized prompting LLMs with course materials and evaluating the generation of KGs through automatic and human assessment. Through a series of experiments, we show the potential of LLMs in enhancing the EduKG construction process, particularly for course modeling. Our findings suggest that LLMs such as GPT-4 can augment EduKGs by suggesting valuable and contextually relevant triplets. This KG creation and augmentation approach shows the potential to reduce the workload on educators and adaptive learning systems, paving the way for future applications in content recommendation and personalized learning experiences.