In this article, we present a modification to the state-of-the-art N-BEATS deep learning architecture for the univariate time series point forecasting problem for generating parametric probabilistic forecasts. Next, we propose an extension to this probabilistic N-BEATS architecture to allow optimizing probabilistic forecasts from both a traditional forecast accuracy perspective as well as a forecast stability perspective, where the latter is defined in terms of a change in the forecast distribution for a specific time period caused by updating the probabilistic forecast for this time period when new observations become available (i.e., as time passes). We empirically show that this extension leads to more stable forecast distributions without causing considerable losses in forecast accuracy for the M4 monthly dataset. Finally, we present a second extension to the probabilistic N-BEATS network which makes it possible to jointly optimize single-period marginal and multiperiod cumulative (i.e., aggregated over multiple time periods) probabilistic forecasts. Empirical results are reported for the M4 monthly dataset and indicate that improvements in accuracy can be obtained over basic but well-established methods to produce probabilistic cumulative forecasts. The proposed probabilistic N-BEATS network and the extensions are all useful in a supply chain planning context.