Intramuscular fat (IMF) in pigs determines the succulency and attractiveness of the meat. This paper presents a non-invasive in vivo method for estimating IMF using deep convolutional neural networks on ultrasound images. The method performs best on moderate to low IMF images <6% giving a correlation of R=0.82 and root-mean-square-error RMSE=1.2. At higher IMF content the convolutional neural network fails to generalize due to image quality and lack of training data.