Monocular depth estimation is gaining much interest in the computer vision community because it has broad applications in autonomous driving systems, robotics, and scene understanding. Significant progress has been made in solving the monocular depth estimation problem using deep learning techniques. Unsupervised learning methods are particularly appealing since the problem can be treated as an image reconstruction task, thereby forgoing the need for ground-truth depths. This paper presents an unsupervised approach to training convolutional neural networks for monocular depth estimation by introducing a novel architecture called DFRNets. DFRNets shares weight parameters between the image reconstruction sub-network and the disparity refinement sub-network and adopts a multi-scale structure for disparity predictions. The proposed method computes dense disparity maps directly from monocular images and refines them in an end-to-end fashion to reduce visual artifacts and blurred boundaries, thereby improving the method’s overall performance. Experiment results using the KITTI test set showed that the proposed method outperformed many state-of-the-art methods, since it achieved the best performance on the two distance ranges: 0−80 meters and 1−50 meters. Moreover, the qualitative results revealed that the method generated more detailed and accurate depth maps of the scenes, with no border artifacts around the image boundary.