In this talk, I will walk through current approaches for using deep learning for 3D reconstruction and 3D recognition tasks, and highlight novel directions to solve these limitations. In the first part of the talk, I will give an overview of methods aimed at learning descriptors of 3D objects from different 3D data representations, focusing in particular on point clouds. I will discuss how current approaches are moving towards unsupervised learning, using different architectures and learning methodologies. In the second part of the talk, I will focus instead on monocular 3D reconstruction, discussing the use of monocular depth prediction for monocular SLAM and omnidirectional images. Finally, I will introduce our recent work in monocular 6D object pose estimation and highlight methodologies to overcome the domain shift.