We discuss a framework for the parallel optimization of a differentiable
(possibly nonconvex) function and a nonsmooth (possibly nonseparable),
convex one. The latter term is usually employed to enforce structure in the
solution, typically sparsity. This framework incorporates successive convex
approximation schemes with parallel and hybrid random/deterministic (greedy)
selection of block determined subproblems. The scheme has been proven to
converge almost surely and numerical results on huge-scale problems
suggests that this hybrid random/deterministic algorithm outperforms the
alternatives.