Learning about Machine Learning

Before I start, I want to say I've looked around at /r/MLQuestions and /r/mlclass, but neither really have what I'm looking for.

I'm a PhD in math (geometric group theory) who recently left academia to work in industry. I'm trying to find resources to learn the "basics" of advanced machine learning. I'm thinking something like Rudin's Real and Complex Analysis or Munkres' Topology for mathematics, but for machine learning; something suitable for a first/second year graduate student in ML how has all the mathematical background. I've been looking around, but I haven't been able to find anything in one place - Ng's Coursera course was good, but not nearly deep enough, and otherwise it seems like I'm limited to random blog posts or cutting-edge papers on the arXiv. Are there any good texts?

On a related note, it seems like the math behind (almost) all of ML is quite aged: to understand most of it you need a good understanding of Linear Algebra, Prob/Stats, and some basic optimization. Almost all the papers I've read are (very) clever tricks to get more out of the existing mathematical toolset, or clever representations of data.

The neatest thing I've seen (from my perspective) is research involving using Grassmanians to model data, and then embedding the that Grassmanian manifold into R^n, and apply an SVM, but even that is basically linear algebra. Is there cutting edge ML involving deep geometric/topological notions?

Thanks!