Are Neural Networks just lots of logical gates?

Hi.

I've seen an example of a neural network in which the weights were tuned so that an input (a, b) had an oupput equal to a AND b, that is, the network was a logical gate. And that got me thinking if every neural network was just a large collection of neural gates.

That is, suppose we wanted to write, say, a recognizer of handwritten numbers. If we were supersmart we could write an incredibly complex deterministic algorithm that we could use to classiy images into numbers. But since we are not that smart (in fact I'm not sure an algorithm like that can even exist) we have to use a neural network. What I thought is that what the network was actually doing is find the set of logic gates that most closely acts like the "ideal" algorithm (which is fundamentally also a set of logic gates).

Is it so?