[Question] How can I combine multiple neural networks into one?

Hi!

I'm implementing Cooperative Q-Learning with multiple agents as described by Ahmadabadi. However, there is a communication overhead (agents communicate over a network and trying to be energy efficient, so the data sent over network is costly) and I'm planning to implement Q function using a neural network instead of a matrix to reduce communication overhead.

But, I could not find a method to combine the neural networks of several agents into one to send them back to agents. Is there a method that does that efficiently? Do you have any ideas?

Or, alternatively, do you have any ideas to minimize the overhead?

In the worst case, I'm planning to reconstruct Q function as a matrix and train a neural network with all of the the extracted Q functions.