information and interaction

A point that I've made, but that has perhaps been better illustrated by Borges, is that extra information is less information; if you have 4MB of data, from which you need to find the 1k you want, you have, on some level, less information than if you just had the 1k.  (Maybe 12 bits less?  I don't know.)  As a related matter, if I need information from you, we may well be able to transmit it efficiently if we can go back and forth a bit than if not.  If I send one of 2ⁿ messages indicating a broad category, and you respond with one of 2^m responses to help me clarify my next request, and that request is l bits, and the final answer is k bits, then we've exchanged a total of n+m+l+k bits; if I had to send a single request, I would need to send l bits for each of the 2^m responses you might send to my initial message (plus perhaps the n bits as well); my request is 2^ml bits, which is huge. If you know I need the information, but have to send it without my request, that's 2^2mlk bits you have to send me to make sure I get what I want.

I kind of got to thinking about this in the context of the Mars rover, for which two-way communication is possible, but with latency.  If the latency doubles, to the extent that analogues for n and l are appreciable, you've basically just halved the rate of information transmission; the ability to recover from that latency by transmitting extra information on spec is basically negligible.