Private vs Public Blockchains: Night vs Day

What is it that qualifies a blockchain as private or public and how do they compare?