The Dwarves ate all of his food. Bilbo decided to go with the company because he would've starved to death otherwise.