A Batch class allows you to define a single job that can be broken up into manageable chunks that will be processed separately.

One example is if you need to make a field update to every Account in your organization. If you have 10,001 Account records in your org, this is impossible without some way of breaking it up. So in the start() method, you define the query you're going to use in this batch context: 'select Id from Account'. Then the execute() method runs, but only receives a relatively short list of records (default 200). Within the execute(), everything runs in its own transactional context, which means almost all of the governor limits only apply to that block. Thus each time execute() is run, you are allowed 150 queries and 50,000 DML rows and so on. When that execute() is complete, a new one is instantiated with the next group of 200 Accounts, with a brand new set of governor limits. Finally the finish() method wraps up any loose ends as necessary, like sending a status email.

So your batch that runs against 10,000 Accounts will actually be run in 50 separate execute() transactions, each of which only has to deal with 200 Accounts. Governor limits still apply, but only to each transaction, along with a separate set of limits for the batch as a whole.

Disadvantages of batch processing:

It runs asynchronously, which can make it hard to troubleshoot without some coded debugging, logging, and persistent stateful reporting. It also means that it's queued to run, which may cause delays in starting.
There's a limit of 5 batches in play at any time, which makes it tricky to start batches from triggers unless you are checking limits.
If you need access within execute() to some large part of the full dataset being iterated, this is not available. Each execution only has access to whatever is passed to it, although you can persist class variables by implementing Database.stateful.
There is still a (fairly large) limit on total Heap size for the entire batch run, which means that some very complex logic may run over, and need to be broken into separate batches.

Reference:

https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_batch_interface.htm
http://salesforce.stackexchange.com/questions/920/why-use-batch-apex