The first step to answering this question is to begin to define your project. Start with making a list of your needs for the website and the needs of your organization. Mom and pop businesses have very different needs from large enterprises. By defining your needs, it will be easier to communicate with a prospective web designer (whether they are an agency, boutique firm or freelancer).
Cost
In terms of pricing, freelancers are going to be less expensive overall. They do not need to support a staff, support facilities or infrastructure that an agency would maintain. Freelancers may be the best option for small businesses and solo entrepreneurs if cost is the main factor. For many businesses, cost is not the only factor. Quality, performance, support, technology and design may be very important factors as well. Many businesses begin building their website with the cheapest option available, only to start over months larger after wasting time, energy and money in the process.
Getting it right
When working with either an agency or a freelance website designer, it is highly recommended to explain your intentions and needs clearly in the beginning. Some clients may not be organized in the beginning of the project, but this can hurt the client more than the web designer. If the web designer can understand the purpose and needs of the website, they will be better equipped to offer the right solution.
Flexibility
In terms of larger projects, typically agencies are better equipped to deal with a broader assortment of client needs. These needs may include long-term support (having technical, design, marketing and account team members available), ability to support larger technical requirements, ability to support a variety of design needs, etc. Sure there are some super-start freelancers out that that produce some great work. This article is speaking in very general terms regarding agencies vs freelancers.
Launch and post-launch support is often neglected in the request for proposal. As you interview agencies and freelancers, consider your needs for the website, as well as your needs long-term.
Interviews
When exploring web design agencies, it is a good idea to ask questions about their approach, process, technical abilities, their team make-up and who will be personally working with you (the client). Will you be working with a project manager? If you don't work well with the project manager, can you ask for an alternative manager? Is there a support team in place for long-term needs? Some web design agencies do not mind working with clients long-term, whereas some agencies and freelancers build and design websites and then prefer to walk away.
Communications
The ability for a freelancer vs a web design agency to communicate will be different. You may need to ask about each entity's process around communications. Do your candidates have an established and clear process? Does client feedback factor into the process at all? Are there reporting milestones or decision points? With freelancers, you may need to ask about their workload. If a freelancer gets busy, it may be challenging for them to be responsive. This can be especially true if they are overwhelmed with projects. Some agencies have processes in place to handle growth or scale. Nevertheless, it is important to understand the capabilities of the specific agency or freelancer that you are speaking to.
Rarely will you find the process of hiring a web design firm an apples to apples comparison.