Now what you need to look for is what services you need. Are you going to serve just HTML-pages, or dynamic content with php, asp or jsp.
What access do you want? Only a web-interface, or also ftp-access? Or perhaps ssh-access?
Do you need a database or not?
Another major part you need to look for is traffic, your site is going to use a certain amount of traffic. Now most providers give you 5, 10, 30 GB traffic a month but there are also providers that give unlimited amounts of traffic (Fair use).
First decision to make is what service to choose:
A dedicated server is a complete computer for yourself. Advantage: more security, a lot of diskspace, you don't have to share CPU-time, the bandwidth can't be absorbed by other websites.
With a virtual-server you "share" one computer with more websites (up to a few hundred). Advantage: its a lot cheaper then dedicated servers.
Dedicated connection and co-location lets you take a lot more responsibility.
With dedicated connection you get a connection (ISDN, cable, T1) to your house (office), but you need your own computer to act as server. You have physical access to you machine but there also a series of negative sides, power-outage, floods, ...
You can also set you computer at your ISP(Internet Service Provider), this is called co-location. The benefit is that your computer is at a secure place, high speed connection. But that place can be far away, and they don't do anything on your computer, maybe re-booting(reset) but that's it.