Building an Ad Server Engine

My next major project will be to build an ad server engine. When I say “engine”, I’m referring only to that part of an ad server that will deal exclusively with ad delivery, basically the part that receives and responds to ad requests from publishers.

Although there’s a lot more going on in a full-featured ad server (user system, campaign management, ad targeting, billing system, rule-based system for ad delivery, etc…), the serving engine will be the most critical component since this part of the system will need to cope with a massive load (think billions of ad requests per month for example).

To measure the success of this project, I’ve defined the following objectives:

• In it’s minimal configuration, the engine should handle at least 1 billion ad requests per month. It should also handle traffic/load spikes of up to 30-40% of the normal traffic/load.
• The system should be scalable horizontally; that is, if additional servers are needed, those can be added relatively quickly and simply to the system with no software change required.
• In the event of a server being added, the system should detect the new configuration and automatically distribute part of the load to the new server.
• In the event of a server crash, all dependent servers (that is, those that relied on the crashed server) should automatically create connections to other still available servers.

As you can see, except for the first objective above, all other objectives hint at the idea of actually building a distributed, scalable and fault tolerant system in general, which can be used to run any kind of application that will be under heavy load.

During development, I will try to post as many times as possible with details about the various parts of the system, architecture and specific implementation details. Your feedback will be greatly appreciated along the way.