My SQL Meetup: The Evolution of Data at Wikia

This meetup covered the following topics and interesting points concerning wikia and mass data storage/query/operation:

  1. How to store a lot of random writes to a hard drive / database quickly.
  2. Threre a large bunker-type data centers in Iowa being de-commissioned by the U.S. military.
  3. Out in Iowa there is not much bandwidth, which needs to be built in.

Sidebar thoughts:

  1. What Google does in terms of organization and speed is incredible.
  2. Google has the information to pretty accurately predict what you are searching for, which probably means at least more than one person searched for the same thing. Though they may also have hired people to vet this system at Leapforce.
  3. Google, deliberately or not, can actually greatly influence a mass consciousness in terms of search and information that a lot of people see. I think it is good that no information source ever dominate the top of google for a wide variety of key terms.

Wikia Stats

  • 32 Million Page Views a Day
  • 2 Million Uniques a Month
  • 1 Gigabyte of HTML a Second
  • Alexa Traffic Rank of 201

Wikia uses Varnish (which claims to make websites fly).

Link to the talk can be found here.

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Printed from: http://www.flintfordreams.com/2010/10/18/my-sql-meetup-the-evolution-of-data-at-wikia/ .
© flintfordreams 2012.

Leave a Reply



Hold fast to dreams
For if dreams die
Life is a broken-winged bird
That cannot fly

Hold fast to dreams
For when dreams go
Life is a barren field
Frozen with snow

-Langston Hughes


Subscribe to the RSS Feed OR

Get posts at your inbox

Email:

  • Blogroll

  • My Other Sites

  • Tags