Starting down the path of Rotten Tomatoes

How watching the mediocre movie made me build something GREAT

With Rotten Tomatoes‘ 20th anniversary rapidly approaching, I’ve decided to do a retrospective on my memories from our early days of Rotten Tomatoes. As  a Rotten Tomatoes founder along with my partners Patrick Lee and Senh Duong, I’ve never really had a chance to document some of the stories from our early days. I ask for forgiveness in advance if there are errors as I’m trying to recall details from 20 or more years ago, really a lifetime in internet years (and yet I still feel like it was all so recent).

Watching Star Wars: Episode I: The Phantom Menace, released in the summer of 1999, was a life-changing moment for me. No, the movie was an utter letdown for a life-long Star Wars fan like me, but I thank George Lucas for making such a mediocre movie as the online reaction to the movie helped change my mindset about what I wanted to do in life.

In May 1999, the week before Episode I came out, I got to see the film at an early, private radio promotion screening that also had a few film critics in attendance. When Rotten Tomatoes co-founder Senh asked me earlier in the week whether I wanted to catch a sneak preview of the most anticipated movie of my life (and probably in all of history), of course I replied ‘yes’ and I made the 90 minute drive alone from the San Francisco Bay Area up to Sacramento.


Senh had launched Rotten Tomatoes nearly a year earlier as a hobby project while working at our web agency at the time. Design Reactor, which our third Rotten Tomatoes co-founder and I had started right after my graduation from Cal in 1997, was making rapid headway as the lead web development agency for Disney Channel and at the time I was making a weekly trip back and forth between the Bay Area and Burbank happy and proud to be working for such a prominent client in the field that I loved.

Senh and I at Big Game in November 2005, right before leaving Rotten Tomatoes and continuing my entrepreneurial dreams in China

Nonetheless, the many sleepless nights of hard work with little outside recognition must had worn down Senh early on. In August 1998 (now 20 years ago!), he had come up with the idea for Rotten Tomatoes. Really, it was quite a genius, thinking out-side-the-box idea at the time and I’m quite envious that I hadn’t thought about it before. Both Senh and I are super movie afficianados. Senh is ethnically Chinese but from Vietnam with a short interlude in Hong Kong before arriving in the relatively rural Sacramento in the 1980’s. For Senh, 80s action stars like Schwarzenegger and Stallone play such a pivotal part of not just his childhood, but also his general perception of Hollywood and America overall. In the same vein, I was equally a movie fan, but my tastes were much more diverse. Senh and I could talk endlessly about our favorite films and directors, but I was also obsessive about small films from mini-majors and indie productions.

I was born in Ohio, but raised in suburban Maryland. When summers rolled around, my brothers and I would live Los Angeles in the increasingly immigrant Chinese community of Alhambra to see my father. As all my school friends were in Maryland rather than LA, with the lack of friends nearby, my brother and I basically spent every summer going to the movie theater and watching tons of summer movies of every sort. For him, it turned in to a profession as he chased his moviemaking dreams in Hollywood after college. For me, it meant that movies were always a major part of my life, even after I began chasing my dotcom dreams in college. It was a point of pride for me to have Disney as my major client so soon after graduating college that, unlike Senh, I didn’t really think much of building something “even greater”.

For Senh, however, he had more of an “artist” mentality (very similar to a “founder” mentality) and wasn’t content with essentially working as “affordable labor” for Disney. We couldn’t even publicly claim our hard efforts building such breakthrough projects as Zoog Disney, ToonDisney.comDisneyChannel.com, and eventually much of ABC.com. When Jackie Chan, his Hong Kong film idol, was about to star in Rush Hour, his first major Hollywood movie, Senh devoted his time to building his version of a “fan page”. He collected all of the news articles and film reviews in the weeks preceding the movie’s release and put them on a single page. In reality, while he meant to build a site for Rush Hour, since the film’s release got delayed, he actually continued the process of aggregating different review quotations and news headlines for other movies about to release. The first film page to launch was Neil Labute’s Your Friends & Neighbors, and by the time it had launched on August 13, 1998, he had already come up with some of Rotten Tomatoes’ key elements: The Tomatometer, Fresh and Rotten icons for reviews, review quotations and links, and the “Rotten Tomatoes” name. Senh had even registered the domain name rotten-tomatoes.com for his new web site. He began posting links to the aggregated Rotten Tomatoes movie review pages to the rec.arts.movies newsgroup and getting decent response from other newsgroup participants.

Early on, I had concerns about the legality of “aggregating quotations and links”. In those early days, the closest comparison would be the Drudge Report, but the quotation aggregation that Senh was doing was even riskier. It’s a commonplace practice now and commonly considered as covered legally under “Fair Use”, but back then it was still indeterminate. I think Senh’s “immigrant outsider” background and his artist mentality were important — they allowed him to think “outside-of-the-box” and make the important leap to come up something wholly new and innovative by aggregating this quotations, links, and ratings into the Tomatometer where as someone like me born and raised in the U.S. would have considered it too legally risky.

In the subsequent months, Senh’s “Jackie Chan fan project” blossomed in to his passion. He dedicated more and more of his time to updating and expanding Rotten Tomatoes including spending daytime going to the Berkeley Public Library to manually cull quotes from print newspapers and magazines (many of which hadn’t gone online yet in 1998) and all-nighters on the critical Thursday nights before movie openings on Fridays. At the same time, though, our little web design firm Design Reactor, had finally landed on the rollercoaster growth path as our initially small business with Disney Channel blossomed into becoming the primary web agency for the whole of Disney Cable Television. In the several months following the launch of Rotten Tomatoes, our company grew from 6 or 7 employees to 20-plus employees and interns and landed a seven-figure, yearlong deal with Disney to maintain and expand the whole of Disney Channel and affiliated web sites. During all of this, Senh became less and less interested in “working for Disney” and more and more obsessed with Rotten Tomatoes. Being a fan of movies and a fan of the web site, I helped initially by helping to do some minor programming, hosting the web site on our Design Reactor servers, and making the process easier for Senh by changing his manual HTML pages to more maintainable and reuseable templates. Nonetheless, both Patrick and I could clear see that Senh’s interest in Design Reactor was waning so we asked him to leave the company and so he could work full-time pursuing his Rotten Tomaotes passion and so that we could bring in a replacement as Creative Director (Joe Huang) who could dedicate his time to our growing web design company.

Instead of hanging around the Bay Area and helping us grow Design Reactor, Senh decided to move back to Sacramento and teamed up with his high school classmates Binh Ngo and Bobby Lee to continue running Rotten Tomatoes from Senh’s garage. In the first couple of months after departing Design Reactor, I don’t think Senh was even 100% certain about using his time to make Rotten Tomatoes a business. There was a short period of time during those early months where the three of them decided to shoot a movie together rather than continue updating Rotten Tomatoes and, as a consequence, there were no updates to the site for several weeks, though they resumed after deciding to not film the indie movie.

At the same time that Senh, Binh, and Bobby were updating Rotten Tomatoes from Sacramento, I was getting run down by my weekly trips between the Bay Area and Burbank simultaneously growing our Disney business to cover Disney Channel, Zoog Disney, and Toon Disney and producing two new Flash/Shockwave games for them every week. All of the newfound business from Disney allowed us to move to professional high rise offices in Emeryville and hire a professional CFO for the company. Despite our success with Design Reactor, I was begining to get run down by the constant client-handling and envious of being able to build and own our own property like Rotten Tomatoes.


This was the situation as I drove up to Sacramento to go watch the early critics screening of Star Wars: Episode I with Senh, Binh, and Bobby that fateful May night. Senh and I had always had a great relationship chatting about movies and, on the car drive over from Senh’s house to the theater, our movie geek conversation about Star Wars continued without missing a beat. After the movie ended, I clearly remember walking out of the theater and noticing other people’s reactions. Before the  screening, the local radio station had also given tickets to fans. One of the super fans dressed in a Darth Maul costume replete with a homemade, dual-blade light saber and face paint. He walked out of the theater with such a dejected look on his face as if he had just realized that his entire childhood was a lie.

Leaving the theater, Senh and I talked about what we had just saw. I think we both realized at the same time that this huge build up to the release of Episode I was a huge opportunity for Rotten Tomatoes. The fact that the movie was just “so-so” was even better for the web site — there was going to be some really split opinions about the movie over the subsequent days. During the car ride home, I told Senh that I really wanted to work on Rotten Tomatoes rather than just doing Disney work day-in-and-day-out. I’d talk to Patrick about having Design Reactor dedicate more time and resources towards helping Rotten Tomatoes. Watching Star Wars: Episode I and talking with Senh on that car ride back was a pivotal moment for me. It made me come to realize that, despite how proud I was of the product and relationship we had built with Disney, that what I really wanted out of life was to build something I could claim for my own. I really wanted to work in earnest on Rotten Tomatoes.

Senh, Binh, and Bobby, while hard-working, didn’t really have much technical knowledge. As a consequence, I sought out at first to help them on the technical end. Firstly, Rotten Tomatoes was still using the “rotten-tomatoes.com” domain name so after getting back home I immediately went about registering “rottentomatoes.com” (no dash) which, luckily, had still not been registered. Secondly, we hosted the site on a web server sitting in our Design Reactor offices which helped accomodate the server load in subsequent months. Ironically, by this point in time, becoming Disney’s web agency of choice meant that we were the only agency to have a dual T1 line guaranteeing top-of-the-line network speeds that connected directly to Disney’s private network so that we could develop and test code for all of Disney Channel before deploying to Disney’s production servers. Thanks to Disney paying for our exorbitant (at the time) network line, Rotten Tomatoes was able to share the network access and web server resources in those important summer months in 1999 as site traffic took off.

True to form, in the several days following that Star Wars: Episode I screening as early reviews began pouring in, the traffic to the site exploded. The day the movie released, the Tomatometer score hovered around 58%-61% and constantly flipped back and forth multiple times between FRESH and ROTTEN as we added newly published reviews (I see now that it’s settled on a more permanent rotten rating of 55%). More and more movie fan sites and message boards began linking and referring to Rotten Tomatoes’ Episode I web page and traffic continued to pour in. For the first time, the Tomatometer rating became an actual point of conversation amongst critics and tons of Star Wars fans and the reaction to Rotten Tomatoes set us on the path to make this a real project.


In the subsequent summer weeks, several important things happened to further push us towards making Rotten Tomatoes a full-time business pursuit:

Around April 1999, my brother, working as an aspiring producer at Sony by this point, sent me a video tape of The Blair Witch Project, which had become the buzz hit of Sundance in January but hadn’t released in theaters yet. I had heard so much of this indie horror movie and was even more enthusiastic since it was filmed not too far away from my hometown of Columbia, Maryland. I popped it into the VHS player at the office one weekend and we all watched it together with Senh, who had I believe had also come down from Sacramento to watch with us that weekend. It scared the crap out of some of our teammates, but more importantly, it was pretty obvious after watching the movie that it was pretty special because opinions on the movie were so divided. Patrick, who also grew up in suburban Maryland even closer to where Blair Witch was shot, absolutely hated the movie, and particularly hated how illogically and childish the characters in the movie acted. I loved the movie and Senh was in the middle with his opinion. We knew that online opinion would also be similarly vociferous.

Following on Star Wars: Episode I in May, the release of The Blair Witch Project and all of the film fan discussion in July further confirmed our feeling that Rotten Tomatoes was on to something. People were constantly returning to Rotten Tomatoes to read all of the film reviews, commentary, and debate on such a controversial movie. Because of Episode I and Blair Witch, Rotten Tomatoes was becoming more and more known — Leslie Miller from USA Today featured Rotten Tomatoes and Netscape selected the site as a “Pick of the Day”. Most gratifyingly, Roger Ebert highlighted Rotten Tomatoes in “Yahoo! Internet Life” magazine, a short-lived publication that educated users on the best sites to surf. Roger was an early hero of both Senh and me; consequently, it’s hard to understate how much it meant to us to have his personal validation during our early days.

Our web design business Design Reactor was growing by leaps and bounds. By this point, our business had expanded from just being 90% Disney work, to having a more diverse portfolio of clients adding on Artisan Entertainment (who, in a touch of fate, had grown to success with the release of The Blair Witch Project) and Warner Bros.. Despite this success, we were increasingly casting an eye towards building our own project that we could claim for ourselves. At the same time that we moved out of our Berkeley office (a little before the release of Episode I), our office mates Lyle and Dennis Fong from the original Berkeley office had created Gamers.com, the world’s first gaming online portal, and in October 1999 raised $11M, an eye-poppingly big investment back then. I think both Patrick and I were envious of their success and wanted to build something that could surpass our friends’ at Gamers. In this light, it was relatively easy to agree to bring Rotten Tomatoes back into Design Reactor as a full-time project. Seeing as how our web design business was already growing quickly, our original idea was for Senh to move back to the Bay Area and for Design Reactor to incubate Rotten Tomatoes. Patrick talked to some of our potential angel investors, and based off of our success with Design Reactor, they were willing to invest in Design Reactor as an “IdeaLab“-style incubator with Rotten Tomatoes as the first incubated project. However, as we began putting the business plan together for Rotten Tomatoes, we quickly came to the realization that our incubator idea was stupid. Life was too short and the three of us were more passionate about building Rotten Tomatoes full-time rather than working across a bunch of projects for others.

Luckily, our  investors believed in us as a team and most of them willingly put money in on the basis of Rotten Tomatoes rather than our former “Design Reactor incubator” plan. In early 2000, I already began transitioning our Disney work (which had expanded to include ABC.com) over to a sister company. We were already sub-contracting some work to this company and they took over our Design Reactor name and portfolio and we took an equity stake and the decent cash and receivables we had built in the Design Reactor ledger.

My final project for Design Reactor was in late March 2000 — I flew down to Hollywood to attend the Academy Awards where Design Reactor ran and updated the official Oscars.com web site during the awards. Subsequent to my work for the ceremony, I flew back to the Bay Area and, in the following week, our one and only round of investment in Rotten Tomatoes arrived in our bank account and I began working on Rotten Tomatoes full-time.

Then, the very next week, the dotcom bubble burst and things began crashing down around us. That, however, is a story for another post…

Rotten Tomatoes leaders circa April 26, 2000: Paul Lee (business development), Patrick Lee (CEO), Lily Chi (CFO), Senh Duong (COO and creator), and me (CTO)
The rest of the Rotten Tomatoes team from our office in Emeryville in a San Francisco Chronicle article published right after we officially started Rotten Tomatoes as an independent company

For more of my Rotten Tomatoes 20th anniversary articles, check out my other memories.

Postscript:

Like I said, I haven’t really had the opportunity to document our early days at Rotten Tomatoes and there were plenty of people who I’ve neglected to publicly thank. First, of course I want to thank my co-founders Patrick Lee and Senh Duong and the rest of the Rotten Tomatoes crew who stayed with us through the whole roller-coaster ride: Lily Chi, Binh Ngo, Paul Lee, and Susan Nakasora. Also, thanks to the many other friends who lent a hand during our startup years.

Thanks to Larry Barber, one of our first Design Reactor clients and an early business advisor. Larry took a look at Patrick and I when we were young, eager 21/22 year old entrepreneurs and unfailingly believed in our potential. Thanks to Larry’s daughter, the late Cara Barber Hamm, who staked her early career at both Disney Channel and Warner Bros. to bring us in to the fold and giving us the first opportunity to shine. Thanks to Brian Bowman, who later came in later to head both DisneyChannel.com and ABC.com and helped us grow Design Reactor from a startup into a real business. Without Brian, I wouldn’t have spent New Year’s countdown of 2000 hard at work programming the “Who Wants to Be a Millionaire” Online Game for launch the following week. The extra-large invoice from that special overtime probably eventually extended Rotten Tomatoes’ runway by a year or two later on when cash was running short.

Finally, I want to give a repeated thanks to both Senh, who came up with the idea for Rotten Tomatoes, and also Lyle Fong, founder of Gamers.com which eventually turned in to Lithium. Both are immigrant entrepreneurs who had the innate drive to build something great that they could claim for themselves. Without the two of them, I might not have been inspired to take the big step of walking away from Design Reactor and Disney to also pursue my dreams of building something of long-lasting impact and value like Rotten Tomatoes

Share

Three times that Steve Jobs demo’d Apple products on-stage with Rotten Tomatoes

I thought I’d first contribute clips of the three times that Steve Jobs demo’d new Apple products using Rotten Tomatoes

The 20th anniversary of Rotten Tomatoes is rapidly approaching. I’m going to begin contributing some of my memories of starting up RT in the early years in future posts this month, I got back last month recently from my very first Apple WWDC event, so I thought I’d first contribute clips of the three times that Steve Jobs demo’d new Apple products using Rotten Tomatoes. #RottenTomatoesTurns20

Share

Help build a better WeChat

Please help my efforts by spreading word that WeChat is hiring in product management positions — Chinese language skills not necessary!

I’ve been on a tear recently trying to improve understanding of WeChat and the WeChat lifestyle in order to seed the ground for better talent recruiting into our growing WeChat team. I’ve even launched a new audio podcast talking about what it’s like to work at WeChat!

Please help my efforts by spreading word that WeChat is hiring in product management positions — Chinese language skills not necessary!


WeChat Product Manager
Location: Guangzhou, Guangdong, China

About WeChat

With over 600 million monthly active users, WeChat has become one of the most impactful mobile products on the planet. Since it’s start as a chat app, WeChat has grown to become a market-leading platform by connecting people services, and businesses in China and around-the-world.

Job Description and Responsibilities

Rather than just reading about WeChat innovations from a blog, we invite talented mobile product developers and product managers to learn and participate firsthand by joining the WeChat Team. Learn about the WeChat product philosophy and the mobile lifestyle as a WeChat product manager. Joining the WeChat team gives you an unique opportunity to help redefine how people use their mobile devices to communicate and interact online and to better understand user behavior and preferences in the world’s largest mobile market. The Product Center team within WeChat is responsible for managing and growing our core product including messaging and social networking and you would be working along side WeChat’s distinguished founding team.

An ideal candidate will

  • Demonstrate and communicate a passion for mobile apps and the mobile lifestyle
  • Provide a portfolio of online or mobile products that you have helped design or develop
  • Give knowledgeable insight into mobile user behaviour and psychology
  • Have discriminating taste in mobile app design and an obsession for superior user experience
  • Demonstrate a firm technical comprehension of web and mobile client-server architecture
  • Have an open-minded, forward-looking outlook by temporarily moving to China and immersing yourself in a fast-moving, mobile-obsessed culture
  • Have two or more years experience in mobile apps development or design

Benefits

  • Competitive compensation and benefits compared to other major technology companies
  • Relocation expenses covered
  • Direct mentorship by senior members of the WeChat product team
  • Contribution from Day 1 to improving a product used by over 600 million people
  • Wish to apply? Send your resumé and a cover letter to [email protected].
Share

BarCamp Beijing: “Growth Hacker” presentation

I recently delivered an impromptu talk on “Growth Hacking”. Rather than speaking of specific, current tactics, I decided to give it a more long-term strategic views on how to approach product marketing from a “hacker’s” perspective. Rather than just talking about “viral loops”, I wanted to take a look at a broader perspective of how to take advantage of dominant distribution channels and to use them in under-utilized or unintended ways to grow your own product. I’m not a growth hacking expert (see the experts who I link to at the end of the presentation, but I think it’s relevant to everyone who is launching a new product whether online or not.

BarCamp Beijing: Growth Hacker

Share

Alive.cn covered in the press and my podcast interview

It’s been a radical summer building out Alive.cn. It’s been a while since I’ve been back to the nuts and bolts of startup life doing everything from programming to hiring to sales to taking out the trash. I’ve had my partners Patrick and Raffi living in my two bedroom apartment for weeks as we rapidly build out the new service together. Guess who gets stuck on the couch…

To that end, it’s good to finally see some coverage of Alive.cn in the press. First, here’s an article that Raffi did with Marketing Interactive introducing the new service.

In addition, I just did a podcast chat with CRI radio host John Artman about my entrepreneurial experiences. His own podcast is just getting started so the talk is all about “startups”. The chat is about an hour long — not sure who in the world is interested in hearing me blabber for that long, but if you wait till the end, you WILL get to hear my billion dollar new startup ideas.

Download: Podcast interview with John Artman

Checkout more China internet talks with John Artman

 

Share

An inspiring story from Pixar’s Oren Jacob

A great story on Techcrunch today regarding Oren Jacob, a former CTO of one of my favorite companies in the world, Pixar:

“… In the winter of 1991, Pixar had some major layoffs and the company shrank to less than 40 people. Oren was still an intern at the time and had attended both the “you’re being laid off” meeting and the “you are the few who are staying” meeting and decided he liked the latter one better. He went home for the rest of the week, didn’t attend any exit interviews, didn’t turn in any keycards and just showed up for work the next week to a company that was now focused solely on animation.

The guy who dealt with keycards never deactivated his key. Oren kept showing up. Folks were surprised to see him in the hallway, but he played it cool and nobody raised a fuss. Many months later, paychecks started showing up in his box again and continued for 20+ years. Over those 20 years Oren worked obsessively hard and rose through the ranks of the company to CTO.”

Here’s a guy who is passionate and won’t take ‘no’ for an answer. The story of how he got INTO Pixar is equally amusing. Plus, he was inspired to get into animation and join Pixar at the SAME “Spike and Mike’s” animation festival screening of Pixar’s early short “Tin Toy” that my middle brother and I attended way back in 1988. My brother has gone on to do movies … I’m equally passionate (if not more) about movies as I am about online startups but I guess I took a different fork in the road than he did when I went to college…

Incidentally, way back in the very early days of Rotten Tomatoes, we used to monitor our incoming traffic on movie release days to see if we had any online visits from the studios. This was even before Rotten Tomatoes was a real company and was just a hobby site that our partner Senh was working on and I helped to host and provide technical assistance. One of my fond memories was during the release of “A Bug’s Life” when we noticed a flood of visitors with Internet IP addresses from Pixar visiting the site. What was funny would be that we could see these same visitors would continually hit “refresh” on the Rotten Tomatoes reviews page for “A Bug’s Life” meaning that they were even more obsessed about the Tomatometer score than we were. What a thrill for our “newborn” web site. At the time Senh, who came up with the idea for Rotten Tomatoes, was pulling all-nighters just to keep the site up-to-date with the latest reviews, but we were all pretty excited when we saw the Pixar visits. Being huge Pixar fans, we had bought tickets in advance to see the film on opening night at Oakland’s Jack London Square AMC theater. I guess the excitement was too much for Senh because he fell asleep right in the middle of the movie — not an indicator of the quality of the movie… he was just was too tired from pulling an all-nighter collecting reviews for the movie the night before.

Anyways, Pixar was in Richmond at the time and would eventually move to become our neighbors in Emeryville and we would go over there to visit our friends and ex-coworkers there on occasion. The most memorable visit was to award Brad Bird the Golden Tomato award for “The Incredibles“.

Go Pixar!

Share

mongoDB Beijing: My presentation on alive.cn and building a new entertainment database

While I’m still relatively new to mongoDB, I’m taking the opportunity to give some insights on building a new multi-lingual, comprehensive entertainment database using linked open data. The presentation will go through an evolution starting with the early days of Rotten Tomatoes when we assembled the movie information manually to my current efforts with Alive.cn.

I wanted to invite technically-minded Beijing folks again to a presentation that I’m doing on Thursday at the mongoDB conference. While I’m still relatively new to mongoDB, I’m taking the opportunity to give some insights on building a new multi-lingual, comprehensive entertainment database using linked open data. The presentation will go through an evolution starting with the early days of Rotten Tomatoes when we assembled the movie information manually to my current efforts with Alive.cn.

mongoDB Beijing Conference (Thursday, March 3)

I’m still not certain yet whether I’m going to deliver my presentation in English or in Chinese. Obviously, I’m much more comfortable speaking English, but would like to make sure that the audience is getting the message correctly. In any case, I’ve presented both English and Chinese versions of the presentation below. I decided to go with a movie theme in the visuals throughout the presentation to keep things in line with my “entertainment database” topic.

Looks like some of the presentation fonts and layout didn’t get transferred too well with the upload to SlideShare, but you can get the general gist below:

Building a super database from linked data

用互相关联的数据创建超级数据库

Anyone care to share some tips on presenting at a conference?

P.S. Thanks to Terry, our awesome UI/UX Engineer for helping me translate the slides and also, of course, for the awesome still-in-progress design work on alive.cn.

Share

Speak of the devil… build your own OpenCalais like supermachine

In line with my recent blog mentioning OpenCalais, the topic extraction tool, DBpedia, one of the awesome linked open data projects I’ve been using a bunch for Alive.cn, just released their own topic extraction tool, DBpedia Spotlight.

In line with my recent blog mentioning OpenCalais, the topic extraction tool, DBpedia, one of the awesome linked open data projects I’ve been using a bunch for Alive.cn, just released their own topic extraction tool, DBpedia Spotlight. If you are okay with downloading 9GB of Lucene indices and setting up their scripts, you can have your own self-hosted topic extraction tool. They basically open sourced something that is worth a lot of money in a previously relatively closed space.

What is topic extraction? Check this demo out and enter any block of text — say, a recent news article. The benefit of using DBpedia’s solution (besides it being free) is that it automatically ties topics back to their DBpedia topics which already have a huge storehouse of Wikipedia-derived linked open data.

Share

The Man versus Machine Jeopardy Challenge

It’s the probably the most public test of the advances in linked open structured data and semantic text analysis, I’m really following closely this tournament pitting IBM’s super-computer Watson against the two most successful Jeopardy champions.

Jeopardy Feb. 14 2011 – Human vs Machine IBM Challenge Day 1 Part 1/2

It’s the probably the most public test of the advances in linked open structured data and semantic text analysis, I’m really following closely this tournament pitting IBM’s super-computer Watson against the two most successful Jeopardy champions. I suspect that they’re using the same publicly available data sets that we’re using for constructing Alive.cn.

I wonder, however, why they chose to rely only on electronically fed questions rather than going the final mile and adding a voice recognition interface on top of the system. Voice recognition accuracy has gotten so good these days, but I wonder if the final few percentage mistakes makes a critical difference against the best human players.

There have been some other truly AMAZING projects in this field. Two I’d like to highlight:

  • Google Squared: This Google Labs experiment is an amazing mash-up of topic extraction and turning unstructured web data into structured data. Simply type in any category (example: “Chinese Emperors”) and it will bring you up a spreadsheet of items in that category and some properties. Next, you can add your own properties (“Inventions”) and it will automatically fill in the results using searched data from the web converted back into structured data. It’s truly one of the most remarkable things to come out of Google, but a bit more work on it (say, a voice recognition interface) and it could be a mainstream breakthrough.
  • OpenCalais Topic Extraction: Another semantic analysis tool that will pull out “topics” automatically and link them against linked open data. Try out the free demo and copy-and-paste a news article. After submitting the article, you’ll see it has linked together topics on the side automatically.

Like I’ve mentioned before, I feel that we’re right on the tipping point in the next several years where there will be advances in knowledge extraction and interpolation that will have a revolutionary effect on everything including how we interact with computing and having exponential advances on data forecasting. Projects like Wikipedia (an unstructured data source) are just the beginning.

P.S. My favorite comment about the Man versus Machine Jeopardy contest: “Why couldn’t they have programmed Watson to use the voice of Sean Connery?”

Share