First off, I am sorry for all of the recent website issues. We are actively working on it. Thank you for your patience.
Growth & Limitations
Over the past few years nearly every stat has doubled, and we have finally maxed out our server hardware with the current database architecture.
One of the things that has made us so successful is that our site was built from the ground up specifically for card collectors. We don’t use any pre-canned shopping cart software because none of them are designed to handle millions of unique products or offer all of the specialized functionality that our users have come to love. This also means that we are often pioneering new frontier, and we discover our limitations and issues as we grow.
New Architecture
I started to see the writing on the wall last fall when DDearing was trying to update 50,000 prices every day. So I priced out new server hardware. Unfortunately it was going to cost about $100,000 for us to make a significant hardware upgrade. So I took a very close look at our database architecture to see if there were ways we could be more efficient with our hardware.
It is actually quite amazing that the core of our database architecture has held up from its original design when we had only a few thousand items to now with more than 3.7 million items. Fortunately I found some opportunities for significant savings. For example, we currently store about 1,000 bytes of historical data every time an item is modified (e.g. asking price changes, ownership changes, book price changes…). I designed a new storage schema that could reduce that to only about 50 bytes of data for the most common data changes. This should mean that we can process much larger transactions very efficiently. However, this will be a pretty big change to implement. Since we didn’t want to rock the boat for the critical holiday season, I shelved these efforts until January.
Since mid January I have been finishing the design and porting our data to the new system. Now I am nearly to the point where I can start comparing the performance of the new architecture to the old one. However… issues on the current site continue to mount, and over the last few days it has been virtually unusable for many users. To keep the site limping along, I experimented with some temporary fixes. Some of them have helped a little, but ultimately it has just reconfirmed that with the current architecture our hardware cannot really serve more than our 8,000 daily visitors.
Search Engine Crawler Traffic
It used to be that Google made up about 90% of the crawler traffic on our site, and they were very responsible with how they did it. Now we are getting crawled by yahoo, bing, google, and many others. Unfortunately they are not all as responsible as Google. Today I just discovered that that Yahoo Slurp engine has been paging through our data 12 items at a time. Each request has been pegging our server for about 30 seconds. So Yahoo alone was basically monopolizing 1 of our 8 processors.
To address this issue, I have updated our robots.txt file to tell search engines not to page through our data. Instead they should be using our sitemap.xml to get all of the data in the most efficient manner. This should free up server resources so that more real visitors can use the site.
Search Scalability
Right now the thing that taxes our servers the most is all of the searching and browsing of cards for sale. We are investigating the possibility of offloading the generation of search results to a farm of servers that will have access to their own cache of the live data. If we can move this to the cloud, we should be able to scale the site to handle virtually any amount of traffic.
In summary, there are several opportunities we have to make the site performance much better, but none of them are quick fixes. Our main focus is on these major improvements, but we will continue to make minor adjustments to the current site if people are commonly experiencing website errors.
Thank you for your patience and understanding. We are all experiencing the growing pains of a successful startup.
Tim Getsch
Check Out My Cards Founder, CEO & CTO
Seriously, you WANT these kinds of problems because their root cause is your success. These are the problems that Friendster had, and MySpace was able to take over. Friendster was built around nothing. Anyone could replicate it. Such things are not an issue here. Hang in there, and keep us posted, even if it’s just to say that nothing is new. Once a week should be enough 🙂
My Only complaint would be that when it went down yesterday I was trying to price cards. I’ve only got limited time to work on this and really advance warning for things like this would be great. Hopefully the fixes will help as I cant spend 1-5 minutes pricing each card.
They always say ….”Growing Up” can have some painful experiences.
Congradulations on your part!! ….you’ve created a very successful enterprise!!!!
I’ve know I am having problems navigating around the site ….but patience is the virtue and I’m waiting.
Love this site ….just keep us posted on the progress …:)
The motto, “for collectors by collectors”, lives on in the detailed explanation to the slow down we all experienced; thank you for explaining. The truth of all this is that we look forward to the progress made to date and the progress made daily as described in this post. I am truly excited at the opportunities that will be available, since these challenges appear to be no small feat financially, they will be managed intelligently.
If needed I employ a 12+ year Network/Security Administrator that can consult/share advice if needed for no charge (for consultation, no strings and no sales pitch). He is much respected in our IT group for our Cloud Migration from our Server Room freeing up the data hogs (like Yahoo Slurp Engine) that were slowing down our servers.
Whatever the changes made, I am sure they will meet with the current and future requirements of our highly enjoyable web site.
Cheers,
Jerry
(COMC-joined 2009)
I am with Jerry on this one, “for collectors by collectors”. I am one of the original consignors of the site having started in November 2007. I have been a continuous buyer and seller ever since. I have truly seen the near full progression of the site. While the recent site slowdowns are a bit frustrating at times, the root cause, exponential growth, is not a bad thing. Thank you for treating us with respect and giving such an honest and detailed explanation.
I’ll say it again that I think a major draw on COMC’s servers has to be the fact that COMC is basically a free subscription to all of the Beckett price guides. I went to a big card show this weekend and saw a seller looking up values on his laptop on COMC when someone asked for a price! Why look through a magazine or get an online subscription to Beckett when you get fast free info on COMC? This must be repeated thousands of times daily when someone wants to compare prices on an ebay item. Even Beckett’s clunky Marketplace doesn’t give you values – only what sellers are asking. Perhaps if non-registered visitors could only see what sellers were asking (or maybe only a percent off) – then that would deter visitors who use COMC solely for pricing. Even taking this idea further – book value could only be seen by members who had a small amount ($5?) in their store credit account. That would further alleviate COMC’s available resources.
I think COMC has shown they are pro-active in their approach and things will get better soon. Growing pains is a good thing in this case. We don’t want to alienate potential buyers/sellers but there is a portion of COMC visitors that just use the site for pricing and do not intend to buy.
I totally agree with you on this one …..there should be a limit on non-registered visitors.
Enough for them to see, but enticing enough to make them want to join.
there are great points. I bet now that becket has raised their online guide prices and have messed up their site again, more people may be coming here to check pricing for those that still go by the book. Not everything is here but there is enough to go on. I can’t imagine how many non users are using comc for that exact thing. You could def. look at giving partial access. I don’t know if you are allowed to try and charge them to get full access or like stated above make they have a few bucks in a user account.
But if they’re going to put limits on non-regs (which I’m fine with), they need to implement a way for a reg’d user to stay logged in for a set duration (day, week, or maybe up to the user). I pop on the site once a day or so to see if there’s any new listings for what I collect. If I have to log in each time to see percents off or what have you, that add a small bit to the server load (going to the login page, then going back to where I just was).
Pingback: COMC Site problems - Blowout Cards Forums
Hey i understand the issues . So no problems here I’m happy with the site and all the things you guys do. Your getting bigger and i hope that the way you handle us stays the same with you growing. Thanks for all you guys do.
All I can say is thanks! I’m always sending people to the site and now I can even send more. You guys are wonderful and I say it over and over again! WE APPRECIATE YOU ALL.
If you want to cut down on non-registered visitors, one possible solution would be to hide the Beckett price and possibly the percentage off guide from people who aren’t logged in. You already have the mechanism to put ads on those pages. So, instead of a Beckett price they might see “log in to see Beckett price” or “log in to see percentage discount”.
Personally, I think COMC is a better price guide than Beckett, because you see the actual cards people are willing to sell and the prices they are willing to sell them for. If there’s 50 of a card being offered by 20 different dealers for 95% off book, then it’s safe to assume that book value might be a bit high. OTOH, if there are none for sale or only one, and the asking price is between high Beckett and low Beckett, then the Beckett price is probably about right.
Since COMC started running ads, I’ve frequently surfed the site without logging in. The idea was that COMC gets money from ad impressions and so I wanted to help out, even if it’s only a few pennies here and there. Is this beneficial to COMC, or would it be better to log in and surf as an officially logged in member?
What kinds of ads are you guys seeing? I’d click on anything about fantasy sports, pizza or magazine subscriptions if it helps the bottom line. I’m talking conversion, too. It’s time to start figuring out whose service to use for fantasy baseball drafts!
You guys are doing fine. Keep up the great work!
Pingback: CheckOutMyCards.com Questions, Suggestions & Testimonials - Page 42 - Blowout Cards Forums
Thanks for letting us know this. However, I think that overcoming the problems the website has been having is something you must do, and do very quickly. I’m afraid if the customers are turned away by a slow non-responsive site (Think Beckett.com) than business will drop pretty quickly. I am praying for you and hope you get everything straightened out soon.
I totally agree. To see prices or book values, people should have to login. My dad used to say there is no free lunch. I would be willing to do this to login process since I am a buyer and occasional seller on the site. I am a collector AND I have a Beckett subscription too!