A little while ago, we promised a post about our development roadmap, as well as what has been worked on in the last year. There is a laundry list of excellent features and updates that have been waiting in our pipeline for what feels like forever. And understandably there’s a lot of questioning on the forums over why they haven’t materialized yet.
Search improvements, ratings improvements, a shopping cart, item support tools, affiliate campaigns and analytics, license improvements and the list goes on. It’s a list of things that would make buying better, selling better, and alleviate problems left, right and center.
The long and the short is that a lot of these types of improvements keep getting put on hold as we cope with growth and expansion.
The search project is an ongoing one. The time spent last year was essentially to change our search server to Solr. As of yet the benefits of this change aren’t apparent on the front-end. That’s the next portion of the search project. It’s always tough having a project that is long, complex, important, but with no visible improvement! But I’m very confident that we will reap huge benefits in the near future from the work last year.
On the scaling front, I checked in with our lead architect Pete who tells me we now have 8 physical machines with 68 cores and 432 GB of RAM , plus 17 virtual machines including 9 front-end servers, and we are serving up some 150 TB of traffic a month. In short, it’s a truckload of horsepower, and just scaling it all is a huge load of work. But it’s paid off, despite the enormous growth in traffic, you will notice that our sites are pretty snappy and rarely run into downtime. Security and fraud also take up quite a bit of time, and the bigger we get the more of an issue they become. Our level of response sophistication keeps rising as the threat level does. We are coping with both the scaling and security and fraud challenges, but it is a large amount of development work every month!