Building things such that they are scalable is one of the core tenets of software development. To begin with, a lot of the value of software comes at its ability to automate tasks and handle copious amounts of I/O that would fry a human brain. Recent events have made me painfully aware of the importance of this value.
I’ve got a friend that I’ve known for the past six years, who has been slowly building up his business over all these years. During this time, he’s repeatedly dabbled in e-commerce but it has only been within the last two years that he’s been investing more heavily into it. The very first shop that he opened was a custom-built PHP solution built by one of our mutual friends. The website was rather rustic, consisting of a few pages which showcased each artist’s work. I remember writing a “micro-framework” for my friend; there was a template product page which stored all the program data as associative arrays and we used URL path information to select which products to display. My friend would use this file manager that we installed on the server to manually manage product details, upload images, and change the source code. Despite not having any programming experience, he actually picked up rather quickly and was autonomous for the vast majority of tasks. The site was served with Caddy 1, which was rather hot at its time due to the seamless integration with LetsEncrypt for TLS certificates. PHP 7.0 run behind Caddy, using FastCGI to process requests. There wasn’t a database or anything; when orders were completed, there would be a redirect to a page that notified my friend about completed sales via email and recorded a row in this CSV file. Fancy stuff. If I recall correctly, payments were handled by either PayPal or Stripe at the time. All of the infrastructure was in AWS. At the beginning, all we needed for everything was a t2.micro.
At a certain point, product and user management were not feasible with such a simple website. While we were capable of handling volumes of tens of orders relatively well, order management, automated emails, and even simple features like product categories were not able to be implemented without significant investment. I was working during those days, so the amount of time I had for some of these ad-hoc developments was rather constrained. It was back to the drawing board to figure out what solutions to use. At that point, the top two contenders were WooCommerce on WordPress versus Shopify.
Ultimately, the decision was driven by the fact that my friend was worried about the cost of running a business on Shopify. The cost was $30 USD per month and the service skimmed additional profits from each transaction. With that in mind, I managed to install MySQL alongside WordPress on the website. The website was still served by the beloved Caddy webserver, and was still miraculously running on the same t2.micro. With WooCommerce, a new window of opportunity opened up with the plethora of features and offerings that running on WordPress enabled.
While the server quietly processed requests, my friend was setting his sights on ever-increasing heights. The traffic reached a point where the default php-fpm settings required tweaking. Upping the settings meant increasing the RAM usage, so the inevitable day came where we had to graduate from the free tier. The next instance type we settled on was the t3a.medium. This was after profiling CPU and memory usage across many chaotic sales. During this time, I’m not sure if I can even count all the times where my friend messaged me asking me to upgrade the server’s resources or restart the server since it had crashed in some inexplicable manner. Of course, whatever moments of peace ultimately ended up being short-lived. There would always be more visitors, more orders. Being unprepared means lost sales, so it’s important to make sure that the server’s ready for whatever gets thrown at it.
At this point, I’ve mostly adopted a vertical scaling strategy. While I had previous evaluated a horizontally-scalable architecture consisting of RDS, EFS, ELB, and ECS, the cost would have been rather prohibitive at the website’s prior scale. Now, there’s a certain flexibility granted as the cost for the single instance has grown a lot faster than anticipated. I’ll probably do some investigation and see how feasible it is. Being able to scale out the application horizontally will ultimately improve the stability of the system and allow for significant savings.
Leave a Reply