Modwest Blog

Internet Successes at Modwest

Modwest hosts websites for thousands of customers in some fifty countries, and I know we host customer sites selling high-def televisions, customizable Belgian chocolates, South American jewelry, and virtually everything in between. We also host gossip sites, faith-based organizations, and business-to-business consulting sites. (This hosting thing is actually a fascinating business to be in.)

Overseeing Support & Operations at Modwest, I most commonly hear about the challenges our customers are facing, and not so much about the successes. But when we do hear from a customer who has overcome a technical challenge, or unveiled their new e-commerce application to the world, or sold 50 televisions in a day, or won a web design award, it's really cool.

The purpose of this brief message is to ask Modwest customers for your success stories. What we have in mind is a periodic post that highlights a specific customer.  Upgrading your site? Just registered your 1,000th user? Just get your grant funding renewed based on website success? Sold your millionth widget? Or just happy to be hosting at Modwest? :) Let us know, and we may share your story with the whole wide world right here on the Modwest Blog.

Email us at feedback at modwest dot com or fill out our contact form to get in touch.

-JM

MIVA Misadventures at Modwest

Modwest has offered MIVA as our e-commerce solution for several years. Despite some administrative complexity (both on the server and client side), we've found it to be the most feature-complete shopping cart application on the market. Recently, though, it's presented some new challenges, some of which have affected a minority of our customers' hosted stores.

Over the last week or two, with about 15% of our hosted stores, we've seen intermittent lock-ups that cause the store to become completely inaccessible. Since there is no good way for us to know immediately upon a store malfunction, sometimes the store owner notices first and notifies us, and sometimes our server administration team notices first. After a great deal of research, we've determined a workaround which frees the store up and allows it to continue functioning, at least temporarily. What we haven't found yet is a way to prevent it from happening in the first place.

While we don't anticipate it'll permanently fix the problem,  we will be upgrading MIVA Empresa on Sunday, just in case it helps (as recently announced in OnSite).

Because our confidence in our ability to reliably provide MIVA software has dipped, at least temporarily, we're suspending installation of new stores until we're confident the situation is resolved appropriately.

Technical analysis of the problem:

Modwest runs a load-balanced cluster of web servers which all attach to a central SAN-attached file server using NFS. There's a concept of 'advisory locking' of files over NFS, which means a file which is locked should probably not be accessed/overwritten by other process, but this is not enforced in the protocol. MIVA, unfortunately, seems to ignore this nuance, and assumes that file locking over NFS is authoritative. MIVA does quite a bit of reading and writing to database files during normal operation, and (as we've now learned) can get itself into a situation in which it believes a file is locked, and waits around for the lock to be freed... forever. This results in a store that never loads for visitors.

Our workaround for digging MIVA stores out of this state isn't pretty, and there's some chance it could cause file corruption and/or data loss, but it does seem to get a store back up and running. Basically we make a copy of /mivadata, remove the original, and then move the copy back into place. i.e.,

# cp -a mivadata mivadata.new && mv mivadata mivadata.old && mv mivadata.new mivadata && rm -rf mivadata.old

It's our hope that MIVA can fix this issue, but we're also working on a Linux kernel module which, if we can do it, might help avoid this situation in the future. In the meantime though, we won't be installing new MIVA stores. We'll post a followup as soon as we have more information.

-JM