Root Access: Safely migrating a large web app to a scalable platform, with startup Ninua

Root Access: Safely migrating a large web app to a scalable platform, with startup Ninua


MARTIN OMANDER: Today
I’m joined again by Waleed Abdullah,
founder of Ninua. Thanks for coming
in today, Waleed. WALEED ABDULLA:
Thank you Martin. MARTIN OMANDER:
So for viewers who didn’t watch previous
episodes, would you give a brief description of
what Ninua, your startup, does? WALEED ABDULLA: Sure. So we two products, Symphony
at symphonytools.com. It’s a social media
management dashboard for businesses to
publish their content to multiple social
networks and to listen to what their customers
say about them on these social networks. And the second product
is Networked Blogs, which is a platform for
bloggers to promote their blogs and interact and network
with their readers and other bloggers and has
about 1.2 million blogs on it. MARTIN OMANDER: Great. So Waleed, in
previous episodes, we talked about
business challenges. And for this one, let’s
talk about technology, a topic that’s very near
and dear to my heart. I’m an engineer. WALEED ABDULLA: Same here. MARTIN OMANDER: You too? WALEED ABDULLA: Yeah. MARTIN OMANDER:
All right, great. So tell us a bit about your
first technology platform that your app was built on. WALEED ABDULLA: Sure. Networked Blogs started
in the hackathon. And for me it was a side project
to learn about the Facebook platform at the time. And most of the examples
were within PHP and MySQL. So I thought, OK, I’ll
just build it with that. It wasn’t planned to
be a long-term project. And it ended up being
a long-term project. But that’s how we
started, so PHP, MySQL. MARTIN OMANDER: Great. And how was the traction
of the platform? How many users did you get? WALEED ABDULLA: It started
growing pretty fast from the beginning. And, at some point, we were
adding about 1,000 new bloggers every day. MARTIN OMANDER: Wow. WALEED ABDULLA: We have
about 1.2 million bloggers and about 7 million users on it. So, yeah, over time, it grew
much bigger than I anticipated. MARTIN OMANDER: That’s great. WALEED ABDULLA: Yeah. MARTIN OMANDER: A
good problem to have. So what happened with
the technology platform with these additional users? WALEED ABDULLA: It
didn’t scale very well. In the beginning, it was great. But over time, the server
was kind of at capacity. And then we added another
server and then a third server. And it wasn’t that stable. I was learning how to scale
a product as I kind of work on it. It would crash occasionally. It would crash in the
worst times possible, like middle the
night or when I’m out watching a movie with friends. And I would have to
drop whatever I’m doing and just go fix it. MARTIN OMANDER: You won’t
be invited to movie night again with those friends. WALEED ABDULLA: Yeah. It was a kind of running
joke about– yeah. MARTIN OMANDER: So
what did you do? You can’t have your
product going down or you’re having to wake up
in the middle of the night all the time. WALEED ABDULLA: Yeah,
that wasn’t fun. So I tried to scale it on PHP
and MySQL for a long time. And it kind of worked. But there was a lot of work. And there was a lot of time
spent on the scaling as opposed to like improving the product. So at some point,
I decided that I’m going to move it to
Google App Engine. I had experimented
with Google App Engine before that on other
small, concise projects. And it worked really well. So I kind of just made the
decision to actually move it. And then it took a
while to actually move. But once we did that, that
solved all the scaling problems. MARTIN OMANDER: It
sounds very easy when you’re saying it
now and after the fact. But this must have been a
very big decision for you. Were you concerned
about being locked in? WALEED ABDULLA:
So that definitely was one of the concerns. One of the experiments I did,
at the time, was to kind of test the theory of how would it
work if I wanted to move out of App Engine, like,
am I really stuck? So there were a couple
Open Source Products. One was called
typhoonae, and the other was AppScale that allows you to
take your app engine up and run it without modifying the code. You can run it on
Ubuntu, for example. So at the time, I used typhoonae
and run this experiment to see if I can take the
app and run it on Ubuntu. And in one day, we
managed to do that. And we had an application
that was working fairly OK. That wasn’t production
ready, but it was working. You can track through it
and navigate and register and, you know. So that gave me
a good confidence that, OK, if the need happens,
in the future, to move out, it is possible to take
the same source code and run it on a
different platform without a lot of modifications. It might take more than a day,
obviously, probably, maybe a week, maybe even a month. But still, at least
it’s possible. So that gave me
this comfort to say, yep, I can take
this [INAUDIBLE]. MARTIN OMANDER: I see. So then did you end up porting
the system to App Engine and decide on a particular D-Day
and switch everything over? WALEED ABDULLA: No. So the app was fairly big, a
lot of pages, a lot of features. And we had a lot of traffic. Our database was really big. And it wasn’t practical to just
rewrite everything and then just switch over in one evening. So what we did was, more
of a piecemeal approach. So we’ll take one page,
migrate it, and after a while we take another. Whenever we have to
update a certain feature or kind of make a
big change to it, we’ll take the opportunity to
rewrite it as well in Python and we moved it to App Engine. So we did it one step at a time. MARTIN OMANDER: Hmm. So I guess that approach
meant that you could do it safely and at your own pace. WALEED ABDULLA: Right. MARTIN OMANDER: But it also
must have led to complexities of having the system now
living in two places, and the data living in two
places, or how did you do that? WALEED ABDULLA: Exactly, yeah. So there were tables that
were, like database tables, that were not changing often. So we simply just have
two copies of them on both platforms. For other tables that change a
lot, we would replicate them. Or we would have
one master table and all the data is there. So, for example, where
we store the articles, we just put that on App Engine. And then whenever a
page that’s on PHP needs to access that
data, we created an API that simulates the
MySQL interface. So a new PHP when you call
MySQL and you get records back, what this does is kind
of takes the request, sends it to App
Engine, gets the data, and then formats it in the
same format that MySQL returns. So the code runs
as if it’s actually pulling the data from
MySQL but actually was pulling it across the
internet from App Engine. And it was fast enough. And we managed to do that,
kind of migrate slowly by doing this kind of abstracting
the data layer. MARTIN OMANDER: Great. All right. So now that you’ve
moved everything over to App Engine, what were
the costs now that you had the full, what was it, millions
of users and huge changing database, how did
the cost work out? WALEED ABDULLA: So I think the
cost was a little bit higher. It’s hard to tell exactly. Because we were also growing. And we had more traffic. So it’s hard to tell. But my sense is that it
is a little bit higher. But at the same time, I save so
much time, and I sleep better. And just the amount
of time that I used to spend on having to
deal with that memory issues and scaling and the CPU, you
know, whatever, all the scaling related issues are all gone. So I just focus on actually
building code and releasing new features. And that’s very valuable. MARTIN OMANDER: Thank you
for sharing your experiences with us, Waleed. And good to hear that you can
go and watch movies with friends nowadays. WALEED ABDULLA: Thank you. MARTIN OMANDER: You’re welcome. That’s it for today. I hope you found this, what
we were talking about today, useful for your startup. If you have comments, please
log them under the YouTube comments section. We love to get comments. And we read
everything you write. Finally, if you
are in a startup, please go to
developers.google.com/startups to find out how Google
can help your startup. That’s it. See you next time.

Danny Hutson

Leave a Reply

Your email address will not be published. Required fields are marked *