[Part 2 of 2] Joshua and the (sprint) juice

Hey everybody, how's it going? It's your favorite Simplero CEO here back in the hot seat.

This morning we were talking, on the team here, we do what we call it daily huddle.

So we meet at 11:00 AM eastern, that seems to fit both the people in Europe and the people on the west coast and everything. Meet for 15 minutes just to check in with what everybody's doing for the day and if they're blocked on anything and just kind of coordinating so everybody has a sense of what's going on on the team and we just get to connect with each other since, obviously we're running a virtual team and so it's nice to just get to sense each other. And people that have watched the Facebook live I did the other day about momentum and juice, getting the juice going and Josh aka Joshua the elder, our latest addition to our development team, he had a really wonderful insight.

He was talking about going for a run last night and how he fell that kind of tired ... as you can. It's like ... And then he was like, "What if I run faster instead and just go with more power?" And what I would say is you add engagement, right? So that was a key that I forgot to mention last time. I'm still trying to put this together, which is great. It's just, you share stuff and then people reflect on it and they add to it. So that's exactly ... I couldn't have asked for anything better. So engagement, right? When you're feeling like ... well, no matter what you do, make sure that you're fully engaged. If you're half engaged and you're slogging and you're there, get that engagement going, it requires more energy, but it gives you energy, right?

When you do that, when you engage, it gives you energy to keep going. And it's like when you're running, you feel that spring in your step again. You're going a little faster, a little more powerful. It's more fun. It's more fun. So it uses more energy, but it gives you energy and it gets you going faster and, again, it gives you that momentum. Right? I've always felt like when you're bicycling and you're doing ... I live near to the Williamsburg Bridge and I bike into the city when I go there into Manhattan and it's hard, especially when I've been, I've been doing like squats or lunges or something the day before. It can be hard to getting up that hill, but I find that if I put the energy in there and just go hard at it, then I feel like it uses less energy than if I just go into kind of a lower gear and just like ....

It will get me up there. But because I'm not going, I'm going so slow. It just feels like you're spending all your energy just overcoming inertia and ... not gravity, but just to get the thing moving. You need to put in a certain amount of energy and it feels like adding that extra layer, everything that you add on top of that base level, that gets you moving forward.

So engagement, engagement, engagement is key to getting that juice going and getting the momentum. So work with that guys.

All right. Wanted to also give you guys a quick update on our video and coding infrastructure. So Simplero obviously hosts videos for you guys for your content and when you upload a video, we need to take it and process it. So we know it all fits into a standardized format and it's optimized for low and high and in between bandwidths. We make an audio version, we generates screen grabs so that you can slide through and kind of see ... find your position in the video. We grab a bunch of stills to use as the poster. And so we've made a bunch of changes to that recently. So we have ... the way it works, the way we have it set up is now we handle all this kind of in house. We have our own servers ... well we don't have our own servers, but we have dedicated servers at Amazon at AWS that does this.

Originally we were using a service called Zen Coder where they handle all the infrastructure and you tell them what you want, and then they charge by the minute of video.

That wasn't economical for us because our videos, people upload very, very long videos, an hour and a half, two hours, calls and whatnot. And so paying by the minute, plus you pay for each version that you make. And then when you make HD, they count double. So all of a sudden like one ... even though it's not a ton per minute, it still ends up costing a lot. So let me ... I can show you guys here. So this is encoder that we used. So it's basically $.04 per minute. Something like that $.04, zero per month, so you can get it down to $.02 per minute. So that means if it's 100 minutes, it's $1, but then times each version. So let's see, normal plus the other one that's $2 plus HD, that's four bucks and then an audio version, et cetera.

And so five bucks for each video. And then if you delete it five minutes later, we've still paid that. And it just adds up. It was thousands of dollars each month.

And we've grown dramatically since we quit using them. It just wasn't economical for us.

And so we're like, should we make fewer versions so that we can't support bandwidth size? Not really when we want to do. Right? So we decided Amazon has a similar service called Elastic Transcoding. You can have them do it. The were like, "No, let's make our own. We can do that much, much cheaper and we have way more control." So that's the other piece is, we can go make these tweaks. So we found that scaling, when you do the sizing, just the number of pixels dimensions, because of the kind of content that people typically upload here, which is slides and things like that.

The algorithm that you use for scaling matters a lot. Excuse me, I've got to sneeze. All right. Bless me.

So we tweaked the scaling algorithm there so that it works, you get crisper slides, things like that. They're creating an audio version on that. You can get them to do that. But the way it is now, I just have to figure out how to get FFmpeg, which is a software that everybody uses for this stuff. How do we get FFmpeg to do what I want to do versus figuring out how to get Zen Coder or Amazon to get FFmpeg to do what I want it to do. Right? So more control. What happened last week was that all of a sudden ... so the way it works, you upload a file, it's stored in S3, our encoding server immediately goes, grabs that file from S3 from the storage. Downloads it, does all its processing magic, creates all these different versions plus the mpeg dash, all that stuff that makes it so you can download and in chunks and all that gunk, all that stuff. Creates all that stuff. And then uploads it back to S3and says, "All right, now all the files in place, it's there, you can start serving it."

Last week we started having problems. We had two things that happened at once. One is we have a big customer who signed up and who started uploading a ton of videos. And so they were kind of hogging all our processing power. Because that's the downside of the setup that we have is that we have limited capacity. We only have so much capacity. We do have a spare server that we can turn on, but that was a manual process and it costs us a lot of money to have that thing running because it's a beefy monster. And so we don't want to do it unless we need to, because we want to spend the money on things that make a difference. There's no point in having to spending money having a server just sitting there waiting to maybe do some things someday that's not us being ... if we did that, we wouldn't be good stewards of the resources that you give us that enable us to create a better product for you.

So that's something we're conscious of.

So anyway, we had one customer who was started uploading a bunch of videos, which is great. We want you to but obviously then that person is using all of our available bandwidth and nobody else can get any ... or not bandwidth in that sense, but all the processing power that we have and nobody else can get anything done or get any videos encoded, they just have to wait. But the other thing that happened was that all of a sudden networking within Amazon, between S3 where the files are stored and we're the EC2 where the computers, the servers, the do the processing, ground to a complete halt. It's worked great for ... when did we move in 14, 15 something years ago? We've switched years ago over there. It's worked great ever since. But all of a sudden it's just completely ground to a halt. And were like, "It must be temporary. I'm sure they're about to fix it." You can't really reach out to anyone in support, at least not without, again, submitting the first support ticket would cost us at least five grand if not more, closer to 10 grand because of the way that they price their support. We're like, "Yeah. Sounds like a little expensive for one support ticket." I'm so sorry guys. I need to blow my nose here.

See if I get all the white stuff in my stubble here. All right. Yes, so we're like ... gave it a few days. It seemed to be like it was in fits and starts. Sometimes it was great, sometimes it was just impossible again. Then we tried to brainstorm all kinds of things, started looking for people that we get a hire to help us figure this out. And then finally we were just searching and searching and then we came across this thing that you can actually ... there's some stuff that you can do inside of AWS that says, "You know what? I actually want it to be fast to connect to S3 from the server." I'm like, "Let's turn that thing on."

Turn it on and it did the trick. It just made it fast.

So apparently that's the thing now that you have to tell it to be fast to reach within Amazon. So that's great. So we that, and so now that's working really, really well. So everything is humming along. So in the meantime, we created this system that automatically turns on. I don't know if I can show you guys. This is our internal monitoring tool. Right now there's two jobs in the queue, nothing's waiting. Ink too is the extra server that we have. So we can do eight at a time on ink one and ink two can 16. And so we've built this system now that automatically turns on the extra server whenever we need it. And then when it's doing everything we wanted to do, then we tell it to just keep finishing what you're doing and then shut down when you're done with all that stuff.

If more videos come in, we can tell it, actually start processing new jobs as well again. And then eventually whenever it's done with what it needed to do and we don't need the capacity, we'll shut it down. So one server's always running and the other we can turn on, on demand. And we find that we run out of bandwidth even with both of them running, which means we can do 24 encoding simultaneously.

We can always add another one of those and just do the same thing. So if ink two was maxed out and we still have a lot of stuff to do, we just turn on number three and we can keep doing that for forever. So now we have that dynamic capacity set up that way, just homegrown system to manage that. We also did a bunch of other little tweaks so when your video is queued will tell you that it's queued and we'll tell you what place in queue it has so that it's not that things are broken, it's just that all the servers are busy. Spinning an extra server, the second instance do to help out takes less than a minute.

It gets up pretty quickly and then it just starts chugging along a jobs right away.

What else? We also made some tweaks to the frame that we grabbed. For the still frames, we grabbed the first and the last, and then we grab a number in the middle depending on how long the video is, just kind of spaced out. And then by default we will always pick the first frame as your still poster frame, you can choose another one. But then a lot of people do a little bit of a fade in, in the beginning. So now we're picking the default where we still grab the very first frame, but we also grabbed the one half a second in and that is now the default is still frames. So we'll see if that seems to work better for people's videos overall. But that's where we're at with that.

What else we have we done?

I feel like just so many little things and tweaks. If we run into network issues again, now I changed it so that we'll keep trying to download a few times just to see if it failed the first time. Let's try it again. And now we're tracking the progress when you're downloading a long file, we'll show how far we've along we are in downloading that large file and as well at the upload at the end, we'll give you progress status updates on that. Just so you know, you can see that things are going on so it doesn't look like it's stuck for a long time while things are actually happening.

And the same thing with if we run into network issues again, after we've downloaded it, done all the encoding work, which can take hours, it can take a long time, even with the fastest server that, that Amazon has available in coding, such a large video can still take hours. And then if the network fails at the end of that, that sucks, right? Because we spend hours and coding this video and then, all right, then we just throw it away and we'll start over. So now instead, it'll retry a few times, uploading it at the end of the video of the end of the process, if it happens to run into a network issue.

I think that kind of covers it.

Anyway, just to a video to show you what we do behind the scenes scenes to keep everything running smoothly and get our internal monitoring tools so we can see what's going on and make sure that everything is progressing the way it's supposed to and everything's running smoothly. So yeah, that was kind of it, little behind the scenes view here. I think that's it for today. I'll talk to you guys soon. Have a fantastic day. I keep forgetting to check if there's a chat situation going on here. 

Anyway, be well everybody, talk to you all soon. Bye.

Calvin Correli.