692

November 13th, 2023 × #caching#performance#stale-while-revalidate

You Need Stale While Revalidate

Scott and Wes explain the stale while revalidate caching technique, when you would use it, and how it allows you to serve cached content while asynchronously generating fresh content.

or
Topic 0 00:00

Transcript

Announcer

Monday. Monday. Monday. Open wide dev fans. Get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing, soft skill, web development, the hastiest, the craziest, the tastiest web development treats. Coming in hot. Here is Wes, Barracuda, Boss, and Scott, El Toro Loco, Tolinski.

Scott Tolinski

Welcome to Syntex. On this Monday, hasty treat.

Scott Tolinski

We're gonna be serving you up some stale, takes and then revalidating them. So we're gonna be talking about stale while revalidate, which is a caching technique that is really pretty simple to add to your projects to really improve overall speed And really make life easier on you when you're trying to get some caching down that, well, still requires information to be up to date. My name is Scott Talinski. I'm a developer from Denver. With me, as always, is Wes Bos.

Topic 1 01:05

Centux Replay for caching error handling

Scott Tolinski

But before we get into the contents of this episode, If you're, you know, caching anything, you gotta make sure you have some error and exception handling traffic on your your services because, hey, Sometimes that cash can get in the way for some reason.

Scott Tolinski

It can cause some problems. You might need to know what's going on. But more importantly, if you have something like centuries replay where you can see what the user's getting on their dashboard. You can even see if they were accidentally getting some potentially cached or incorrect Old stale data.

Scott Tolinski

So, head on over to century.i0 and use the coupon code tasty treat, all lowercase and all one word. You'll get 2 months for free. All right, so let's talk about the stale while revalidate.

Topic 2 01:47

Explanation of stale while revalidate caching header

Wes Bos

This is a header which you're probably thinking I don't need a podcast about headers. This is such a cool header. You're going to want to listen to this one, and we're going to explain, first of all, like like why you might need it and then and then how it works. So For the purpose of this podcast, we'll talk about 2 examples.

Topic 3 02:07

Caching HTML pages to avoid database queries

Wes Bos

First, one of them simply just being an HTML page that when visited.

Wes Bos

That HTML page needs to be cached so that the database queries, the rendering, all that stuff that needs to happen on the server doesn't need to happen all the time.

Wes Bos

Second 1 is the open graph images on the Syntax website.

Topic 4 02:28

Open graph images take long time to generate

Wes Bos

So those take a while to generate and we need to cache them so that Once you take the 15 seconds to generate it, they're going to be loaded instantly after that. However, like we're constantly changing the names of podcasts. And like, you don't want that data to be cached for too long in there because then you have to have a whole whole thing where you invalidate your cache. So those are the 2 examples we'll refer to this, But you, the listener, can think of any time you want a asset, whether it is an HTML page, an image, some JavaScript, some CSS.

Topic 5 03:11

Why you might want to cache something

Wes Bos

If you want that to be cached for some amount of time, then you might Want to dip into using the stale while revalidate header. So why? Scott, you wanna grab the first one? Why might you want to cache something?

Scott Tolinski

Yeah. Well, the thing is with caching is data oftentimes doesn't really change that much. You know? You can think about it. If you have a CMS, You update some data. Right? And you're serving that same data to a lot of people.

Topic 6 03:26

Avoiding expensive operations with caching

Scott Tolinski

And whether it is caching your your server response or your data.

Scott Tolinski

If you think about caching as a whole, just a concept, what you're doing is you're basically saving that information So that way when someone else requests it, it doesn't have to do the expensive operations, and it can just serve up The already saved data typically could be saved in memory, could be saved in your browser's cache.

Scott Tolinski

Caching is a technique used just about in every single facet of computer science. Right? You're often needing to save things for later so that they're easily accessible.

Topic 7 04:08

Caching used in many areas of computer science

Scott Tolinski

Yeah. That way expensive operations don't get run more than they need to. Exactly. Like, you you might have a

Topic 8 04:20

Avoiding expensive operations with caching

Wes Bos

Page that takes 800 milliseconds to do database query.

Wes Bos

But what if that was 0 milliseconds for. Yeah, right. For absolutely everybody. Right. If it's the same database query or even if you if you only cache it for 1 minute, you know, that's you can do that.

Scott Tolinski

Another one is you can Sorry. Yeah. Sorry. Let me even just hop in there too to make things a little bit clearer for people who don't have any experience with caching. Think about images. Right? You have a a big old image.

Scott Tolinski

And if you download that image once, that takes some time.

Topic 9 04:55

Caching images to avoid re-downloading

Scott Tolinski

And if you cache it, It stores that image on your computer. So the next time you click refresh on that web page, it's not having to go off and repull it down. It's just there. It already exists.

Wes Bos

Another use case for caching is you can put the caches closer to your users. So, Akash can live in 2 spots. It can live on your computer, which you visit a website and then you go back to that website later.

Topic 10 05:11

Putting caches closer to users with CDNs

Wes Bos

Your like Scott just said, you don't have to redownload that, but it can also live on a CDN so you can put a CDN in front of your assets or you use something like Cloudflare or Vercel.

Topic 11 05:26

Local browser caches assets

Wes Bos

They will automatically CDN eyes a lot of your assets And they'll say, all right, well, you generated this Open Graph image. Let me go throw a copy in Australia.

Topic 12 05:40

CDNs cache assets globally

Wes Bos

Let me throw a copy in Boston, And then that's gonna load fast for people

Scott Tolinski

all around the world. Yeah. Well, they can also be a cache can also be on that just a straight up server too.

Scott Tolinski

Stuff. Just, you had said a cache can be in 2 places on the browser or a CDN. I just wanna make sure that that's clear. They can all stop by yeah. I'm talking about, like, the actual cache header. So you're right. Your server

Wes Bos

itself can respect those cache headers.

Topic 13 06:13

Server can respect cache headers

Wes Bos

But generally what you're doing is, yeah, you're sticking a CDN in front of it. Yeah. Sorry. I was also speaking on a cache broadly rather than. Oh, yeah. Yeah. I'm more thinking about like like a cache header here where because stale while revalidate is a header that you set on your responses, then stuff? It's either I don't even wonder. I think probably like your ISP might even might even do caches as well. I would think so. Yeah, it's a question. We should ask the Cloudflare folks that. They would probably know that. Hey. I got a an interesting sidebar question. Yeah. You know when they have, like, Movies on an airplane that are stored onto a hard drive. Do you call that a cache? Oh, yeah. For sure. Yeah. That's a, like, a local cache where it's sitting in. Like, Like lots of ISPs have Netflix caches, right? Yep. So they don't have to go all the way to Netflix to do that.

Wes Bos

Another reason for a cache is you might limit how often you're allowed to hit an API. So if you have and maybe, for example, you might only be allowed to hit an API 30 times an hour, So you might have to cache it for 2 minutes. Otherwise you're going to run out of API hits, right? You can't be hitting at every single request. So it's good to You can either cache like the output or like Scott said earlier, you could also just have like an object or a key value database. But that's not really what we're talking about here. Talking about that, the actual outputted header.

Topic 14 07:07

Caching to limit API hits

Wes Bos

Or you might be doing something expensive, such as generating images. So Every time we generate an Open Graph image, it takes about between 8 and 15 seconds, depending on if the Headless browser is hot or not. Basically, I kept the browser around so that if you come If you try to generate another one, like 10 seconds later, the browser's already started up and that that'll cut 5, 6 seconds off of everything.

Topic 15 07:40

Caching expensive image generation

Wes Bos

But if you think about that, like, oh, I'm generating hundreds of thousands of these for every single request.

Wes Bos

You're paying the bill to generate all of those. So it often makes sense to just generate it once for however long and then

Topic 16 08:24

Serving from cache avoids expense of regeneration

Scott Tolinski

serve up the cash version. Yeah. Well, okay, let's get into then just straight up normal caching headers, which you typically see in just as max age or s Max Age. And that basically is a is it is it it's total seconds. Right? That's how what a Max Age is said too? Yeah. It's It's it's one of the few things that are seconds based. Yeah. I know. I wanted to say milliseconds, but then I've made a max age calculator for myself because I was so annoyed with having to calculate it out all the time. Yeah. And so I'll post a link to that. I made it in Svelte if you wanna see it.

Topic 17 09:02

Max age sets cache lifetime in seconds

Scott Tolinski

But, like, Basically, it's how many seconds you want this thing to be cached for before, the browser

Wes Bos

Or the CDN goes off and gets a new version of it. And the difference there between the the 2 that Scott just said is max age is going to be cached on your browser, whereas SMAS Max Age, the S stands for shared, and that will tell your CDN. It is okay to share this between multiple people. So one possible I've seen this a couple of times security issue is If you accidentally s Max age somebody is like logged in state, you could possibly be serving up They're logged in state to the next user that comes because you said this is a shareable cache. So you kind of have to be careful with that because you have to think, okay, If I'm saving a copy of this user's rendered out, like, let's say it's your eBay, you're logged in and you see all of your account details.

Topic 18 10:06

Don't cache sensitive user-specific data

Wes Bos

If you if they cache that on a CDN, then the next person that comes along is going to see the wrong logged in state.

Wes Bos

And that's a security issue there. So certainly be careful. Don't just think, oh, s Dash is Like, super or something.

Scott Tolinski

Oh, yeah. S dash is better because it's on the CDN. In in that regard, if you wanted something cached for both of those, Would you use both of those headers, Wes, or just 1 or the other? That's a good question, actually.

Topic 19 10:22

S-maxage caches on CDN, maxage on browser

Wes Bos

I would think that S Max age is it will both be respected by the CDN as well as the client. But let's pause a second, double check that.

Wes Bos

I asked Chat GPT, it says yes.

Wes Bos

I'll tell you this. I've never set both of them on a single response, and I've done this a lot. So I would assume so that it would be cached both by your browser as well as, And you can tell that pretty quickly by I guess we could go to the the syntax website or go to the syntax. Fm/og forward slash 600.

Wes Bos

JPEG and that will give you the actual rendered out open graph image. And then you can look at it In your network tab, see if it is cached.

Wes Bos

Oh, but then I have the dev tools disabled cache on There We Go. Yes, it does. So S Max Age will cache it both at a CDN level as well as at a browser level.

Wes Bos

So again, if you're.

Wes Bos

Which makes sense because you're you're not spending the bandwidth to get to the CDN If you already have it downloaded into your browser cache.

Wes Bos

Now enter stale while revalidate, right? The Max, you give an access a Max Age header. That is how long the asset is good for.

Topic 20 11:52

Max age sets total cache lifetime

Wes Bos

Then you give it a stale value.

Topic 21 12:05

Stale while revalidate sets stale cache lifetime

Wes Bos

And the idea with this is it tells the client How long it's okay to reuse or serve up a stale asset.

Topic 22 12:16

Stale value tells how long to serve stale asset

Wes Bos

So in the example of the Syntax Open Graph images, We were hitting 10 seconds to generate the image, and then You could put like a let's say we put a 1 minute max age on there. So every time for the next minute you refresh the page, You're going to get fast reloads, but after 1 minute, that is that asset is considered stale.

Wes Bos

So the browser goes back to the server and says, Hey, I need a new one. This is a stale image, and then you got to wait another 15 seconds.

Wes Bos

Stale while revalidate will do is you can say, Okay, this is good for a minute, but for up to 10 minutes past that, It's okay to serve the stale one up, right? This is stale bread. You can still eat it for the next day or so, but we should probably make some more, Right? So stale while revalidate will just say, Oh, oh, shoot. Okay, all I got is this stale PNG of the the open graph image. You can have it for now because it's still fine. They said it would be fine for the next 10 minutes. It's okay. But It's okay. I'm going to go out back and make a new one.

Topic 23 13:33

Stale while revalidate serves stale but refreshes

Wes Bos

So the next time somebody comes, I'm going to have a fresh one for them. And then I've renewed the max age on there. So what it does is it kicks off a background task and it doesn't make the user it gives you the stale version immediately, which is often fine.

Wes Bos

And then it'll kick off a background task that nobody is sitting around waiting for, and it will regenerate a new version. And then you come back A minute later and you've got the new version instantly. So the idea is, and this is the same with like a React website that would be rendering, is that If you visit a page, you might be getting stale content, but you should know that in the background you're regenerating a new version And you are never going to be sitting around, sitting on your hands waiting for a database query that is slow to resolve or a weird rendering or a download of hitting an API that is really slow. So all of those things can happen in the background while you're waiting for it. And that's what the stale revalidate says. Keep it for this long and it's okay to serve a stale version within this many minutes.

Topic 24 14:41

Stale while revalidate avoids waiting for background tasks

Wes Bos

But if I'm serving up a stale version within this many minutes, I'll go make a new one so the so we can have fresh.

Scott Tolinski

But, you know, you gotta feel like it it has some degree of accuracy there because I think the reason why that metaphor is perfect because it also the use case in which stale while revalidate probably isn't a good idea.

Topic 25 15:11

Not for content that must always be fresh

Scott Tolinski

If you were to be serving up stale bread to the queen Yeah. Or just somebody. Right? Somebody of highest importance. The the bread must be at its very best. This bread must be perfect.

Scott Tolinski

Then at that point, You're going to probably not want to serve stale content to those people.

Scott Tolinski

So there are instances where The fresh content always needs to be there. And in that case, yeah, you probably you know, you're you're gonna be Finding alternative caching methods or caching for much less time. Yeah. Maybe even, like, on the seconds. I I've I heard that, you know, you probably don't wanna use this Content that's updating really quickly, like like sports scores that need to be served. Like, if you go to, like, espn.com And you load the site. You want the scores to be exactly up to date all the time, so you're probably not gonna wanna serve stale Scores TO Users? Yes.

Wes Bos

I think the way that you said it is it makes sense.

Topic 26 16:16

Accuracy important for stale cache suitability

Wes Bos

Sometimes the fresh version is worth waiting for.

Wes Bos

Correct.

Wes Bos

But also, like, with the let's say ESPN, let's say every 3 seconds They are.

Wes Bos

You could cache the scores for 3 seconds and then.

Wes Bos

Say still while revalidate after another 3 seconds. So what is happening? ESPN gets A 100,000 traffic visitors in one second. Right.

Wes Bos

That's not uncommon. Right. Lots of people watching hockey. Lots of people refreshing wanting to know the score that if you get 100,000 people a second, that's 300,000 requests that are being served up as cached. And then once you get over that 3 seconds, you're on the 4th 2nd.

Wes Bos

Now people are getting stale content, But it's only 4 seconds old, and there's been a background thing kicked off to regenerate the latest version of it. So It really has to do with like how much traffic you're getting with the syntax website, the old syntax website.

Wes Bos

We had a one second stale while we validate on it. Why? Because we needed a way to regenerate the website for the newest shows, right? Because there was no like cron that would automatically populate it. So what was happening? But we didn't want every time you visited the website, we didn't want the person to say, Alright. Well, I got to sit here and regenerate the whole website.

Wes Bos

And just in case there's a new episode. So what we did is We put a one second still while we validate on it, meaning that you visit the website, you get the cached version, but you immediately kick off a regen for the next person.

Wes Bos

Next person comes along. They get your the one that you you generated to be cashed, and they kick off a generation for the next person and the next person. So you're always kind of off by 1 in that generation, meaning that You're getting one second old content, which might be okay, but you're also getting it instantly because you somebody else kicked off the generation for you That that's been cooked beforehand. Why wouldn't you just toss stale while revalidate

Scott Tolinski

on everything?

Topic 27 18:35

Why not use stale while revalidate everywhere?

Wes Bos

Stuff? That's a good question. Like, first of all, you might not be getting that much traffic and you don't want to have to be dealing with with caches?

Topic 28 18:47

Caching adds complexity

Scott Tolinski

I mean, they're pretty simple.

Wes Bos

There's there's stuff. Yeah. Yeah. It's true. May maybe there is, like, a use case where one second is not long enough or sorry, is too long like Imessage, you know, or you're doing like a Zendesk back and forth. One second could be kind of annoying to get that type of thing. And in that case, The caching doesn't really matter if it's rendering fast enough for you. Do you really need to sit around and wait for cash and have this like off by one error? Stuff.

Topic 29 19:22

Need traffic to enable stale revalidations

Wes Bos

And in that case, you also need enough traffic to kick off those next renders If you don't have enough traffic in the example of the syntax website that I gave you, if there wasn't enough traffic for somebody to kick off the next regenerate, then Then you're just sitting there waiting for the site to generate. And it only took a couple of milliseconds or 100 milliseconds or whatever to generate the site. It wasn't really that long, but It's still the difference between a couple 100 milliseconds and 0 is quite a bit.

Wes Bos

So, yeah, it's not something that I would just I don't know. Like, Should you throw it on anything? Everything

Scott Tolinski

for me? I really feel like it's sriracha at this point or something or whatever people just like to put on everything.

Wes Bos

I'm sure that there'd be some some issue. And also, like, the one second example, Like it's got to take longer than that to propagate. If you've got a CDN network, let's say if you have a CDN like Vercel or Cloudflare and they've got 50 locations, you know, How long does it take to take that HTML file and propagate that through the world? So I don't know if the one second would be ideal in all of those cases because you have to literally wait for it, but even even a minute would make a lot of sense.

Topic 30 20:41

CDN propagation delays limit speed

Scott Tolinski

Yeah. I know. I well, that's my biggest question is is, I guess, why would I just not use this all the time? Because it feels like I want to. Yeah.

Wes Bos

If you were if you had, like, a, like, a Twitter or a YouTube where you have, like, the header and it has, like, notifications in it, Like you probably don't want to cache that because now you're spending CDN space to cache that data Interesting. For absolutely every but you could Just cash it on the on the user's client as well. Word. Cool. Yep. That's it. I thought that's I told Scott this would be a 10 minute episode, but here we are 22 minutes in.

Topic 31 21:18

Show wrap-up

Scott Tolinski

I think we had to get our food metaphors in. Exactly. We got it.

Wes Bos

But tweet us at Syndax FM if you have the different caching headers, which is episode 464.

Wes Bos

So go to syntax. Fm/464.

Wes Bos

You can listen to the episode Cash control headers explain.

Wes Bos

We go through all of them.

Wes Bos

Immutable, no mutable, private, stale if error. That's another one is like, Well, what happens if it breaks? Are you allowed to give them the old version for the next 10 minutes? Max age? Stuff. So listen to that, but I thought we we should go entirely in-depth just for stale while we're validated. Yeah. Absolutely. I love it. Alright. Peace.

Scott Tolinski

Head on over to syntax.fm for a full archive of all of our shows, And don't forget to subscribe in your podcast player or drop a review if you like this show.

Share