Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
https://ngrok.com/blog/prompt-caching/
By the end of this post you will...
Understand, at a deeper level, how LLMs work
Have built some new intuition for why LLMs work the way they do
Understand the exact 1s and 0s that get cached, and how they reduce the cost of your LLM requests
about REDbot
http://redbot.org/about/
RED is a robot that checks HTTP resources to see how they'll behave, pointing out common problems and suggesting improvements. Although it is not a HTTP conformance tester, it can find a number of HTTP-related issues.
about REDbot
http://redbot.org/about
RED is a robot that checks HTTP resources to see how they'll behave, pointing out common problems and suggesting improvements. Although it is not a HTTP conformance tester, it can find a number of HTTP-related issues.
Using Flat Files So Elections Don't Break Your Server - NYTimes.com
http://open.blogs.nytimes.com/2010/12/20/using-flat-files-so-elections-dont-break-your-server
In the Times Interactive News Department, we pay careful attention to caching strategy to make sure it’s a good fit for the project at hand.
Displaying PHP APC Cache Information
http://www.electrictoolbox.com/apc-php-cache-information
"APC is the Alternative PHP Cache, which is a free, open, and robust framework for caching and optimizing PHP intermediate code. I posted about how to install APC on Linux a couple of days ago, and will now look at the apc.php script which comes with APC and shows information about how much of the cache is being used, what files are being cached, the number of times they've been accessed etc."
Holy Shmoly! » WordPress, Nginx and WP Super Cache
http://ocaoimh.ie/89495533/wordpress-nginx-wp-super-cache
"If you host your own WordPress blog, it’s probably on Apache. That all fine and good. For most sites Apache works wonderfully, especially as it’s so easy to find information on it, on mod_rewrite and everything else that everyone else uses.
One of the alternatives is Nginx, a really fast webserver that streaks ahead of Apache in terms of performance, but isn’t quite as easy to use. That’s partly because Apache is the default webserver on most Linux distributions and hosts. Want to try Nginx? Here’s how."
A Bunch of Great Strategies for Using Memcached and MySQL Better Together | High Scalability
http://highscalability.com/bunch-great-strategies-using-memcached-and-mysql-better-together
"The primero recommendation for speeding up a website is almost always to add cache and more cache. And after that add a little more cache just in case. Memcached is almost always given as the recommended cache to use. What we don't often hear is how to effectively use a cache in our own products. MySQL hosted two excellent webinars (referenced below) on the subject of how to deploy and use memcached."
Coding Horror: Behold WordPress, Destroyer of CPUs
http://www.codinghorror.com/blog/archives/001105.html
"Personally, I think it's absolutely irresponsible that WP-Cache like functionality isn't already built into WordPress"
Simon Wistow - Standing in Line
http://deflatermouse.livejournal.com/145533.html
"What I actually want out of this is a kind of de facto standard like Memcached or Apache or whatever that's so simple to understand and use that it changes the way people develop their apps such that they just start with the assumption that some of their
mnot’s Web log: Cache Channels Beta
http://www.mnot.net/blog/2008/01/04/cache_channels
"The default implementation uses an archived Atom feed to represent the contents of the channel, which the cache to stay in touch"
mnot’s Web log: Two HTTP Caching Extensions
http://www.mnot.net/blog/2007/12/12/stale
stale-while-revalidate and stale-if-error