There are a few types of cache memory, so I'm not sure just what you mean. The processor has cache memory that uses very fast and very expensive memory. Unless you spend big bucks it's unlikely that the processor in a machine you might buy tomorrow will have more than 1 MB for the L2 cache, and the L1 may only have 16 to 64KB. Most people don't do much where a bigger cache (or even a faster processor) will improve their performance. Newer games and engineering require a lot of computing power, but web surfing doesn't.
Random access memory, RAM, is often a bottleneck and adding more will almosy certainly improve performance. Right now I've got 3 browser windows open, plus a game of Freecell and my machine is using 296MB of RAM. Most of that is Windows itself (XP), anti-virus and firewall programs and other things that run in the background, and I've disabled a number of things (Quicktime and Adobe reader, for example) that normally run by default whether you need to use them or not. RAM is dirt cheap nowadays, and virtually everyone who hasn't added RAM since buying the machine has amachine that can use 2 to 8 times what they have.
Your web browser (whether it's Internet Explorer or AOL) has a browser cache, which uses space on your hard drive to store most of what you look at while on the web. That can speed up your browsing by loading files from your hard drive instead of having to download them. If you have a slow internet connection that can make a big difference. Since you refer to logging off, I'm guessing you're still using a dial up connection. Most websites aren't very friendly to dial up users anymore. Web pages tend to be big, so it takes a long time to download the page. I don't know about AOL, but IE and other browsers let the user determine how big the cache is. If you want a bigger cache and you have enough free space on your hard drive you can make the changes to IE from the tools menu (Tools > Internet options > General tab > Temporary files > Settings).
As far as having the ISP clear your cache, they can't do it, and why would you let them have any control over your machine? Clearing the cache sometimes eliminates problems with viewing a website, but it can also slow things down because every piece of every web page will have to be downloaded when you open it. As with so many other things, a good rule of thumb is don't fix it if it's not broken. If you have problems on a website that used to work clearing the cache may solve the problem, but if things are working then the cache isn't likely to be slowing things down.
I just noticed that you said the machine you're using belongs to your kids. There's an excellent chance that's a big source of problems, too. Depending on how old they are and how computer literate they are, kids tend to download a lot of stuff that will slow a machine down, if not cripple it. Comet Cursor may be just the tip of the iceberg if your kids have downloaded stuff that sounded fun or useful.