How To Write Simple In-Memory Cache in Java Tutorial

I want to show you my implementation of lightweight and simple in-memory key-value cache mechanism in Java.

Database queries could take a time and it’s a good idea to store frequently used data in the cache to retrieve it faster.

Java caching frameworks like Spring Cache allows to define your own in-memory cache implementation, so you can adopt mine.

But first of all, let’s define criteria for our Java cache implementation:

  • store data in memory
  • allow putting object by key for some amount of time
  • memory usage is not restricted, but cache shouldn’t be a reason for OutOfMemoryError
  • the cache should remove expired objects
  • thread-safe

Let’s define an API:

It looks similar to Map API and I’m gonna use a  ConcurrentHashMap for our example.

Keep reading…

Java Cache Implementation

Let’s take a look at an example first and then I’ll explain my architecture decisions:

Note that I used some Lombok annotations to generate boilerplate code like @AllArgsConstructor  and @Getter, you can replace it if you want.

So let me explain what is going on here:

  • I took  ConcurrentHashMap because I have thread-safe requirement.
  • I used SoftReference<Object> as a map value because soft reference guarantees that referenced object will be removed in case of lack of memory before OutOfMemory will be thrown.
  • In the constructor, I created a daemon thread that scans every 5 seconds and cleans up expired objects, 5 seconds is a random number and you should think about cleaning interval.

What’s wrong with this solution?

  • If map contains a big amount of cached objects scan and clean up can take a time because it’s iterating through all values
  • size() method takes O(n) time because it needs to filter out expired objects

How can we improve this?

Let’s try to use a queue for removing expired objects.

Caching in Java Using Delay Queue

DelayQueue allows adding elements to the queue with delay (expiration period in our case), so we can schedule removing of expired objects.

Let’s take a look at code example:

We need to implement a Delayed interface and override 2 methods: getDelay(TimeUnit unit) and compareTo(Delayed o).

A getDelay() method defines a period of time before the object will be available in the queue.

A compareTo() method should be ordering consistent with getDelay().

The delayed object appears in the queue only when getDelay() value is 0 or negative, so we’re sure that object is removed in time.

Now we can remove all isExpired() checks at all.

size() method is up to date and doesn’t require to filter out expired objects as well, now it takes constant time.

It’s not needed to iterate through the whole map to find what to delete.

This implementation is more elegant but needs a little bit more memory because of the queue.


It’s a simple example of self-written in-memory cache, you can use it to store some long-running queries from DB or frequently used data.

My implementation is really simple, for more complex cases you should use distributed cache solutions like Memcached, ehCache etc.

I can’t say what is a best Java cache library, it depends.

You should think about what is the best choice for cache management in your application and make decision.

Do you have any question? Propositions? Ask me.

Scroll Up