Writing Thread-Safe Programs – Analyze Cost Of Locking

One way of writing a thread-safe class is to use synchronization when its shared variables are being modified so that a lock is obtained by the executing thread leaving all the other threads in the waiting state to get the access of the shared variables. This way we ensure that only one thread can modify the shared variable and no race condition occurs.

In the cache implementation to store frequently searched programs the DAO class fetches the program by program number from the database, if it does not exist in cache and when the program is found it adds it to the cache. The code is given below in Listing 1.

program = programsCache.get(programNumber);
if(program == null) {
      try {
            //program is not found in cache. Fetch it in the database
            program = ProgramDAO.getInstance().findByProgramNumber(
                        programNumber);
            programsCache.putIfAbsent(programNumber,program);
            ...
      }
      catch(EntityNotFoundException exp) {
            ...
      }
}

Listing 1. Fetch the program from cache and if not found fetch it from database

This code is part of the doPost() method of servlet and the programsCache is a ConcurrentHashMap<String,Program> type variable shared by multiple request threads. What happens when more than one request thread tries to fetch the program with the same program number concurrently when it is not found in the cache? While executing these threads, each thread will call the findProgramNumber() method to fetch the same program resulting in multiple calls to the remote database tier. The intention of implementing the cache here is to minimize the database calls. So should we use the synchronized block as shown in Listing 2 to ensure that only one database call is made to fetch a program?

//put this code in try-catch block to handle EntityNotFoundException
synchronized(this) {
      program = programsCache.get(programNumber);
      if(program == null) {
            //program is not found in cache. Fetch it in the database
            program = ProgramDAO.getInstance().findByProgramNumber(
                        programNumber);
            programsCache.put(programNumber,program);
            ...
      }
}

Listing 2. Locking the servlet instance to avoid multiple calls to the database to fetch same program

Let’s analyze the approach used in Listing 1. Though multiple database calls can be made to fetch the same program, only one program instance is stored in the cache. The possibility of fetching the same program from the database is a tradeoff but it is not as costly as locking the servlet instance in Listing 2.

The synchronized statement in Listing 2, locks the entire servlet object. When a request thread is executing the synchronized block, it obtains a lock on the servlet instance’s monitor thus blocking all the other request threads. The lock will be released when the thread leaves the synchronized block. During this time, the thread is hardly running within the servlet instance’s monitor as it first executes in the programCache’s monitor to execute get(). It then goes ahead and executes the ProgramDAO.getInstance() and further runs in the monitor of ProgramDAO’s instance while executing findByProgramNumber() method. While these methods are being executed by the thread, the servlet instance’s monitor is idle leading to monitor starvation.

Since the lock is obtained on the servlet instance in Listing 2, no request thread can concurrently retrieve or put program in the cache. This approach can lead to thread contention. Thus, this is an expensive locking approach. Also, there is no need of using ConcurrentHashMap<String, Program> as not more than one thread can access the cache concurrently.

Which locking approach is better? There is no monitor starvation in Listing 1, since there is no lock obtained on the servlet instance. When a request thread leaves the servlet instance’s monitor for executing the methods of ConcurrentHashMap or ProgramDAO, other request threads can execute take over the servlet instance’s monitor for execution. The locking approach used in Listing 1 is the winner!

Analyze the cost of your locking approach

While writing thread-safe programs, you’ll often experience that there are many ways of exclusive locking. The rule of thumb is always prefer non-blocking algorithms due to their high throughput. When you don’t have a non-blocking algorithm, use fine-grained locks.