Uploaded image for project: 'Stripes'
  1. STS-931

ReflectUtil.findClass(String) causing lock contention on web app class loader.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects versions: Release 1.5.7, Release 1.5.8
    • Fix versions: Release 1.6
    • Components: None
    • Labels:
      None
    • Environment:
      Jetty 9.x
      Oracle 1.7.0_75
      RHEL x64
      Dual socket, 6 cores (12 cores / 24 hyperthreaded)
    • Sprint:

      Description

      We have noticed that under high load, that when we thread dump, we see lots of threads blocked in RelectUtil.findClass(String) when trying to make the following call (which is synchronzied)

      Thread.currentThread().getContextClassLoader().loadClass(name);

      INFO | jvm 1 | 2015/03/17 15:06:44 | "qtp560683258-213" prio=10 tid=0x00007fe39c001000 nid=0xa29 waiting for monitor entry [0x00007fe5805cb000]
      INFO | jvm 1 | 2015/03/17 15:06:44 | java.lang.Thread.State: BLOCKED (on object monitor)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:396)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:389)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at net.sourceforge.stripes.util.ReflectUtil.findClass(ReflectUtil.java:158)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at net.sourceforge.stripes.tag.StripesTagSupport.getActionBeanType(StripesTagSupport.java:191)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at net.sourceforge.stripes.tag.StripesTagSupport.getActionBeanUrl(StripesTagSupport.java:226)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at net.sourceforge.stripes.tag.UseActionBeanTag.setBeanclass(UseActionBeanTag.java:198)
      INFO | jvm 1 | 2015/03/17 15:06:44 | at

      :~$ grep "waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)" wrapper.20150317.log
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)
      INFO | jvm 1 | 2015/03/17 15:06:44 | - waiting to lock <0x00000001cef0a870> (a org.eclipse.jetty.webapp.WebAppClassLoader)

      It does appear to be a limiting factor when trying to scale our application.

        Attachments

          Activity

            People

            • Assignee:
              rgrashel Rick Grashel
              Reporter:
              lord.buddha David Wade
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: