当前位置: 首页 > solr, 搜索 > 正文

solr引用计数

关键字:
1 星2 星3 星4 星5 星 (2 次投票, 评分: 5.00, 总分: 5)
Loading ... Loading ...
baidu_share

据我不完全统计,solr代码中使用引用计数的用途有两种:一个是引用资源,一个是引用对象。技术上来说引用计数的使用没多少可大说的,不过如果没有正确的close获得的资源和对象,泄漏的bug就出现了。

solr中供使用的IndexSearcher是SolrIndexSearcher,使用时它是寄存于RefCounted。RefCounted的代码如下:

public abstract class RefCounted<Type> {
  protected final Type resource;
  protected final AtomicInteger refcount = new AtomicInteger();
 
  public RefCounted(Type resource) {
    this.resource = resource;
  }
 
  public int getRefcount() {
    return refcount.get();
  }
 
  public final RefCounted<Type> incref() {
    refcount.incrementAndGet();
    return this;
  }
 
  public final Type get() {
    return resource;
  }
 
  public void decref() {
    if (refcount.decrementAndGet() == 0) {
      close();
    }
  }
 
  protected abstract void close();
}

SolrIndexSearcher关联于SolrCore。SolrCore成员变量

1
private RefCounted<SolrIndexSearcher> _searcher

表示正在使用的SolrIndexSearcher,而成员变量

1
private final LinkedList<RefCounted<SolrIndexSearcher>> _searchers

保持了未close的SolrIndexSearcher列表。在主从模式下,当新的索引到来时,solr会创建了新的

1
RefCounted<SolrIndexSearcher

实例赋为_searcher并添加到_searchers。上面的RefCounted的close方法是抽象的,在SolrCore中有其实现:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
private RefCounted<SolrIndexSearcher> newHolder(SolrIndexSearcher newSearcher) {
    RefCounted<SolrIndexSearcher> holder = new RefCounted<SolrIndexSearcher>(newSearcher) {
      public void close() {
        try {
          synchronized(searcherLock) {
            // it's possible for someone to get a reference via the _searchers queue
            // and increment the refcount while RefCounted.close() is being called.
            // we check the refcount again to see if this has happened and abort the close.
            // This relies on the RefCounted class allowing close() to be called every
            // time the counter hits zero.
            if (refcount.get() > 0) return;
            _searchers.remove(this);
          }
          resource.close();
        } catch (IOException e) {
          log.error("Error closing searcher:" + SolrException.toStr(e));
        }
      }
    };
    holder.incref();  // set ref count to 1 to account for this._searcher
    return holder;
  }

可以看到,当调用RefCounted的close方法,如果引用计数refcount不大于0,则会调用SolrIndexSearcher的close方法来释放其保持的文件句柄,并且将其从_searchers中移除。
再来看查询时对SolrIndexSearcher的使用。每个请求是要构造SolrQueryRequest,看下SolrQueryRequestBase代码片断:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 protected RefCounted<SolrIndexSearcher> searcherHolder;
  public SolrIndexSearcher getSearcher() {
    if(core == null) return null;//a request for a core admin will no have a core
    // should this reach out and get a searcher from the core singleton, or
    // should the core populate one in a factory method to create requests?
    // or there could be a setSearcher() method that Solr calls
    if (searcherHolder==null) {
      searcherHolder = core.getSearcher();
    }
    return searcherHolder.get();
  }
  public void close() {
    if (searcherHolder!=null) {
      searcherHolder.decref();
      searcherHolder = null;
    }
  }

当调用SolrQueryRequestBase的getSearcher()时,如果是第一次调用,会转而调用core.getSearcher(),其会使得到的searcherHolder的引用计数增一。而对称的,SolrQueryRequestBase的close()方法使searcherHolder的引用计数减一,一增一减平衡了。这也使得当新的索引到来时,仍旧提供查询服务的SolrIndexSearcher不会立即关闭,直到其引用计数减为0才关闭。所以,在使用SolrQueryRequest时,要确保请求的最后调用其close方法,否则那些无用的SolrIndexSearcher就不会被释放,直到句柄耗尽或者OOM掉。
引用对象

引用对象的典型使用是SolrCore,看下相关代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
private final AtomicInteger refCount = new AtomicInteger(1);
 
  final void open() {
    refCount.incrementAndGet();
  }
  public void close() {
    int count = refCount.decrementAndGet();
    if (count > 0) return; // close is called often, and only actually closes if nothing is using it.
    if (count < 0) {
      log.error("Too many close [count:{}] on {}. Please report this exception to solr-user@lucene.apache.org", count, this );
      return;
    }
    //释放对象和资源
  }

引用计数就是refCount了。SolrCore实例的获得是通过调用CoreContainer的 SolrCore getCore(String name)得到,其实现是:

1
2
3
4
5
6
7
8
public SolrCore getCore(String name) {
    synchronized(cores) {
      SolrCore core = cores.get(name);
      if (core != null)
        core.open();  // increment the ref count while still synchronized
      return core;
    }
  }

也就是说CoreContainer显示的open了SolrCore,所以在得到SolrCore实例后,也需要显示的close它。因为SolrQueryRequestBase用到了SolrCore,所以在处理请求的最后,要确保调用了SolrCore的close方法。当然,对于查询端,SolrCore实例在整个生命周期内通常并不会真正被close,除非显示的调用了reload等操作。

本文固定链接: http://www.chepoo.com/solr-refcounted.html | IT技术精华网

solr引用计数:等您坐沙发呢!

发表评论