solr引用计数
据我不完全统计,solr代码中使用引用计数的用途有两种:一个是引用资源,一个是引用对象。技术上来说引用计数的使用没多少可大说的,不过如果没有正确的close获得的资源和对象,泄漏的bug就出现了。
solr中供使用的IndexSearcher是SolrIndexSearcher,使用时它是寄存于RefCounted。RefCounted的代码如下:
public abstract class RefCounted<Type> { protected final Type resource; protected final AtomicInteger refcount = new AtomicInteger(); public RefCounted(Type resource) { this.resource = resource; } public int getRefcount() { return refcount.get(); } public final RefCounted<Type> incref() { refcount.incrementAndGet(); return this; } public final Type get() { return resource; } public void decref() { if (refcount.decrementAndGet() == 0) { close(); } } protected abstract void close(); } |
SolrIndexSearcher关联于SolrCore。SolrCore成员变量
1 | private RefCounted<SolrIndexSearcher> _searcher |
表示正在使用的SolrIndexSearcher,而成员变量
1 | private final LinkedList<RefCounted<SolrIndexSearcher>> _searchers |
保持了未close的SolrIndexSearcher列表。在主从模式下,当新的索引到来时,solr会创建了新的
1 | RefCounted<SolrIndexSearcher |
实例赋为_searcher并添加到_searchers。上面的RefCounted的close方法是抽象的,在SolrCore中有其实现:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | private RefCounted<SolrIndexSearcher> newHolder(SolrIndexSearcher newSearcher) { RefCounted<SolrIndexSearcher> holder = new RefCounted<SolrIndexSearcher>(newSearcher) { public void close() { try { synchronized(searcherLock) { // it's possible for someone to get a reference via the _searchers queue // and increment the refcount while RefCounted.close() is being called. // we check the refcount again to see if this has happened and abort the close. // This relies on the RefCounted class allowing close() to be called every // time the counter hits zero. if (refcount.get() > 0) return; _searchers.remove(this); } resource.close(); } catch (IOException e) { log.error("Error closing searcher:" + SolrException.toStr(e)); } } }; holder.incref(); // set ref count to 1 to account for this._searcher return holder; } |
可以看到,当调用RefCounted的close方法,如果引用计数refcount不大于0,则会调用SolrIndexSearcher的close方法来释放其保持的文件句柄,并且将其从_searchers中移除。
再来看查询时对SolrIndexSearcher的使用。每个请求是要构造SolrQueryRequest,看下SolrQueryRequestBase代码片断:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | protected RefCounted<SolrIndexSearcher> searcherHolder; public SolrIndexSearcher getSearcher() { if(core == null) return null;//a request for a core admin will no have a core // should this reach out and get a searcher from the core singleton, or // should the core populate one in a factory method to create requests? // or there could be a setSearcher() method that Solr calls if (searcherHolder==null) { searcherHolder = core.getSearcher(); } return searcherHolder.get(); } public void close() { if (searcherHolder!=null) { searcherHolder.decref(); searcherHolder = null; } } |
当调用SolrQueryRequestBase的getSearcher()时,如果是第一次调用,会转而调用core.getSearcher(),其会使得到的searcherHolder的引用计数增一。而对称的,SolrQueryRequestBase的close()方法使searcherHolder的引用计数减一,一增一减平衡了。这也使得当新的索引到来时,仍旧提供查询服务的SolrIndexSearcher不会立即关闭,直到其引用计数减为0才关闭。所以,在使用SolrQueryRequest时,要确保请求的最后调用其close方法,否则那些无用的SolrIndexSearcher就不会被释放,直到句柄耗尽或者OOM掉。
引用对象
引用对象的典型使用是SolrCore,看下相关代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | private final AtomicInteger refCount = new AtomicInteger(1); final void open() { refCount.incrementAndGet(); } public void close() { int count = refCount.decrementAndGet(); if (count > 0) return; // close is called often, and only actually closes if nothing is using it. if (count < 0) { log.error("Too many close [count:{}] on {}. Please report this exception to solr-user@lucene.apache.org", count, this ); return; } //释放对象和资源 } |
引用计数就是refCount了。SolrCore实例的获得是通过调用CoreContainer的 SolrCore getCore(String name)得到,其实现是:
1 2 3 4 5 6 7 8 | public SolrCore getCore(String name) { synchronized(cores) { SolrCore core = cores.get(name); if (core != null) core.open(); // increment the ref count while still synchronized return core; } } |
也就是说CoreContainer显示的open了SolrCore,所以在得到SolrCore实例后,也需要显示的close它。因为SolrQueryRequestBase用到了SolrCore,所以在处理请求的最后,要确保调用了SolrCore的close方法。当然,对于查询端,SolrCore实例在整个生命周期内通常并不会真正被close,除非显示的调用了reload等操作。
本文固定链接: http://www.chepoo.com/solr-refcounted.html | IT技术精华网