Alex, I'm confused about the error handling in *wt_lsm_tree_drop(), *wt_lsm_tree_rename() and __wt_lsm_tree_truncate(). They all have the same sequence of calls:

       /* Get the LSM tree. */
        WT_RET(__wt_lsm_tree_get(session, name, 1, &lsm_tree));

        /* Shut down the LSM worker. */
        WT_RET(__lsm_tree_close(session, lsm_tree));
        
        /* Prevent any new opens. */
        WT_RET(__wt_try_writelock(session, lsm_tree->rwlock));

But, it looks to me like if either *lsm_tree_close or *wt_try_writelock fail (where EBUSY is a potential "failure" for *wt_try_writelock), we'll return with the LSM tree's reference count incremented, and, if the tree wasn't already open, we'll have called *lsm_tree_open(), so we'll leak a whole tree-in-memory structure.

I'm particularly curious about the call to try-writelock: we're setting the "exclusive" flag in the calls to __wt_lsm_tree_get (which makes me think it's possible the tree is already open), can one of those other open references have the tree's write-lock?

Also, the error handling in __wt_lsm_tree_truncate():

        if (0) {
err:            __wt_rwunlock(session, lsm_tree->rwlock);
                __lsm_tree_discard(session, lsm_tree);
        }
        return (ret);

Why aren't we always discarding our reference, in other words, why isn't it this way instead:

        if (0) {
err:            __wt_rwunlock(session, lsm_tree->rwlock);
        }
        __lsm_tree_discard(session, lsm_tree);
        return (ret);