Sesat > Docs + Support > Architecture Overview > Design Proposals > New design proposal for SearchCommand and AbstractSearchCommand

Design proposal for SearchCommand and AbstractSearchCommand

Issue SKER2149: (Divide & Conquer AbstractSearchCommand to delegates)

Current problems

  • combination of various concern implementations leads to excessive subclass count or restrictive subclass implementation, eg simple or advanced filter combined with simple or advanced query gives four possible implementations with 100% duplication,
  • poor parameter handling, offset, sortby, results-to-return, etc
  • too many visitors, or unclear of various responsibilities involved,
  • filter confusion: fielded clauses filters versus custom filters versus QueryTransformer.getFilter()
  • ...

Current API

SearchCommand Context

ResourceContext
DataModelContext
SearchConfigurationContext
TokenEvaluationEngineContext

SearchCommand Concerns

Currently search commands have the methods:

SearchCommand

getSearchConfiguration()
call()
handleCancellation()
isPaginated()
isUserSortable()

AbstractSearchCommand

execute()
isCancelled()
performQueryTransformation()
performExecution()
performResultHandling()
getFilter()
setFilter()
getAdditionalFilter()
visitImpl
visitXorClause(..)
getSearchResult(..)
getOffset()
isPaginated()
geUserSortBy()
isUserSortable()
getParameter(String)
getSingleParameter(String)
getQuery()
getQueryRepresentation(Query)
getTransformedTerm(Clause)
getTransformedTerms()
initialiseTransformedTerms()
appendToQueryRepresentation(String)
insertToQueryRepresentation(String)
getQueryRepresentationLength()
escapeFieldedLeaf(LeafClause)
isEmptyLeaf(Clause)
getFieldFilter(LeafClause)
getTransformedQuery()
getTransformedQuerySesamSyntax()
setTransformedQuerySesamSyntax()
newSesamSyntaxQueryBuilder()
updateTransformedQuerySesamSyntax()
getEngine()
statisticInfo(String)
MapVisitor
FilterVisitor
SesamSyntaxQueryBuilder

AbstractXmlSearchCommand

getResultsToReturn()
getXmlResult()
getHttpReader()

AbstractSimpleFastSearchCommand

createnavigationFilterStrings()
setSearchEngineFactory(IFastSearchEngineFactory)
addNavigatedTo(String)
getNavigators()
getResultsToReturn()
getSearchEngine()
getSortBy()
setAdditionalParameters(ISearchParameters)
collectSpellingSuggestions(IQueryResult, FastSearchResult)
createSpellingSuggestion(String)
collectResults(IQueryResult)
createResultItem(IDocumentSummary)
createQuery()
getDynmicParams(Map,String,String)
getNavigatorsString()
flattenNavigators(Collection<Navigator>, Navigator)
collectModifiers(IQueryResult,FastSearchResult)
collectModifier(String,IQueryResult,FastSearchResult)
getModifierComparator(Navigator)
createProperNameSuggestion(String)
collectRelevantQueries(IQueryResult,FastSearchResult)

AbstractESPFastSearchCommand

ReservedWord enum
appendFilter(String,String)
getSortBy()
modifyQuery(IQuery)
createSearchResult(IQueryResult)
getIQueryResult()
isNavigatable()

NavigatableESPFastCommand

createNavigationFilterStrings()
addNavigatedTo(String)
getNavigatedTo(String)
getNavigatedValues()
getNavigatedValue(String)
getNavigatedTo()
collectModifiers(IQueryResult, FastSearchResult)
collectModifier(String,IQueryResult, FastSearchResult)
getModifierComparator(Navigator)

NewsEspSearchCommand

addMedium(Clause)
createCollapsedResults(NewsEspCommandConfig,int,IQueryResult)
addResult(NewsEspCommandConfig,ResultList,IDocumentSummary)

NewsClusteringESPFastCommand

createClusteringSearchResult(ClusteringEspFastCommandConfig,int,IQueryResult)

A number of clear concerns can be seen:

  • Execution handling call() execute() handleCancellation() isCancelled() performQueryTransformation() performExecution() performResultHandling()
  • Parameters getParameter(String) getSingleParameter(String)
    • Result list size getResultsToReturn()
    • Pagination & Offset isPaginated() getOffset()
    • Sorting isUserSortable() getUserSortBy() getSortBy()
  • Command applicable Query getQuery()
  • Command Query construction --> internal visitor visitImpl(*) getTransformedTerms() getTransformedTerm(Clause) initialiseTransformedTerms() appendToQueryRepresentation(String) insertToQueryRepresentation(String) getQueryRepresentationLength() escapeFieldedLeaf(LeafClause) isEmptyLeaf(Clause) getTransformedQuery() MapVisitor
  • Command XorClause handling visitXorClause(XorClause)
  • Filter (fielded clauses) getAdditionalFilter() FilterVisitor
  • Custom Filters getFilter() setFilter() QueryTransformer.getFilter()
  • Modifiers (Domain Navigators)
  • Stream handling/manipulating
  • Reserved words
  • Collapsing (& Clustering)
  • Suggestions: Spelling, ProperName, Relevant queries
  • Result Construction
  • Utility (almost static methods getSearchResult(..)

Solutions

Execution handling

Can remain as is.
This is in fact a concrete implementation of Callable's "ResultList<ResultItem> call()" method by AbstractSearchCommand, and it introduces the concepts of QueryTransformation, Execution, and Result Handling. AbstractSearchCommand express solely this, everything else it does should be moved out.

ToDo The SearchConfiguration classes SearchConfiguration, AbstractSearchConfiguration, and CommandConfig do not match the behaviour separation defined between SearchCommand and AbstractSearchCommand. For example, a brand new fresh SearchCommand implementation requires a SearchConfiguration with little of the properties actually defined in SearchConfiguration.

Execution handling generics

Although the use of generics requires a review. Using generics in method signatures intended to be subclassed is something a little more complicated to design properly and IMHO was not done correctly the first time.
For example:

public interface SearchCommand extends Callable<ResultList<? extends ResultItem>> 
public abstract ResultList<? extends ResultItem> execute()
public ResultList<? extends ResultItem> call()
protected final ResultList<? extends ResultItem> getSearchResult(String, DataModel)
protected final ResultList<? extends ResultItem> performExecution()
protected final void performResultHandling(final ResultList<? extends ResultItem> result)

No subclass can actually subclass these methods and provide exact generics to the signatures because each command, for example, returns a BasicResultList upon cancellation or handled internal checked exception.

Parameters

Parameters typically have three sources, and they first found used: a url parameter, a user parameter, the command's configured parameter.
Sometimes (eg userSortBy and pagination) the configuration actually comes from the presentation layer. The command's configuration here must simply point to where in the presentation layer this configuration can be found. Strictly speaking the domain ayer should be isolated from the presentation layer but here we access only the presentation layer's configuration through the datamodel.

ResultToReturn is an interesting example. It should be both overridable from url and user parameters. But the configuration exist in both the presentation layer and the domain layer. The domain layer's only responsibility is to ensure at least the amount of results are returned that the presentation layer wants. Up until now its just been presumed that the command's configuration is hardcoded to a value larger than any possible presentation value.

An example implementation can be viewed here: SearchCommandParameter code example.

Command applicable Query

Command Query construction

Magnus Eklund's solution follows:

Introduce a new interface called QueryBuilder. Query Builders are used to transform the query object into a string representation.

  • Remove "extends AbstractReflectionVisitor" from AbstractSearchCommand.
  • New interface QueryBuilder
  • Add property queryBuilder to SearchConfiguration to hold the FQN of a QueryBuilder class. This property will correspond to a <queryBuilder>-tag in modes.xml to allow for configuration of the query builder itself. (e.g. <queryBuilder class="no.sesat.search.mode.command.query.PrefixQueryBuilder" supportsAndNot="false" />)
  • Create QueryBuilder implementations:
    • PrefixQueryBuilder
    • SesamQueryBuilder
    • InfixQueryBuilder.
    • Fast4QueryBuilder (if needed, should be possible to configure one of the above to produce suitable syntax)
    • Fast5QueryBuilder (if needed, should be possible to configure one of the above to produce suitable syntax)

These would still visitors over the query but transforming from the transformedMap into a string representation.
The transformedMap is the mutable state of each clause through the search command's query transformations.
QueryBuilder also needs to have an API to deal with XorClauses, ReservedWords, and supported filter clauses. The definition of these things would be supplied through the context.

Backward Compatibility could be helped by, when no QueryBuilder is specified, creating a proxy against the search command and using it presuming it contains all the QueryBuilder methods required.
The alternative backard compatibility would be to refactor AbstractSearchCommand to DeprecatedSearchCommand, so that existing search commands work as is. Problem here is that we'd end up with a score of DeprecatedXyzSearchCommand classes as many of the subclasses are rewritten to the new AbstractSearchCommand implementation.

Example starting implementation can be read QueryBuilder code example.

Command XorClause handling

This can stay how it is. It must be provided to any QueryBuilder's context.

Reserved Words

Only exists in AbstractESPFastSearchCommand. Should be formalised and brought down to the SearchCommand interface.
Since the enumeration belongs to a particular SearchCommand implementation the enumeration should also return how each item is to be escaped in any query.
It must be provided to any QueryBuilder's context.

Fielded clause (& Custom) filters

Where filters (included custom filters) go is a prickly trick. Some command's have the filter simply appended to the query, while others have no notion of a filter at all.
Worse yet is that some filter must go into the query and others defined as separate parameters.

Should the transformedMap state be made available to multiple QueryBuilder within any one request, so that a command that request filters in a separate parameter to the query could run two separate QueryBuilders. The query QueryBuilder would ignore filters, the filter QueryBuilder would ignore non-filter terms. Exaggerating it, commands that requested each filter in a separate parameter could have multiple filter QueryBuilders each ignoring everything but the one filter it is responsible for.
But how does this deal with custom filters that come from the configuration, not the query. Should each custom filter be deserialised into a Query tree/object and QueryBuilders be able to visitor multiple query trees???

Custom filters

Stream manipulation & Result Construction

Stream manipulation exists in AbstractXmlSearchCommand. Should be formalised. A little awkward, for example FAST commands work against a java api.
Could be defined in auxiliary interfaces, eg RestfulSearchCommand, StreamReaderSearchCommand, that could be mixed and matched.
Example implementation can be read Stream manipulation code examples.

Result Construction is a higher level concern than stream manipulation. Likely will be the user of stream manipulation where applicable.
While the return values of such methods are easy to define, the parameters are not. Some commands parse streams, some local files, some use a webservice or local library api.
Example can be seen within the Stream manipulation example here

Collapsing (& Clustering)

Modifiers

Defined in an auxiliary interface, eg NavigatableSearchCommand.
Modifiers today are generic, built from the Navigator which holds the configuration, and the search command's results.
Defined behavior by an interface would result in the FastNavigationController becoming a ModifierNavigationController.

I do not see a need for delegation in the design as in most cases the parsing of the information required to construct the modifiers is specific to the search command.
Keep in mind that this is a subset to the Result Construction concern.

Utility behavior

Stay as is. Actual static methods can be moved to a SearchCommandUtility class.

 © 2007-2009 Schibsted ASA
Contact us