Overview

To provide a single unified box which "does the right thing" in handling what is currently done with (a) the go box, (b) the search page and several search boxes.

Solutions

• Photon Search - new search cgi with interwiki support
• Onebox Search/Go - small change to twiki/bin/search to add unified go/search support (this one scheduled for new release)

Requirements

I really like using the same tool for more than one purpose. In this case the "Go" box, or Go-To box as it has been called on another thread, should always be a search box, and that how it operates is based on how it used.

1. A user enters some text
   1. if there is no whitespace or other WikiWord breaking character (hyphens etc.), execute topic name search .
      1. if topic name search comes up with an exact match (case insensitive) automatically open the page. Think of Google's "I'm feeling lucky".
      2. if no exact match, display all similarily named pages along with a "create this page" link.
   2. if there is whitespace or breaking character execute simple text search of this web
   3. if text is prefixed with [keyword]: execute advanced [keyword] search where keyword is one of "text, topic, regexp, webs" etc.

• The label of the Go box, whatever it is called, should be a link to the standard WebSearch page.
• The keyword list should be a variable so that the list can be easily expanded to add custom searches. For example bugs: could search BugReport, todo: for FeatureToDo etc.

If you want to see this principle carried to very useful ends see Do The Right Thing and Dave's Quick Search Taskbar Toolbar Deskbar. Use these for awhile and you quickly begin to feel disabled if you have to use a computer without access to them. : )

-- MattWilkie - 13 Aug 2002

Recently, I hacked a Go-box, which offers limited search plus interwiki support. It works like this:

1. a select list for a wiki prefixes the text box for the page
2. if you select a wiki like "Go...", (which is the default) then
   1. try, if text is page in current web
   2. if text contains blank or : then assume first word in text to be a wiki from select list
   3. else, search for text in topic names (in all webs, better be the current)
3. if you select an explicit wiki, then
   1. try, if wiki / text exists locally
   2. else try, if wiki is an interwiki alias, then build link
   3. else show InterWikis page (better be search in all webs)

I primarily needed this to re-use the interwiki linking from non-twiki pages with URLs like this: http://my.host/go/C2/ExtremeProcess. To save TWiki's extra seconds, I hacked that in ... erm ... awk.

This could be rewritten and extended to handle Matt's ideas: If you select "Search..." and enter

1. word -> search topic name
2. word1.word2 -> check for local web/topic "word1.word2"
3. word1:word2 -> interwiki jump to "word1:word2"
4. word1 word2 -> full text search

If you select "Sth" where Sth is a local web, "cd" to that web

1. text -> search topic name
2. word1 : . word2 -> full text search

If you select "Sth" where Sth is not a web

1. any thing -> interwiki jump to "Sth:any%20thing"

-- PeterKlausner 14.Aug.2002

I've created a new script in /bin named smart_search that does the following:

• check if the user entered a WikiWord or Web.WikiWord combination
  • if so, send them to that page
  • if not, execute a normal web search using the entered text as search term

Recently I added a feature to look for certain 'special case'. E.g if the query starts with 'google:', a google search is done, if the query starts with 'whois:' a whois query is executed.

If there's a demand I can package it as sort of a plugin.

-- JeroenVanDongen - 15 Aug 2002

There is demand!

In fact, I would like another special case with a short keyword -- something like 'p:' to request a page name (topic) search within the current TWiki web. (And maybe 'pg:' to do a page name search throughout all TWiki webs.)

-- RandyKramer - 15 Aug 2002

How about handling the 'special cases' through the InterWikis page - so we have to figure out these messy URLs only once.
And as there are so many interwiki aliases, a selection list as reminder would be kewl like so:

Search... Acronym ApacheBug CVS CVSget Dictionary Google Group ISBN MSKB MozillaBug RFC - - - - Legacy.AbbeNormal Advogato APW ASTWiki C2 C2find Legacy.DolphinWiki EfnetCppWiki EfnetPythonWiki EfnetXmlWiki EmacsWiki FoxWiki HammondWiki LegoWiki MbTest MeatBall Legacy.MoinMoin MuWeb Legacy.OpenWiki OrgPatterns Legacy.PhpWiki Pikie PolitizenWiki PPR PyWiki Squeak TWiki UseMod VisualWorks Wiki WikiForum WikiOrg Wikipedia ZWiki

And add case insensitivity, so that isbn:1234567890 works, if you type it right away.
And: Matt's proposal of defaulting to name search sounds good to me.

-- PeterKlausner - Demanding 15 Aug 2002 ;- )

#Legacy.PhotonSearch

Solution: Photon Search

I've just finished a "smart search", similar to that you're discussing. I have just discovered this topic, so I suppose you will have interest in it. It's called PhotonSearch and follows these rules:

• If the contents of the box begin with an uppercase, its behaviour is the same as the standard Go box (you can also use the Web.TopicName notation).
• If the contents of the box begin with /, it performs a standard full text search.
• Else, it performs a custom search in the node names. Basically, it searchs the words typed in the node names (AND operation). If there is only one result, it redirects to the topic itself.

I've hacked the base templates to make the «Go» box refer to my CGI and works very fine. I've also hacked a sidebar for Mozilla and Opera and an easily accesible popup window to copy and paste topic names.

I'm attaching the CGI to this topic. If you're interested I can post the modified templates and the sidebar code. BTW, the CGI has a Perl sub which could be interesting to print the results. It makes easier writing custom searches, IMHO, because it decouples the search itself from the results print.

-- EstebanManchado - 01 Oct 2002

Comments

I'm interested Esteban, I'd especially like to see how you handled the sidebars. Initial observations:

• performance is quite good
• lowercase initial character: I would prefer it to default to local web only topic name search and use a prefix to search all public webs. Perhaps a question mark ? ? E.g. typing '?go is search' would search all public webs while 'go is search' would search the local web.
• I really like being able to type in a topic name with spaces, and in any order, and being taken directly to that page. (E.g. entering "is go search" opens GoIsSearch)
• Upper case initial: possible bug - always searches %mainweb%? perhaps something wrong with my template
• /full text search: again, should default to local web by default while prefixed with ? (or whatever) would go global.

-- MattWilkie - 01 Oct 2002

The idea of the initial uppercase is that you can use it as the regular «Go» box, i.e. you can use the topic name to refer to the current web or use the Web.TopicName syntax. What do you mean?

When I can, I will look at your feature requests (I did it the way it is now because my boss told me so, I will have to ask him if it's a good idea adding your suggestions) and create the PhotonSearch topic with some documentation and the modified templates for the sidebars and the popup window.

-- EstebanManchado - 02 Oct 2002

While I like the clean implementation and the customisable output, I don't think, that non-techie users will appreciate the ? / whatever syntax. The strategy for the direct go/search box should be IHMO:

• try simple things first
• full text search last
• leave even more complicated options to the search link

I attached my current hack, which works well, besides being a bad implementation. Its strategy is

1. If you supply one word, assume the current web as scope,
   else assume first word being the web | interwiki alias.
   The rest is the actual search text.
2. Return local page, if just one match of web/*text*.txt in file names
3. Return list of pages, if multiple matches for web/*text*
4. Return interwiki URL, if first word matches an alias like google, dict, isbn
5. Return search result for text otherwise

Actually, I have an almost complete description for a "Plugins.GoPlugin" lying around, if it weren't for the implementation... Anybody any usability feedback from "normal" users regarding the search box?

-- PeterKlausner - 02 Oct 2002

intial capital bug: I mean that if I am in Sandbox web and I type in "TestTopic4", which exists, I end up in "Main" With the message TestTopic4? does not exist, do you want to create this topic?". If I type in a fully qualified name, Sandbox.TestTopic4, I do indeed end up at the right page.

/ ? whatever syntax: While non-techie users won't appreciate the syntax as long is it doesn't get in their way it doesn't matter. E.g. So long as one is not forced to use /|? to achieve results, so what? Same argument applies to advanced options existing only on the WebSearch page.

Simple things first, full text last: yes I definately agree. Actually the only part I think I disagree with is that multiple words automatically mean a global search, which different from my ideas when I started this topic. I now think:

If the search text is: info bug render

1. do any of: InfoBugRender, RenderBugInfo, BugRenderInfo, etc... topics exist in the current web? (case and plural/insenstive)
   1. If only one yes, open page immediately,
   2. else list matching topics
2. do any of ... topics exist in any public web?
   1. One yes: open topic,
   2. many yes: list topics
3. If no topic name match in current web AND no topic name match in public webs, then
   1. full text search in local web, if no matches then
      
   2. full text search all public webs

Power users can shortcut to any step in the process by using the appropriate yet-to-be-defined prefix.

Interwiki links should only be processed as interwiki links if the first word is followed by a colon. Otherwise a search for "meatball spaghetti sandwich" would open MeatBall:SpaghettiSandwich instead of MeatballSpaghettiSandwich.

-- MattWilkie - 03 Oct 2002

/ ? whatever syntax: if they are shortcuts to specifically control the search "escalation" from simple to full, I certainly like that.

multiple words: Initially I figured, you wouldn't really need that. Why? I guess the typical TWiki intranet web should be <~ 1000 pages. So just one clever word should trim down the result list so much, that you can easily select the target page. That's the way I currently like to use it. But this largely depends on the fact, that the ugly ugly ugly results page of my hack is blazingly fast. You just type sandwich, then click on one of, say, 10 sandwiches. Faster than typing meatball spaghetti sandwich. (This should be a lesson for a clean implementation: present a minimal bullet list, don't waste precious time for date/ revision/ author sugar.)
Additionally: if it really falls through to full search, TWiki doesn't handle multiple words reasonably (yet).
One more thought: what should sth like twiki var return? I'm inclined to say: all *var* pages in TWiki web! Hmm... This could be a step after the simple name search. And enforced by the twiki.var syntax.
Still, now I can see this beeing useful. Especially if there are loose naming conventions with many permutations. You know, there was a foo page for windows, but was it WinFooBar or FooBarWindowsDoc or ...? So you type foo win to get that one page immediately.

interwiki links: in practice, I do need these a lot. So it would be a nuisance, to type google:foo bar instead of just google foo bar. I admit, it's a very techie artifact of the original intent: provide sth like http://your.site/go/ISBN/1-2345-67890-x. Still, if the interwiki check is put after all name checks before the full search, that should serve all needs. The explicit ":" syntax would be fall back, where the site alias matches local names; this nicely fits the / ? . approach to control search escalation.

-- PeterKlausner - 04 Oct 2002

Initial capital bug: Matt, what's the value of the action property of the main form in view.tmpl and search.tmpl? It should be

   action="% SCRIPTURLPATH%/photonsearch% SCRIPTSUFFIX%/% WEB%/"

(without spaces, of course) Perhaps you have omitted the /% WEB%/ part.

• Your right, I forgot % WEB%. Thanks Esteban -- MattWilkie - 07 Oct 2002

-- EstebanManchado - 04 Oct 2002

Some rules and enhancements have been added to PhotonSearch. Please see the page for details and further specific discussion.

-- EstebanManchado - 10 Jun 2003

Solution Onebox Search

I just created a "onebox" search box. What I did was just four lines of perl code in twiki/bin/search and changing the search box form.

• added a new paramater to searching "dogobox" if it is non-empty, then the normal search cgi will first check to see if there is a pagename which matches the search, and go there. If not, it performs a normal search.
• changed the 'go box' to point at the search cgi, and added the hidden paramater "dogobox"="yes"
• I've attached my modified twiki/bin/search cgi, it's just four lines of code near the bottom, search

    my $doGoBox       = $query->param( "dogobox" ) || "";

    # emulate "go" behavior during a search
    if (($doGoBox ne "") && &TWiki::Store::topicExists( $webName, $search ) ) {
      TWiki::redirect( $query, &TWiki::getViewUrl( "", $search ) );
    }

-- DavidJeske - 07 Jun 2003

Scheduled for Dakar, big usability enhancement. The spec needs to be finalized...

-- PeterThoeny - 04 Aug 2004

I like the way Wikipedia has a go and search button for the same input box. If the user hits the go button (or presses enter) then it trys to find a topic of that name, if none exists it displays the search page instead. The search page, if entered from either button displays a create/edit link for the search term in addition to search options.

I think it would be better to let the user select what happens rather than trying to second guess them. For instance, if you wanted to search for SVN you would be presented with the page instead and have to find the ref-by link.

-- SamHasler - 19 Aug 2004

I've always kind of leaned towards Wikipedia's approach as well. Then I came across a discussion on Legacy.JoelOnSoftware.com among seemingly informed users who were completely baffled by this interface. They never do figure it out. Go figure...

-- LynnwoodBrown - 30 Oct 2004

Postponed to EdinburghRelease as Dakar is now "security and perfomance only" changes.

-- MattWilkie - 06 Dec 2004

If the change is as simple as DavidJeske makes out above, perhaps this should be reconsidered for Dakar?

-- AntonAylward - 28 Jun 2005

sure, i'll be happy to apply any forthcoming patches...

-- WillNorris - 29 Jun 2005

I see that the current WikiMedia site makes use of TWO buttons and a combined CGI. A parameter tells the CGI whether it is a 'search' or a 'jump'. This must be to get around the cognitive problem that Lynwood pointed to.

-- AntonAylward - 29 Jun 2005

Commenting on Joel Spolsky's discussion (can't find the entry, but I remember vaguely) - why can Google get away with 2 search buttons without users getting confused? Must be the button label "I'm feeling lucky" that suggests playfulness, and does not cognitively conflict with "Search".

Suggestion: button 1 "Search", button 2 "I'm lucky".

-- ArthurClemens - 30 Jun 2005

-- BrianGupta - 21 Sep 2005

Raised as Tasks:Item691

-- MartinCleaver - 13 Oct 2005

there are a lot of good design ideas here in this topic; however, as peter mentioned, a final spec does really need to be worked out. if we can get a simple version of this into DakarRelease, here's a drastically simplified version that gets a lot of what's wanted done:

• if the search text is a WikiWord, it redirects to the view page
  • if the topic exists, that topic is displayed
  • if the topic doesn't exist, then view will display the oops with an option to create, and with a search performed for similiar topic names (note this is much more likely to yield results than a topic search when the search criterion is a WikiWord)
• otherwise, perform a topic text search for the words

• oh, i also made the "Jump" into a link to WebSearch (per matt's suggestion)

Index: lib/TWiki/UI/Search.pm
===================================================================
--- lib/TWiki/UI/Search.pm      (revision 7010)
+++ lib/TWiki/UI/Search.pm      (working copy)
@@ -75,6 +75,11 @@
     my $webName = $session->{webName};
     my $topic = $session->{topicName};

+    my $gobox = $query->param( 'gobox' ) || '';
+    my $search = $query->param( 'search' ) || '';
+    $session->redirect( $session->getScriptUrl( $webName, $search, 'view' ) )
+       if ( $gobox && TWiki::isValidWikiWord( $search ) );
+
     TWiki::UI::checkWebExists( $session, $webName, $topic, 'search' );

     # The CGI.pm docs claim that it returns all of the values in a

full patch at Tasks:Item691

-- WillNorris - 13 Oct 2005

This is an enhancement that got forgotten in Dakar, caused by duality of tracking in Tasks and Codev.

• actually, it was about dakar being a slimmer, performance-focused release. -- WN

There are probably other ones as well.

At this time I do not suggest to add above feature since this would imply a spec change. With the current spec, a string that does not match an existing Topic or Web.Topic name shows a list of similar topics. This is a very useful feature to quickly narrow down on topics where you remember part of the name. Especially useful in large webs. Above spec changes that to a fulltext search, which is useful too, but you loose the narrow down on topic feature.

Therefore: Postpone this to EdinburghRelease and define a spec that works well fo new users and power users. Such as:

• foo bar -- full text search, token
• "foo bar" -- full text search, literal
• UpperLower -- topic jump
• Web.UpperLower -- topic jump, different web
• Partial -- topic name search

Or:

• foo bar -- full text search, token
• "foo bar" -- full text search, literal
• j/ topicInAnyCase -- topic jump
• j/ Web.topic -- topic jump, different web
• j/ partial -- topic name search
• r/ cand(y|ies) -- full text search, regex

-- PeterThoeny - 13 Oct 2005

i very much agree with you that searching for a list of similiar topics is handy. the above patch continues to do that when what's typed in is a WikiWord that doesn't match a topic name. what the above patches failes to do, however, is support the OtherWeb.TopicName format, so it's not fully acceptable, either.

all in all, this feature continues to be deferred until EdinburghRelease; Tasks:Item691 has been deferred.

-- WillNorris - 13 Oct 2005

'This is an enhancement that got forgotten in Dakar, caused by duality of tracking in Tasks and Codev.'

no, it was never forgotten, we were simply waiting for someone to step up, and do the work. the people that have been actually working on DevelopBranch, have been flat out the whole time, and did not take on any more, simply because they did not have time to do so.

-- SvenDowideit - 13 Oct 2005

Argument to forgo the gobox: ReplaceGoBoxBySearchBox.

-- ArthurClemens - 14 Oct 2005

I very much like Will's implementation but think BasicSearchPlusJump is a better name for it. I elaborate on this in my response to ReplaceGoBoxBySearchBox

-- LynnwoodBrown - 15 Oct 2005

Here is an implementation on the wikipedia type function using only html & twiki (new to both so if you have a better way please post)

---

---

-- StephenSkinner - 16 Feb 2006

Well, doesn't look like the search quite works (almost). Someone who understands the syntax for that can u help? Sorry if posted this in wrong spot.

-- StephenSkinner - 16 Feb 2006

Commenting is disabled while not logged in