SQL 2005 Full Text Search

Live Search from Desktop to Enterprise...

Fri, 19 May 2006 15:09:44 GMT

Thi week Microsoft anounced new Search products that extend Windows Desktop Search (WDS) to the Intranet (Corporate Enterprise Search) to links with its Internet search engine - MSN Search or soon to Windows Live Search. Below are some recent links to Microsofts product anouncments:

Worst Microsoft Product Name Ever (source: http://www.microsoft-watch.com/)
On Wednesday, the Microsoft branding department hit a new low. Microsoft has decided to christen the new Windows desktop search application (that can search your desktop/Intranet and Internet), due to go beta later this year, as "Windows Live Search." But there already is a Windows Live Search – the Internet search service that is currently in beta. Are the two products the same? No, the Softies said. Are they related? Nope. We've decided we're going to try using Windows Live Search A (for the desktop app version) and Windows Live Search S for the MSN service. We're not the only ones who think two "Windows Live Search" products are one too many.

Enterprise Search Offerings: Microsoft Versus Google
http://www.eweek.com/article2/0,1895,1963321,00.asp
http://ideas.live.com/programpage.aspx?versionId=9d28cdac-851d-4e72-96db-fd95dfc0f7c2
http://blog.seattlepi.nwsource.com/microsoft/archives/103541.asp?source=rss

New Microsoft Enterprise Search Products in the Pipeline
Microsoft Chairman Bill Gates on Wednesday is hosting 100 or so of the world's top CEOs as part of Microsoft's tenth annual CEO Summit. As part of the event, Gates is taking the wraps off several new Microsoft search offerings under development, including a new desktop search application, a new SharePoint Sever for Search SKU and a Knowledge Network add-on to SharePoint that is designed to harness the internal brain power of customers' "experts." The desktop search app is due to go to beta in the summer or early fall. The new SharePoint Server for Search and Knowledge Network products are slated to ship in fall 2006.
--
Enterprise Search Offerings: Microsoft Versus Google
http://www.eweek.com/article2/0,1895,1963321,00.asp

SharePoint: Microsoft's Sleeper Hit?
http://www.microsoft-watch.com/article2/0,2180,1962379,00.asp

Re-launch of SQL 2005 Full Text Search in 2006...

Wed, 17 May 2006 17:32:34 GMT

I've been pre-occupied with other issues lately and have not updated this blog in some time, so today and going forward I'll start updating this more frequently with both personal notes and views as well as technical information on both SQL Server 2000 Full-text Search and SQL Server 2005 FTS!

If there are any topics you want me to write about or provide more details, let me know by posting a comment and I'll reply with a new blog entry on that topic. It helps me to know that others are reading this and getting value out of it. I'm also going to move forward with a new website and blog from there as well, but for now I plan on continuing to blog here!

Thanks!
John

Desktop Search news items...

Fri, 04 Nov 2005 15:34:50 GMT

Google Desktop 2 has upgraded, and has supposedly graduated from being a beta. There is a new mapping plug-in in the side bar, but this in particular seems very beta quality to me. (It's very hard to use - the sidebar does not accept drag and drop, so you must type in where you want to go, and there is no "go" or "Search" button, so you can't simply paste with the mouse, since you must then touch the keyboard to use the enter key).

Ask Jeeves Desktop Search has been upgraded - see the blog for full details. One of the features they particularly claim is increased stability.

Yahoo have introduced a new mapping beta. This goes against the grain, and requires Flash, which I think is a backwards step, and also falls in to the same trap as Virtual Earth did of believing that America is the only location that counts.

Yahoo also slightly redeem themselves by offering a new mapping API, which does offer an AJAX based embedded map, and also a REST based geocoding service (again unfortunately only for America).
-- via Zmarties blog

The relatively new market for Desktop Search is approx. 1 year old, so these new upgrades at this time is not surprising, but so far no news on upgrades to Microsoft's Windows Desktop Search (WDS). Although, a lot of speculation in the microsoft.public.msn.search newsgroup about a Corporate / Enterprise / Intranet version of WDS...

New XQuery 1.0 and XPath 2.0 for Full Text Search

Fri, 04 Nov 2005 15:21:26 GMT

Michael Rys, SQL Server PM for XML features provides news on "XQuery 1.0 and XSLT 2.0 are now in Candidate Recommendation " as well as XQuery 1.0 and XPath 2.0 for Full Text Search:

W3C has released a new working draft for the XQuery 1.0 and XPath 2.0 Full-Text [Search] part:

SQL Server 2005 has RTM'ed !!

Thu, 27 Oct 2005 22:31:13 GMT

The Developer Edition of SQL Server 2005 is available for subscribers on MSDN. A few editions of Visual Studio 2005 are also available. I don't have any word on when other editions will be available.

via SQLTeam.com

Michael Rys - The light at the end of the tunnel is bright sunshine! - We have RTM'ed today!

Today, we have released SQL Server 2005 and VisualStudio 2005 and the .Net Framework 2.0 to manufacturing. This means the code is baked, done, shipped,... :-). It will be downloadable from MSDN within the day and available through the other channels in a couple of days. The following is just a teaser for all the cool things SQL Server 2005 will provide. Read the rather impressive list here.
via SQL PASS Blog

UPDATE: From Michael Rys blog:

Enterprise Readiness:

Multi-platform support: X32, X64 (WoW and native), IA64 Native, Numa Aware
Multi-instance support with 7.0, 2000 and of course 2005
Mobile, Tablet, Express (free), Workgroup, Standard and Enterprise – If we don’t have it, you don’t need it…
Table Partitioning – partition by key easily – drop a partition and add a new one, no need to delete millions of rows
Upgrade Advisor – (scan your existing 7.0 and 2000 servers and SP’s and get ready for an easy upgrade)
SQL Server Management Studio – built in the VS IDE and able to manage all services from one UI
Enhanced Full Text Search – Integrated backup/restore, thesaurus support, 100x faster for index build
ONLINE: index build, page/file restore, concurrent log and db backup, memory add, fast recovery (online during rollback) – all operations on-line…
Column level encryption
Off-by-default – reduced attack space
Code signing
Vastly improved diagnostics – Watson dumps allow us to fix 80%+ of customer issues – NO CUSTOMER REPRO -, Dedicated admin thread, Dynamic Management Views (see current server activity)
Database Snapshots – point-in-time query and reporting
IPV6 support
No more “general network error” – specific error messages for all errors.
Automatic replication of schema changes
Merge replication 2-5x faster
Built-in Oracle to SQL Server replication – the only way to go… :-).
HTTPS synch for mobile users

More to follow relative to SQL Server 2005 Full-Text Search features, functionality, tip & tricks! I'll use the new keyword "SQL2005FTS" with future postings. You heard it here first!

John Kane

SQL 2005 Full Text Search Blog
http://spaces.msn.com/members/jtkane/

New Vertical (Blog) Search Engines...

Mon, 17 Oct 2005 04:59:09 GMT

Sunday evening blog update... combining multiple entries into one posting... Enjoy! -- John

A review of Sphere - a new blog search engine targeting relevance as key differentiator

Fellow "former VC who returned to the bright side" Tony Conrad kindly gave me a preview of Sphere, ex-Yodel Search, at the Web 2.0 conference, and re-kindly gave me access to the beta version of this new blog search engine, “that uses an advanced algorithm to discover high-quality, relevant, and timely blog posts”.

Sphere Features

Software Only posts a short review of Sphere, the new blog search engine. What interests me most is the screen shot. Each result has a link to a 'profile' of the blog (or blog author). But even more interesting is the little histogram logo on the left of the page - is that a link to a trending mechanism?

SonicHealth is Yoogli

SonicHealth appears to be the Intelliseek competitor built on top of Yoogli's technology.

Sphere is Yodel and Waypath

Battelle points to Malik about Sphere. It turns out that Sphere based on the original Yodel Inc blog search engine I posted about a while back, but they have taken the two founders of Waypath on board as well.

Yoogli has ways of looking at content on the web that you will have to see to believe. Yoogli is delivering the next generation of search products by producing deep understanding of users and meaning.

Previewseek is a new entry in the alternative search engine wars. Their most unique feature is pinching description fields from Wikipedia and repurposing the wiki links as additional search terms. Quite a creative solution to providing context around what someone is searching for and allowing them to add additional terms to the query.

Google Maps Mashups for Movies & Theaters and Hotels:
MashMap - http://www.mashmap.com/ - It allows you to pick a city and then get a map of the movie theaters in that city... and use http://www.TripMojo.com to search for a hotel name, and view it in a Google map.

Building WinFS Solutions
Learn more about WinFS, the new relational file system for Windows. As an essential piece of Microsoft’s Integrated Storage strategy, WinFS bridges the gap between file systems and databases and provides a unified, rich programming platform for all data: structured, semi-structured, and unstructured. (intersting as this is what I've blogged about here since Dec. 2004). See also, http://www.winfsdirectory.com/WinFS_articles.htm

Quintura Search

Quintura is a new search technology that will be ready for download in October. The company has called their approach to searching “The Way People Search”. The new search software promises to help users find information on the web easier and faster by defining the context or meaning of the search term. Quintura also uses dynamic clusterization that builds or changes clusters on the fly depending on user input.The new search software will also offer visual semantic maps that let you add or subtract keywords and change context with the click of your mouse.

kbalertz.com + rss feeds

About 10 times a week, I get an email asking if I can add RSS feeds to www.kbalertz.com. What this means is that I've done a REALLY REALLY bad job showing people where the RSS feeds are, since they've been live on the site for over a year. Now all the RSS feeds are listed on a single page, which should make it easier to subscribe and the SQL Server 2000 KB Article RSS Feed.

DEEP WEB SEARCH
WHAT IT IS: Technology that boldly goes where no search engine has gone before.
WHY IT'S HOT: Google may have already indexed 8 billion webpages, but that's just the tip of the iceberg. Many more pages are hidden behind corporate firewalls or in databases waiting to be indexed. By some estimates, this so-called dark Web is 500 times bigger than the World Wide Web as we know it. Unlike the public Internet, however, it can't be retrieved by the usual Web crawlers. Instead, the information must be fed into search engines' mammoth databases using special retrieval techniques.

Before the advent of desktop search, our PCs were part of that invisible Web -- connected to the Internet but not indexed. File-sharing networks already search your PC for MP3s, but there are tricky privacy and security issues to resolve before your hard drive can join the visible Web. There are also millions of digitally transcribed books waiting to be connected. Ultimately, deep Web search could answer a direct question better than hundreds of links, because many of the most authoritative sources have yet to make it online.
KEY PLAYERS: Endeca, Glenbrook Networks, Google, IBM, Kozoru, and Yahoo.

SQL Server links & post-Yukon: Katmai

Tue, 11 Oct 2005 05:00:49 GMT

Counting occurrences of a substring within a string (UDF)

PASS 2005 conference overview with pics

Yukon Beta Forums > Full Text Search (viewed often, but few replies )

Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
By Arun Marathe (Published: July 2004)

Post-Yukon, the next version of SQL Server codename: Katmai
Next SQL Server Stop: Katmai
By Barbara Darrow CRN (Sep. 13, 2005 )
As in Katmai. That's the internal name for the successor to Yukon--the soon-to-ship SQL Server 2005. In a PDC interview with CRN, none other than Bill Gates said that the WinFS server implementation will come sometime in the "Katmai" wave. At that time the WinFS client and server technology will unify all the various data repositories, he said.
Microsoft SQL Server Katmai - Articles (only two as of 10/6/2005)

"Full text search" as defined by Wikipedia, the free encyclopedia

Blogs and Wiki's - convergence technologies...

Tue, 11 Oct 2005 04:46:18 GMT

Interesting convergence of Blogs & Wiki's from Portals and KM:

The Wiki and the Blog: Toward a Complex Adaptive Intelligence Community

Here is an article by Calvin Andrus of the CIA on how they can use blogs and wikis to help them change, The Wiki and the Blog: Toward a Complex Adaptive Intelligence Community, which is not a bad idea. As the abstract says in part:

“…The only way to meet the continuously unpredictable challenges ahead of us is to match them with continuously unpredictable changes of our own. We must transform the Intelligence Community into a community that dynamically reinvents itself by continuously learning and adapting as the national security environment changes.

Recent theoretical developments in the philosophy of science that matured in the 1990's, collectively known as Complexity Theory, suggest changes the community should make to meet this challenge. These changes include allowing our officers more autonomy in the context of improved tradecraft and information sharing. In addition, several new technologies will facilitate this transformation. Two examples are self-organizing knowledge websites, known as Wikis, and information sharing websites known as Blogs.”

Personalized Search, Wikis & Disruptive Technology...

Thu, 29 Sep 2005 04:49:25 GMT

Thoughts & quotes on the use of Wiki's for Personalized Search and how Technology can be disruptive / constructive and lead to more interesting possiblities...

Vanderbilt Guide To Using Google Personalized Homepage

Steven Cohen points out that Vanderbilt University’s wiki is now instructing how to add their staff and news feeds to Google’s personalized homepage. Guess the page is getting some serious usage. (via Findory)

Wikis: Disruptive Technologies for Dynamic Possibilities (ppt)
Gerry McKiernan, Science & Technology Librarian, Iowa State Univ.

"A wiki is a freely-expandable collection of interlinked Web 'pages', a hypertext system for storing and modifying information - a database, where each page is easily editable by any user..."

"Wiki is a ... collaborative space ... because of its total freedom, ease of access, and use, [and] simple and uniform navigational conventions...". "[It] ... is also a way to organize and cross-link knowledge...".

'Wiki wiki' (pronounced ‘wicky’, ‘weekee’ or ‘veekee’) (encyclopedia.laborlawtalk.com/Wiki#Pronunciation) is a Hawaiian term for 'quick' or 'super-fast' (en.wikipedia.org/wiki/Wiki).

Wiki Advantages:
Incorporates the assistance of experts, peers and other professionals
Annotate evolving issues where spare notes, thoughts, and file formats exist
Facilitate the exchange of ideas for small group projects
A more creative environment and expanding knowledgebase in project management
Level the playing field and allow all opinions to be heard
More productive than ‘back-and-forth’ emails & IM chats
Get everyone on the same page through the writable Web (Web 2.0?)
Harness the power of diverse individuals to create collaborative works
A forum for improving knowledge and advancing thought processes
Improve project management, and provide innovative reference repositories
for all aspects of planning, operation, and implementation

The Wiki for Finding Web Service and Open APIs
List of Web APIs
List of Common misspelled words
Microsoft's "Channel9 Wiki" lists the Windows Desktop Search IFilter components.
Using Wikipedia and the Yahoo API to give structure to flat lists
Wiktionary - A collaborative project to produce a free multilingual dictionary in every language, with definitions, etymologies, pronunciations, quotations and synonyms.
MSN quietly launched a preview version of MSN Filter (http://filter.msn.com/), a cross between a blog and a wiki?

"A Disruptive Technology is a new technological innovation, product, or service that eventually overturns the existing dominant technology in the market,despite the fact that the disruptive technology is both radically different than the leading technology and that it often initially performs worse than the leading technology according to existing measures of performance."
-- http://en.wikipedia.org/wiki/Disruptive_technology

Random thoughts & blogs...

Mon, 26 Sep 2005 02:50:07 GMT

Just a few of the many blog entries that caught my eye over this weekend...

MSN vs WebMachine

CNET takes an interesting look at Microsoft's history in the Web era and suggests that MSN may now be a key part of Microsoft's Web 2.0 strategy. They write that MSN is already being used as a platform for Windows software releases:

"The search service in Windows Vista, for example, shipped earlier as MSN Desktop Search. In addition, Internet Explorer features, like tabbed browsing, and protection against phishing techniques [...] shipped first through MSN."

What's more interesting is an old Microsoft memo from May 1995 that CNET dug up again, called "The Web is the Next Platform", and what that tells us about MSN's future. The memo was written by Microsoft engineer Ben Slivka (where is he now, I wonder?). Here's TheRegister's coverage of it from December 1998, around the time of the Microsoft anti-trust trial:

By 27th May 1995 Slivka is on version 5 of the document, so we can presume that if Bill Gates read an earlier version, he didn't understand it. Nor indeed does Slivka seem to understand the implications of what he's saying completely. He leads in with "The Web is an application platform (complete with APIs, data formats, and protocols) that threatens Windows."

Wow - that was an incredibly prescient thing to say in 1995! Swap "Siemens or Matsushita" with Google, and "WebMachine" with WebOS - and that's precisely the competitive threat Microsoft faces today.

In response Microsoft is integrating MSN into its platform product development group, where Windows is developed. It's too early to know how this will play out, but one thing's for sure - the Web is on an equal footing with Windows for Microsoft now. It took them 10 years to fulfill the destiny that one of their smart engineers, Ben Slivka, mapped out for them in May 1995. But will it be enough to stop the WebMachine?

Google Planimeter

It remembers your most recent position/zoom/map-type for the next time you visit.
Area computation is done using spherical geometry, so it's correct for large regions.

UPDATE: Quick Google Maps / Zip Code Tool

There is a useful tool at http://maps.huge.info/; enter a zip code and you'll get a refreshed page and an outlined map of the zip code for which you're searching. You can zoom in on it a big and still have the outline. Matt Cutts delves deeply into the how to use the Google Maps API, and how you could do something similar - http://www.mattcutts.com/blog/fun-with-zip-codes/.

Employee blogs - continued...

A great example of the power of employee blogs can be found at Mini-Microsoft right now. The blog does act as a virtual water cooler for hundreds of Microsoft employees, who feel comfortable venting their frustrations without fear of retribution. Note that both he (the anonymous blogger who writes the Mini Microsoft blog) and Balmer were profiled in BusinessWeek this week.

Employee blogging

Intelliseek and Edelman just released a new white paper on employee blogging (here - pdf). In this day and age, where more and more goods are being bought based on word of mouth, the paper analyses the reasons and motivations for employee blogging as well as the benefits when done right.

Search startup Intellext partners with Microsoft...

Sat, 24 Sep 2005 23:34:35 GMT

It looks like Windows Desktop Search (WDS) is starting to be integrated with other tools for a more personalized search enviroment:

Search startup Intellext partners with Microsoft

Software upstart Intellext has joined forces with Microsoft. Intellext have developed the automated Watson search tool, which will be available as an add-on for Microsoft’s MSN Desktop Search.

The Watson add-on is available for a free 30-day trial. However, a subscription to Watson costs $9.95 a month or $100 a year. The Watson software also is available at Intellext, though not as an integrated tool.

The Watson add-on for MSN Desktop Search is a sidebar that displays links to information as the user works with Microsoft Word, Microsoft PowerPoint, Microsoft Outlook and Internet Explorer.

You don’t have to type a query – Watson analyses your activities in the background and searches a wide array of sources for relevant information. Watson does not extract, store or transmit information any of this information.

Watson 2.0 Desktop & Search Assistant Alternative to Google Sidebar

CNet’s Elinor Mills reports that Watson 2.0, an AI based “context-sensitive” desktop search tool, is about to launch as an application which will be seen as an alternative to Google’s Desktop Search and Sidebar. Like Google Sidebar, the Watson 2.0 sifts through web history data, feeds from blogs and news sites visited, and desktop documents to serve relevant desktop search and web news information in real time - faster than Google Sidebar.

Microsoft Full-Text Search Technology summary

Thu, 22 Sep 2005 15:12:42 GMT

Below are two links to Microsoft's current list of products that use the same Search technology. While these resources have not been updated in many years, they still do provide basic comparison information. However, they do not reference SQL Server 2005 (Yukon), SharePoint Portal Server 2003, nor MSN Search or Windows Desktop Search (WDS), I'm sad to report...

Microsoft Full-Text Search Technologies (pub. 2001)
http://www.microsoft.com/technet/prodtechnol/sppt/sharepoint/evaluate/featfunc/mssearch.mspx

Microsoft SharePoint Portal Server 2001 Resource Kit:
Chapter 5 - Introducing Microsoft Full-Text Search Technologies
http://www.microsoft.com/technet/prodtechnol/sppt/sharepoint/reskit/part1/co5spprk.mspx

-- John

Random & Interesting blog items...

Sun, 18 Sep 2005 15:41:23 GMT

Some interesting and random blog entries found via Findory SQL Full Text Search and BlogBridge:

The Blook has Arrived... hackoff.com
Tom Evslin has launched his newest project - hackoff.com. It's a blook (an online book distributed as a blog). I've been watching this evolve and occassionally helping with some of the tech ideas from the sidelines. In addition to being awesome content (this is the book that every entrepreneur from 1997 - 2001 wanted to write), Tom is using (as well as inventing) lots of blog / Web 2.0 publishing technology into the experience.
via: Feld Thoughts

Internet Explorer Developer Toolbar Beta
The IE Developer Toolbar provides several features for deeply exploring and understanding Web pages.
-- Locate and select specific elements on a web page through a variety of techniques.
-- Selectively disable Internet Explorer settings.
-- View HTML object class names, ID's, and details such as link paths, tab index values, and access keys.
-- Validate HTML, CSS, WAI, and RSS web feed links.
-- Display image dimensions, file sizes, path information, and alternate (ALT) text.
-- Immediately resize the browser window to 800x600 or a custom size.
-- Selectively clear the browser cache and saved cookies.
-- Link to the Internet Explorer team weblog (blog), and other resources.
-- Display a fully featured design ruler to help accurately align objects on your pages.

Attensa - RSS Reader for Microsoft Outlook
The first of its kind, a RSS reader for Microsoft Outlook. Attensa for Outlook is the first RSS news reader and news aggregator to provide integration of RSS capability into Microsoft Outlook, that brings up-to-date news and content from website, blogs, and Podcast sites, directly into Microsoft Outlook.
via: Coolest-Gagets.com

Microsoft's 'troubling exits'
This week's Business Week cover story by Jay Greene is an exhaustive examination of the current state of Microsoft -- looking behind the recent series of employee departures to explore the company's disproportionate reliance on Windows and Office, its sluggish pace of major product releases, its layers of bureaucracy, and calls from inside and outside to institute reform.
via: SeattlePI.com Microsoft Blog

Two new SQL 2005 Full Text Search KB articles...

Fri, 16 Sep 2005 04:17:14 GMT

Two new SQL Server 2005 (Community Technical Preview or CTP) Full-text Search KnowledgeBase (KB) articles have been published this month:

SQL Server 2005 full-text search includes improved and updated noise word files
Microsoft SQL Server 2005 full-text search includes improved and updated noise word files. These noise word files are located in the following directory:

$SQL_Server_Install_Path\Microsoft SQL Server\MSSQL.1\MSSQL\FTDATA\

This directory is created and the noise word files are installed when you set up SQL Server 2005 together with full-text search support. Note If you used customized noise word files in earlier versions of SQL Server, the customized noise word files will no longer work after you upgrade to SQL Server 2005. To continue to use the customized noise word files, you must follow the steps in the "More Information" section.

BUG: The full-text search service incorrectly installs under the LocalSystem account on an SQL Server 2005 CTP-based computer

SYMPTOMS
You add the full-text search service to an existing instance of Microsoft SQL Server 2005 Community Technology Preview (CTP) 16 by using one of the following methods:

• Microsoft Office Maintenance Mode
• An upgrade installation of the full-text search service that is taken from an edition of SQL Server 2005 that is not SQL Server 2005 CTP.

When you do this, the full-text search service does not run under the domain account that is specified for SQL Server 2005 CTP.

Via Google site search of Support.Microsoft.com

Technorati tag SQL Server 2005

Google Blog Search uses XML feeds vs HTML...

Wed, 14 Sep 2005 21:59:22 GMT

Google Blog Search

"Blog Search is Google search technology focused on blogs. Google is a strong believer in the self-publishing phenomenon represented by blogging, and we hope Blog Search will help our users to explore the blogging universe more effectively, and perhaps inspire many to join the revolution themselves." (Google)
via InfoDesign

Google launches Blog Search

"Google Blog Search works by crawling XML feeds, rather than simply crawling the HTML output of a blog. Because feeds are, at least ideally, better structured than the published HTML of most blogs, it's possible to extract information like authorship of a post in a fairly consistent way. The potential for getting the value of structured data out of feeds is one of the reasons we've created technologies like the Six Apart Update Stream."
via SixApart ProNet Blog

Technorati tag Google Blog Search

MSN Search APIs now available...

Wed, 14 Sep 2005 03:48:27 GMT

I've been monitoring MSN Search and various blogs of developers, and while this may not be directly related to SQL FTS, it is related to Search:

Today, the MSN Search API's are live at http://msdn.microsoft.com/msn/, the Search API is available here. Other APIs available at MSN include; MSN Messenger, MSN Virtual Earth, MSN Search Toolbar API, and MSN Start.com API. Forums at Search Engine Watch Forums.

MSN on MSDN: http://msdn.microsoft.com/msn/
MSN Search on MSDN: http://msdn.microsoft.com/msn/msnsearch/
MSN Search Toolbar: http://addins.msn.com/devguide.aspx
Start.com API: http://start.com/developer/

via SERoundTable.com

The web (and SQL) Developer community will be interested to learn that the MSN Developer Center (Danny provided an overview of last week), is now live. The center contains information, developer toolkits, and links to forums for the following MSN services:

You'll also find a link to the MapPoint Developer Center, MSN Search Dev Team Blog, and MSN Messenger Dev Team Blog.

via: SEW Blog.

What will be interesting to follow is how many independent developers create new and exciting “mash-up’s” with these new Search API’s as MSN Search is still way behind Google APIs and somewhat behind Yahoo! APIs… Time will tell if MSN will be successful in this area as I too intend to develop some interesting applications that use the above API’s and (of course) SQL Server 2005 Full Text Search!

Technorati tag MSN Search APIs

-- John

Blog Mining and Text Mining...

Mon, 05 Sep 2005 17:56:15 GMT

I recently found this these links & resources for Blog Mining as mining blogs is a form of Text Mining:

Blog Mining Gets Real
"Like unstructured content captured on Web forms that never really gets used, blogs' explosive growth is generating raw data sets that your company really can't afford to ignore. At the beginning of the year blogs were considered by many industry watchers one of the top ten trends. It's becoming very clear that blog mining is certainly part of that mix."

Text Mining Encyclopedia of Terms
"This page lists the most significant terms for the Text Mining knowledge domain, organized in order of popularity. This list of terms was extracted from a random sample of Internet documents that provide information about this knowledge domain. There is also a taxonomy view, showing the list of terms organized into a hierarchical cluster."

Technorati tag Blog Mining, Text Mining
-- John Kane

New SQL Dev Team Blog - Customer Advisory Team

Sun, 04 Sep 2005 16:51:47 GMT

It seems that Mark Souza and his SQL Server Customer Advisory team (SQLCAT) have a new blog at http://blogs.msdn.com/sqlcat/default.aspx

Welcome to the blogsphere, Mark, Gert, and Lubor! I've listened to Mark give a high-end SQL Server presentation at a past MVP conference and he is impressive. Gert's one of the best & most frequent presenters at PASS, PDC and TechEd and Lubor's a long-time SQL PM within the SQL Dev group. All are SQL Server "heavy hitters" and have worked with high-end customers from all over the world...

I'm just curious if they've worked with any high-end customers who have implemented SQL Server 2000 FTS or SQL Server 2005 FTS...

UPDATE: I'm starting to add Technorati tags...
Technorati Tag SQL, SQL Server 2005

Full Text Search Adobe PDF files stored in SQL Server...

Mon, 29 Aug 2005 06:16:28 GMT

I've often been asked how to Full-text Index (FTI) Adobe PDF files stored in SQL Server 2000 and then use CONTAINS or FREETEXT to Full-text Search (FTS) the contents of binary files... Below is a TSQL script that I've often posted in the public fulltext newsgroup:

use pubs
go
if exists (select * from sysobjects where id = object_id('FTSTable'))
drop table FTSTable
go

CREATE TABLE FTSTable (
KeyCol int IDENTITY (1,1) NOT NULL
CONSTRAINT FTSTable_IDX PRIMARY KEY CLUSTERED,
TextCol text NULL,
ImageCol image NULL,
ExtCol char(3) NULL, -- can be either sysname or varchar(4)
TimeStampCol timestamp NULL
) ON [PRIMARY]
go

-- Insert data... (Note: Initalizing IMAGE column with 0xFFFFFFFF for use with TextCopy.exe)
INSERT FTSTable values('Test TEXT Data for row 1', 0xFFFFFFFF, 'doc', NULL)
INSERT FTSTable values('Test TEXT Data for row 2', 0xFFFFFFFF, 'xls', NULL)
INSERT FTSTable values('Test TEXT Data for row 3', 0xFFFFFFFF, 'ppt', NULL)
INSERT FTSTable values('Test TEXT Data for row 4', 0xFFFFFFFF, 'txt', NULL)
INSERT FTSTable values('Test TEXT Data for row 5', 0xFFFFFFFF, 'htm', NULL)
INSERT FTSTable values('Test TEXT Data for row 6', 0xFFFFFFFF, 'pdf', NULL)
INSERT FTSTable values('Test TEXT Data for row 7', 0xFFFFFFFF, 'xml', NULL)
go

-- Select data
SELECT * from FTSTable
go

declare @query varchar(200)
-- Insert HTML_file.htm into Row 5 !!
-- NOTE: Ensure the correct path for textcopy.exe!!
set @query = 'D:\MSSQL80\MSSQL$SQL80\Binn\textcopy /s '+@@servername+' /u sa /p /d pubs /t FTSTable /c ImageCol /f D:\SQLFiles\Shiloh\<HTML_file>.htm /i /k 5000 /w "where KeyCol=5"'
print @query
exec master..xp_cmdshell @query
go

declare @query varchar(200)
-- Insert MS_Word.doc into Row 1 !!
set @query = 'D:\MSSQL80\MSSQL$SQL80\Binn\textcopy /s '+@@servername+' /u sa /p /d pubs /t FTSTable /c ImageCol /f D:\JKane\Docs\<MS_Word>.doc /i /k 5000 /w "where KeyCol=1"'
print @query
exec master..xp_cmdshell @query
go

-- Select data
SELECT * from FTSTable
go

-- FTI
use pubs
go
exec sp_fulltext_database 'enable'
go
exec sp_fulltext_service 'clean_up'

-- Drop FTI, if necessary...
exec sp_fulltext_table 'FTSTable','drop'
exec sp_fulltext_Catalog 'FTSCatalog','drop'

exec sp_fulltext_catalog 'FTSCatalog','create'
exec sp_fulltext_table 'FTSTable','create','FTSCatalog','FTSTable_IDX'
exec sp_fulltext_column 'FTSTable','ImageCol','add', 0x0409, 'ExtCol'
exec sp_fulltext_column 'FTSTable','TextCol','add'
exec sp_fulltext_table 'FTSTable', 'activate'
go

-- Start FT Indexing...
exec sp_fulltext_catalog 'FTSCatalog','start_full'
go
-- Wait for FT Indexing to complete and check NT/Win2K Application log for success/errors..

select * from FTSTable
go
-- Search for search_word_here in HTML file..
select KeyCol, ImageCol from FTSTable where contains(*,'<search_word_here>') order by KeyCol
go
-- Search for search_word_here in .DOC file...
select KeyCol, ImageCol from FTSTable where contains(*,'<search_word_here>') order by KeyCol
go

-- Confirm FT Properties...
use pubs
go
sp_help_fulltext_catalogs 'FTSCatalog'
go
sp_help_fulltext_tables 'FTSCatalog'
go
sp_help_fulltext_columns 'FTSTable'
go
SELECT fulltextcatalogproperty('FTSCatalog', 'PopulateStatus')
go

-- Remove FT Indexes & Catalog & table..

exec sp_fulltext_table 'FTSTable','drop'

exec sp_fulltext_Catalog 'FTSCatalog','drop'

drop table FTSTable

The key to the above procdure is the use of TextCopy.exe to upload the files along with the column "ExtCol" defined as char(3) and populated with "doc" as the file extension. If you have file extensions longer than 3 characters, you should use varchar(4) and ".doc" (note the use of period in the actual value) and if longer file extensions, use syname as the data type.

Below are related KB articles for uploading and downloading (import & export) binary files (MS Word, Adobe PDF, Excell xls, PowerPoint ppt, etc) into a SQL Server table's IMAGE datatype:

258038 HOWTO: Access and Modify SQL Server BLOB Data by Using the ADO Stream Object

201785 HOWTO: Import FileSystem Data Using DTS and Index Server

309158 HOW TO: Read and Write BLOB Data by Using ADO.NET with C#

308042 HOW TO: Read and Write BLOB Data by Using ADO.NET with VB.NET

326502 HOW TO: Read and Write BLOB Data by Using ADO.NET Through ASP.NET

Precision and Recall for Vertical Search

Mon, 29 Aug 2005 05:57:53 GMT

Precision and Recall of Five Search Engines for Retrieval of Scholarly Information in the Field of Biotechnology
-- http://www.webology.ir/2005/v2n2/a12.html

Information Search Strategies on the Internet: A Critical Component of New Literacies
-- http://www.webology.ir/2005/v2n1/a9.html

Home: http://www.webology.ir/index.html

Photo Album: Random Photos

Fri, 12 May 2006 05:58:34 GMT

Random Photos

full_moon_eclipse

Mount Assiniboine

Opte_Internet_graph_700x700

Organ Pipe Cactus

Phoenix_Fractal

Portland Headlight before sunrise

The Long Look

Bent Palm tree on beach

Guy walking in foggy woods

julia-set

More Photos...

Book List: Book List

Mon, 11 Jul 2005 01:17:54 GMT

Custom List: SQL Server

Sun, 03 Jul 2005 19:46:55 GMT

SQL Server

Blog list: Blog list

Sun, 03 Jul 2005 07:11:19 GMT