Where do search engines start crawling?
What do search engine bots use as a starting point? Is it DNS look-up or do they start with some fixed list of well-know sites? Any guesses or suggestions? ...
search-engine
Counting number of views for a page ignoring search engines?
I notice that Stack Overflow has a views count for each question and that these view numbers are fairly low and accurate. I have a similar thing on one of my sites. It basically logs a "hit" whenever the page is loaded in the backend code. Unfortunately it also does this for search engine hits giving bloated and inaccurate numbers. I ...
What's the best Django search app?
I'm building a Django project that needs search functionality, and until there's a django.contrib.search, I have to choose a search app. So, which is the best? By "best" I mean... easy to install / set up has a Django- or at least Python-friendly API can perform reasonably complex searches Here are some apps I've heard of, please sug...
How can I find what search terms (if any) brought a user to my site?
I want to create dynamic content based on this. I know it's somewhere, as web analytics engines can get this data to determine how people got to your site (referrer, search terms used, etc.), but I don't know how to get at it myself. ...
What are some Search Servers out there?
I'm looking to find alternatives to Solr from the Apache Software Foundation. For those that don't know, Solr is an enterprise search server. A client application uses a web-services like interface to submit documents for indexing and also to perform search queries. Solr has other features built in like caching and replication. I belie...
How do you keep search engines from indexing text ads?
Is there any way to keep search engines from indexing text ads? These are basically stylized links. I have thought about generating images with text or using javascript to write them into a DIV. What is the best and most accepted way? ...
In-house full-text search engine for source code and SQL scripts
I like to run search engine on code, and if you aren't I recommend it (especially if you code in a team and you don't have all the lines of code in your head). When I vaguely remember some variable name or class name, but I can't remember where I saw it, I just run the search. Pre-indexed full-text search system is way faster than grepin...
What's a good source code search engine?
The codebase I work on is huge, and grepping it takes about 20 minutes. I'm looking for a good web-based source code search engine.. something like an intranet version of koders.com. The only thing I've found is Krugle Enterprise Edition, which doesn't post its prices... and if you have to ask, you can't afford it. I'd really prefer a...
Building a web search engine
I've always been interested in developing a web search engine. What's a good place to start? I've heard of Lucene, but I'm not a big Java guy. Any other good resources or open source projects? I understand it's a huge under-taking, but that's part of the appeal. I'm not looking to create the next Google, just something I can use to ...
Source code search with Google Desktop
Is there a indexing plugin for GDS that allows for source code search? I see some for specific types (Java, C++, ...) and one for "any text". These are nice, but I would like one that allows for many/configurable extensions (HTML, CSS, JS, VB, C#, Java, Python, ...). A huge bonus would be to allow for syntax highlighting (http://pygme...
How to Google for --depend?
The latest makefiles we've received from a third party vendor contain rules with --depend on the end of build rules, so I thought I would look it up on Google, but try as I might, I can't persuade it to display any pages with exactly the characters --depend I've tried surrounding it with quotes "--depend": I've tried the Advanced Search...
WikiRank Search
Hi, I am making a new concept in searching, I am calling lt as WikiRank, In this , a extension will be installed on firefox browser. This extension will see watch the user activity, and he will send visiting url and its visit time also user can give ranking to the material also. So in nutshell,. there will not be any change in user inte...
How can I hide content in a HTML file from search engines?
Say that I write an article or document about a certain topic, but the content is meant for readers with certain prior knowledge about the topic. To help people who don't have the "required" background information, I would like to add a note to the top of the page with an explanation and possibly a link to some reference material. Here'...
What URL do I post to for Live Search SOAP service?
Its possible I am just really really thick. However, looking over the SDK for the live search (MSN search) that uses SOAP, doesn't tell me what URL the service is at?? I can download SDKs for C# or VB which probably encapsulate, but that doesn't help me (I am using ruby). http://search.live.com/developer/ ...
What is a good search engine for embedding in a web site.
I am thinking of changing my web site's homegrown search engine. Before I break out Visual Studio, I wondered if anyone can suggest an alternative that gives me what I need. This being: Works with an ASP.NET site (is a .NET project) Creates a file-based index Fast search across hundreds or thousands of pages Performs word-stemming to f...
Search Engines Inexact Counting (about xxx results)
When you search in Google (i'm almost sure that Altavista did the same thing) it says "Results 1-10 of about xxxx"... This has always amazed me... What does it mean "about"? How can they count roughly? I do understand why they can't come up with a precise figure in a reasonable time, but how do they even reach this "approximate" one? I...
Microsoft Search Server 2008 Express Edition from Classic ASP or ASP.NET
We have a new installation of Microsoft Search Server 2008 Express Edition on one server and it's nicely indexing our intranet (on another server) which we can search from the provided search form on the search server. I'd like to customise the search results so that they actually look like our intranet has generated them and also place...
Anyone implemented Endeca with .NET? Would you recommend Endeca or FAST?
Which search engine would you recommend for a Commerce website? We have millions of products in a catalog and we want it to be as quick as possible. We would also want to make sure that the marketing driven through the search engine will be fast and effective. What are your opinions? ...
how to structure data for searchability
I am writing a search application specifically for music playlists. The genre and file format differs from playlist to playlist, and sometimes within the playlist there are differences too. There is also a concept of "synonymous" tags (e.g. urban would cover both hiphop and r&b, but not the other way around). Below is a list of search...
How can I use the Twitter Search API to return all tweets that match my search query, posted only within the last five seconds?
I would like to use the API to return all tweets that match my search query, but only tweets posted within the last five seconds. With Twitter's Search API, I can use the since_id to grab all tweets from a specific ID. However, I can't really see a good way to find the tweet ID to begin from. I'm also aware that you can use "since:" in...