search-engine

Escaped characters in meta title/description/keyword content.

Do any search engine [spiders] take issue with escaped special characters in a page's meta tags, or do they all generally interpret them correctly? I suppose a quintessential example would be: & => & ...

Does rewriting a URL help a search engine find the page?

My URLs look like "/pages.php?page=pageName" because I am using a database to supply the page's content. Does rewriting URLs to something like "/pageName" help search engines find the pages? If so how do I rewrite them? ...

Protect Plesk login page from search engine eyes

Hi Folks, I was wondering if would it be possible to protect the "https://www.DOMAIN.com:8443" URL from search engines listing perhaps using the htaccess to redirect the 8443 port from "DOMAIN" to elsewhere? This is the VZPP Plesk - Login Login to Plesk. Please enter your login information. Username. Password. Interface Languag...

What's the important part of search technology?

What's the important part of search technology? Distributed file system Cloud computing Storage system Ranking algorithms Search algorithms Or something else? ...

Ajax page part load and Google.

I have some div on page loaded from server by ajax, but in the scenario google and other search engine don't index the content of this div. The only solution I see, it's recognize when page get by search robot and return complete page without ajax. 1) Is there more simple way? 2) How distinguish humans and robots? ...

How does a website highlight search terms you used in the search engine?

I've seen some websites highlight the search engine keywords you used, to reach the page. (such as the keywords you typed in the Google search listing) How does it know what keywords you typed in the search engine? Does it examine the referrer HTTP header or something? Any available scripts that can do this? It might be server-side or J...

Can I make this search relevancy function better without "doing advanced stuff"?

Hi! My search engine uses the following function to calculate relevancy. private static int calculateScore(String result, String searchStr, int modifier) { String[] resultWords = result.split(" "); String[] searchWords = searchStr.split(" "); int score = 0; for (String searchWord : searchWords) { for (Strin...

tag search library (ASP).NET

I seen some awful forums with horrible searching. Its highly important to be able to find things in my db/app. I am considering writing my own but before i do what do you guys think are good tag search libraries? C# .NET or possibly ASP.NET NOTE: I do not want text searching, only tags. ...

How does a crawler ensure a maximum coverage?

I read some articles on Web crawling and learnt the basics of crawling. According to them, the web crawlers just use the URLs retrieved by the other web pages and going through a tree (a mesh practically) In this case how does a crawler ensures the maximum coverage. Obviously there may be a lot of sites that don't have referral links f...

Do any search engines index web page source code?

As far as I'm aware, search engines (on the web) typically index only the "user facing" content of a web page. But what if you wanted to find pages(s) that contain a certain fragment of HTML or JavaScript. Is this possible? Do any web-wide search engines provide this feature? Or do you think that it would not be worth implementing? ...

Google indexed my test folders on my website :( How do I restrict the web crawlers!

Help Help! Google indexed a test folder on my website which no one save I was supposed to know about :(! How do I restrict google from indexing links and certain folders. ...

Visual similarity search algorithm

I'm trying to build a utility like this http://labs.ideeinc.com/multicolr, but I don't know which algorithm they are using, Does anyone know? ...

Search Engine way to take .ASP pages to .PHP

I have basic .ASP pages running on a windows box. I want to migrate them over to a Linux host and of course have to convert them to .PHP and keep like Google happy. If my site's fairly popular in the search engines how is the best way to do this? There's a lot out there on the net about this, I just want to make sure I'm going at it t...

How to find the non-SEF URL whilst SEF is enabled (Joomla 1.5)?

I know you probably don't get many questions like this... I am working on a component that I want to be able to deal with the non-SEF URLs whilst SEF is enabled, whether it be the built-in SEF or something like sh404sef. Does Joomla store the ORIGINAL non-SEF URL anywhere ie. index.php?com=com_fred&view=homepage? I've found that any ...

How can I Increase the number of visitors to a website?

I saw someone else ask this question recently but it was closed as spam before many decent answers appeared because they included links to their site. I am however interested in genuine answers to this on a more general basis. Avoiding talking about specific websites, what techniques and practises can people recommend to raising the le...

i look some example of using Swish Search Engine Extension in php

did somebody use this Extension in php or have an example of code, edit : i want to hear an opinion on this Extension if somebody deal with it. a tutorial with code , large examples .. ...

Are there any building blocks for a search engine that will scrape other sites?

I want build a search service for one particular thing. The data is freely available out there, via free classified services, and a host of other sites. Are there any building blocks, e.g. open-source crawlers that I would customize - rather than build from scratch, that I can use? Any advice on building such a product? Not just techni...

How to write a search engine?

I am trying to write a search engine that will give meaningful results when a user enters words that occur within a bunch of documents. For that I want to know how exactly a search engine works, what data structures and algorithms it uses to build indexes, store and query indexes etc. Plus some pointers on how to make a search engine giv...

Search engine redirecting

Hi I don't know the official terms for what this is called, but would greatly appreciate any advice. The scenario is as follows: I have an index.html page on my domain to serve as a 'Under Construction' page, while I develop the website behind that using Drupal on an index.php page. Is there any way in which I can have Google index th...

Web crawlers and GET vs POST requests

I have heard that web crawlers are supposed to follow only GET requests and not POST ones. In the real world is this a valid assumption? ...