Do any search engine [spiders] take issue with escaped special characters in a page's meta tags, or do they all generally interpret them correctly? I suppose a quintessential example would be: & => & ...
My URLs look like "/pages.php?page=pageName" because I am using a database to supply the page's content. Does rewriting URLs to something like "/pageName" help search engines find the pages? If so how do I rewrite them? ...
Hi Folks, I was wondering if would it be possible to protect the "https://www.DOMAIN.com:8443" URL from search engines listing perhaps using the htaccess to redirect the 8443 port from "DOMAIN" to elsewhere? This is the VZPP Plesk - Login Login to Plesk. Please enter your login information. Username. Password. Interface Languag...
What's the important part of search technology? Distributed file system Cloud computing Storage system Ranking algorithms Search algorithms Or something else? ...
I have some div on page loaded from server by ajax, but in the scenario google and other search engine don't index the content of this div. The only solution I see, it's recognize when page get by search robot and return complete page without ajax. 1) Is there more simple way? 2) How distinguish humans and robots? ...
I've seen some websites highlight the search engine keywords you used, to reach the page. (such as the keywords you typed in the Google search listing) How does it know what keywords you typed in the search engine? Does it examine the referrer HTTP header or something? Any available scripts that can do this? It might be server-side or J...
Hi! My search engine uses the following function to calculate relevancy. private static int calculateScore(String result, String searchStr, int modifier) { String[] resultWords = result.split(" "); String[] searchWords = searchStr.split(" "); int score = 0; for (String searchWord : searchWords) { for (Strin...
I seen some awful forums with horrible searching. Its highly important to be able to find things in my db/app. I am considering writing my own but before i do what do you guys think are good tag search libraries? C# .NET or possibly ASP.NET NOTE: I do not want text searching, only tags. ...
I read some articles on Web crawling and learnt the basics of crawling. According to them, the web crawlers just use the URLs retrieved by the other web pages and going through a tree (a mesh practically) In this case how does a crawler ensures the maximum coverage. Obviously there may be a lot of sites that don't have referral links f...
As far as I'm aware, search engines (on the web) typically index only the "user facing" content of a web page. But what if you wanted to find pages(s) that contain a certain fragment of HTML or JavaScript. Is this possible? Do any web-wide search engines provide this feature? Or do you think that it would not be worth implementing? ...
Help Help! Google indexed a test folder on my website which no one save I was supposed to know about :(! How do I restrict google from indexing links and certain folders. ...
I'm trying to build a utility like this http://labs.ideeinc.com/multicolr, but I don't know which algorithm they are using, Does anyone know? ...
I have basic .ASP pages running on a windows box. I want to migrate them over to a Linux host and of course have to convert them to .PHP and keep like Google happy. If my site's fairly popular in the search engines how is the best way to do this? There's a lot out there on the net about this, I just want to make sure I'm going at it t...
I know you probably don't get many questions like this... I am working on a component that I want to be able to deal with the non-SEF URLs whilst SEF is enabled, whether it be the built-in SEF or something like sh404sef. Does Joomla store the ORIGINAL non-SEF URL anywhere ie. index.php?com=com_fred&view=homepage? I've found that any ...
I saw someone else ask this question recently but it was closed as spam before many decent answers appeared because they included links to their site. I am however interested in genuine answers to this on a more general basis. Avoiding talking about specific websites, what techniques and practises can people recommend to raising the le...
did somebody use this Extension in php or have an example of code, edit : i want to hear an opinion on this Extension if somebody deal with it. a tutorial with code , large examples .. ...
I want build a search service for one particular thing. The data is freely available out there, via free classified services, and a host of other sites. Are there any building blocks, e.g. open-source crawlers that I would customize - rather than build from scratch, that I can use? Any advice on building such a product? Not just techni...
I am trying to write a search engine that will give meaningful results when a user enters words that occur within a bunch of documents. For that I want to know how exactly a search engine works, what data structures and algorithms it uses to build indexes, store and query indexes etc. Plus some pointers on how to make a search engine giv...
Hi I don't know the official terms for what this is called, but would greatly appreciate any advice. The scenario is as follows: I have an index.html page on my domain to serve as a 'Under Construction' page, while I develop the website behind that using Drupal on an index.php page. Is there any way in which I can have Google index th...
I have heard that web crawlers are supposed to follow only GET requests and not POST ones. In the real world is this a valid assumption? ...