search-engine

Open Source Search Engine

I'm learning search technology and reading the paper about Google prototype. Is there an open source search engine written in C++ that is easy to read? ...

Hibernate Search querying ?

Greetings My domain model is as follows class Species { private String name; .. .. List<Family> families; } class Family{ private String name; private String locusId; .. List<Member> members; } class Members{ private String name; private String repTranscript; } I want to use 'Hibernate Search' to execute queries like o...

Does server-generated data get indexed be search engines?

Let's say I have an ASP.NET web application. I create an aspx page that shows a table containing users and email addresses. The user data is stored in a database, and when the page is requested by a logged-in user, html is generated to display the data. If the users requesting the page are not logged in, they are redirected to a sign-in ...

Why doesn't Googlebot index pages it crawls?

Three months ago I published my small personal website (~10 pages), submitted the URL to Google, and a few days later Googlebot showed up. Over the course of the last couple of weeks, Googlebot visits my website approximately twice a week and crawls maybe every other page. Ever since Googlebot first crawled my website, whenever I run a...

Blocking to be indexed

I am wondering is there any (programming) way to block that any search engine indexes the content of a website. ...

permission issue when editing a page using SharePoint Designer

Hello everyone, I am using Search Server 2008 based on SharePoint Server 2007 Enterprise with Windows Server 2008 Enterprise. I have tried the search function by using IE to access address http://, and everything is fine with search functions (e.g. I can search page content by keywords). Now I want to use SharePoint Designer to customi...

Exclude indexing of part of page in yahoo omnifind

Hi, im using yahoo omnifind as search engine in a site. It is working fine except that I cannot figure out how to exclude a certain part of a page. For example a tag cloud which is appearing on a lot of pages and therefore "pollutes" the serach results. Any ideas? Cheers Bjorn ...

Methods for preventing search engines from indexing irrelevant content on a page

I'm looking for ways to prevent indexing of parts of a page. Specifically, comments on a page, since they weigh up entries a lot based on what users have written. This makes a Google search on the page return lots of irrelevant pages. Here are the options I'm considering so far: 1) Load comments using JavaScript to prevent search engin...

SERPs: Google Scrapers

Hello all, How do sites that offer SERP tracking work? I mean, under the Google ToS - scraping is forbidden on search results. I have seen huge sites, I won't name any, offer SERP tracking services (paid) and these guys have not been spotted or Google choses to ignore them! Why? Clearly not a programming question, hopefully its ok as...

Search engine parser flow diagram

Do you guys know where i can find a search engine parser design diagram? I need to understand how it processes user input. what functions / algorithms are being used? conditions. etc. It doesn't have to be Google's. Updated question to search engine parser ...

Is there a mini-search engine or documentor for undocumented code?

I'm often working with undocumented code. Much of the time, PHPxref works great for this, but often there's code that isn't a php variable or function, so PHPxref won't search for it. For example, right now I'm working with Smarty templates, and it's not straightforward to search for the smarty variables in the rest of the code. A sea...

can search engine read jquery action

i want to popup a div with iframe content. dose search engine can read this when i'm using jquery to perform that ? OR is there a way to detect search engine in server side and remove the option of this popup ? thanks ...

lucene index file randomly crash and need to reindex

how you all deal wich such issue of occasionally need to reindex? what recommendation do you suggest to minimize this? ...

mysql - return results grouped in a column

I am working on a search/tag system. My original query I wrote was for when I was storing 'title', 'description' and a comma seperated 'tags' column in my article/video table. I have since realised the advantage of normalising my tags. I now have three table to deal with... tbl_Articles article_id title description content tbl_tag_i...

Do search engines process Javascript?

According to this page it would seem like they don't, in the sense that they don't actually run it, but that page is 2 years old (judging from the copyright info). The reason I'm asking this question is because we use Javascript to replace text on our site with other more typographically sound content. We're worried that this may affect...

Is implementing search for symbols impossible ?

I tried to Google for symbol £ but surprised with a response Your search - £ - did not match any documents. '£' must be a part of some font, any special reason that its not considered by Google ? EDIT: Even SO doesn't give any results... ...

Can I do a 301 redirect in robots.txt?

I have a site which has a whole host of legacy links, which now need to be mapped to new links. We need to update search engine results so that the legacy ones disappear and are replaced. Because of the CMS we can't do this programatically, but I was wondering if we could set up a 301 redirect in the robots.txt file, which would update...

How to have calendar events indexed by search engines

I'm developing a site for which one of the main components is an events calendar. For users, the calendar is offered in 3 formats an interactive AJAX web calendar (monthly view) an iCal subscription feed a downloadable PDF None of these seem particularly search-engine friendly. What would be the best way to make sure the individual...

What Language(s)/technology was used to make the old school search engines like Excite, Altavista, Lycos?

I'm just doing some research into the Language(s) and Technologies used to build the old school search engines like Excite, Altavista, Lycos and Ask Jeeves? What i would love to know is the Technology used to create them originally and the databases used?for example the first version of Google.com was built using Java and Python. Than...

How could I integrate Arachnode.Net web crawler with some of goodness in Searcharoo for a Search Engine Interface?

Dear friends, I am currently 70% of finishing my portal in asp.net using C# for a local government agency. The portal is quite big with integration with SQL Server 2008 Analysis, Reporting, and Integration services. Plus, I had successfully deployed and configured it into Windows Server 2008 under Active Directory environment. The next ...