Validation: Beyond email addresses/Phone numbers

Question

Several years ago I developed a website for a wholesale company that wanted to keep its online catalog (and wholesale pricing) private. Short of manually reviewing each submitted application, I was really stuck on how to accurately accomplish this.

This has also come up in other web projects requiring registration to access certain "slightly secret" information - including members of the press, and dealers. With most of these applications receiving very little in terms of actual submissions, it was easy to dismiss the task of automatically validating the form as impossible (or too much effort).

Lately however, there has been a very large increase in registrations, and it would make sense to automate this process if at all possible.

The site is developed in PHP, and I have tried the following:

• Scraping Dunn and Bradstreet against business phone
• Scrapping Yellow pages against phone/address
• Basic Regex

The regex is just standard, any-user validation. The scraping was unreliable (and questionably following the terms & conditions)

How do other developers deal with the issue of business/press validation, and how do they justify this to their clients?

Thanks loads

Answer 1

How about having them populate the name of their organization and use a web service to verify. I found this http://www.business.gov/about/features/api/business-licenses/, its only for US based companies but I would suspect their are others like it for other countries.

Answer 2

Another way is to ask them a mail address from their domain name, and send them an email. It does not work for each company, but you can use this tip to reduce the number of fraud attempts.

In order to be more efficient, you can also check the whois : http://www.webservicex.net/whois.asmx

If you're paranoid, you can also try to establish a connection with their server and follow this tutorial : http://www.coveryourasp.com/ValidateEmail.asp

Answer 3

Why not crowd source the solution?

This might sound weird, but bear with me. They all have something in common, they are all people of the same or similar industry. So for somebody to register and be approved, they need to be "vouched" for by somebody with preexisting or preapproved access. Since it is likely that reporters know other reporters, and wholesalers know other wholesalers all a person needs to do is register and request somebody to "vouch" for their legitimacy.

The exact implementation of course if up to you, you could set it so that any person needs at least 2 different people to vouch for them before their account is approved. The system will require some initial manual work, because you'll need to manually approve a few people, but as more people register the more they'll probably know somebody who already has access. Over time the system becomes more and more self-sufficient.

The only issue I see is if the number of users are small, and people who register don't know other people on the site.

Answer 4

Prefaced with: I'm not a lawyer nor e-commerce expert.

If this is an international deal, there is no universal standard to check against. Furthermore, you shun businesses whose licenses are pending or independent owners not interested in the registration bureaucracy. Automation only works if a universal standard or single datasource pool for whatever it is you're automating is in place.

Lacking that, you need an EULA to explain your business policy and to hold the user liable for fraudulent submissions. In lieu of the EULA, you may (depending on the jurisdiction and local laws) have the client complete an affidavit of some sort and require them to fax or scan/email it back to you before account activation. Follow up all submissions with a phone call to their business phone number as given to their local town council, BBB, commerce agency, or other local government, public service, or private organizational registrar just to confirm registration. The contact phone number, name, or email address they submit during registration on your site should only be something you double-check against, but never as a definitive contact resource.

Furthermore, additional verification can be done by requiring a credit card purchase of a general service fee of the absolute minimum the card requires (typically $1 it seems). Though never full-proof, this credit card transaction serves only to ID the client. See Craigslist, USPS.

I think I just answered your question with a "Ask a lawyer experienced in the field".

Answer 5

Once you get past the human vs. bot screening, you're looking for some way of distinguishing the merely curious non-target visitor (say, the proverbial 14-year old kid) from those you want to let in. As the other commenters say, there's no universal Turing machine way to identify, let alone evaluate, a purported reporter or business person who might be interested in your site.

One thing you might consider is posing the question "please briefly describe your interest in [the site] and the specific aspects of our products or services that interest you." Then develop some experience based heuristics for automated screening. First, run it through a spam filter, next score for keywords, etc.