How to protect yourself from XSS when you allow people to post RAW embed codes?

Question

Hi.

Tumblr and other blogging websites allows people to post embeded codes of videos from youtube and all video networks.

but how they filter only the flash object code and remove any other html or scripts? and even they have an automated code that informes you this is not a valid video code.

Is this done using REGEX expressions? And Is there a PHP class to do that?

Thanks

Answer 1

Take a look at htmlpurifier to start. http://htmlpurifier.org/

Answer 2

Generally speaking, using regex is not a good way to deal with HTML : HTML is not regular enough for regular expressions : there are too many variations permitted in the standards... And browsers even accept HTML that's not valid !

In PHP, as your question is tagged as php, a great solution that exists to filter user input is the HTMLPurifier tool.

A couple of interesting things are :

• It allows you specify which specific tags are allowed
• For each tag, you can define which specific attributes are allowed

Basically, the idea is to only keep what you specify (white-list), instead of trying to remove bad stuff using a black-list (which will never be quite complete).

And if you only specify a list of tags and attributes that can do no harm, only those will be kept -- and the risks of injections are lowered a lot.

Quoting HTMLPurifier's home page :

HTML Purifier is a standards-compliant HTML filter library written in PHP.
HTML Purifier will not only remove all malicious code (better known as XSS) with a thoroughly audited, secure yet permissive whitelist, it will also make sure your documents are standards compliant, something only achievable with a comprehensive knowledge of W3C's specifications.

Yes, another great thing is that the code you get as output is valid.

---

Of course, this will only allow you to clean / filter / purify the HTML input ; it will not allow you to validate that the URL used by the user is both :

• correct ; i.e. points to a real content
• "OK" as defined by your website ; i.e. for example no nudity, ...

About the second point, there's not much one can do about it : the best solution will be to either :

• Have a moderator accept / reject the contents before they're put online
• Give the website's users a way to flag some content as inappropriate, so a moderator takes actions.

Basically, to check the content itself of the video, there is not much choice but have a human being say "ok" or "not ok".

About the first point, though, there's hope : some services that host content have APIs that you might want / be able to use.

For instance, Youtube provides an API -- see Developer's Guide: PHP.

In your case, the Retrieving a specific video entry section looks promising : if you send an HTTP request to an URL that looks like this :

http://gdata.youtube.com/feeds/api/videos/videoID

(Replacing "videoID" by the ID of the video, of course)

You'll get some ATOM feed if the video is valid ; and "Invalid id" if it's not

This might help you validate at least some URL to contents -- even if you'll have to develop some specific code for each possible content-hosting service that your users like...

Now, to extract the identifier of the video from your HTML string... If you're thinking about using regex, you are wrong ;-)

The best solution to extract a portion of data from an HTML string is generally to :

• Load the HTML using a DOM parser ; DOMDocument::loadHTML is generally pretty helpful, here
• Go though the document using DOM methods ; either, depending on your situation :
  • DOMDocument::getElementsByTagName, if you need to iterate over all elements that have a specific tag name ; might be great to iterate over all <object> or <embed> tags, for instance
  • Or, if you need something more complex, you could do an XPath query, using the DOMXPath class and its DOMXPath::query method.

And using DOM will also allow you to modify the HTML document using a standard API -- which might help, in case you want to add some message next to the video, or any other thing like that.

Answer 3

This will never be safe. Browsers have those funny little functionalities that help people display content of their pages even if html is messy. There are endless opportunities to get something through :)

check here to see the tip of the iceberg

What You need to do is use a single input for just a link and aditional inputs for width and height and filter those. THEN generate the object tag Yourself.

This might be safe.

Answer 4

I have implemented an algorithm for this for the company i work for. It works just fine. BUT, it was quite complicated to implement.

I would definitely check out HTMLPurifier to see if that works in an easy way for you. If you insist on doing it the old-school-way like I did, this is the basic steps:

1. First of ==> get friendly with stripos()

2. You have to make an recursive function to identify the start and stop tags for the widget, that includes all combinations of <embed></embed> or <embed/> (selfclosing) or <object></object> ... or <object><params>...<embed/></object>

3. After this, you have to parse out all attributes and params.

4. Now, all <object> tags should have <param> tags as child elements. You have to parse all of these to get all the data you need for finally generating a new embed or object tag. Escpecially the params and attributes that holds with, height, data source are important.

5. Now, you don't know if the attributes are enclosed by single or double-quotes, so your code has to be lenient in this way. Also, you dont know if the code is valid or well formed. So, It should be able to handle nested embed/object tags, embed tags that are not enclosed correctly etc etc... As it is user generatede content, you can't really know and trust the input. You will see that there are lots of combinations.

6. If you manage to parse the embeded element with all its attributes (or object element and its child params), the whitelisting of domains is easy...

My code ended up to be about 800 lines of code, which is quite large, and it was filled with recursive methods, finding correct stop and end tags etc. My alghorithm also removed all the SEO-text that often are included in the cut&paste embed-code, like links back to the site holding the widget.

Its a good excercise, but If i where you... Don't start walking this road.

Recommendation: Try find something ready made, open source!

Answer 5

http://php.net/manual/en/function.strip-tags.php and allow certain tags?

Answer 6

The most simple and elegant solution: Allowing HTML and Preventing XSS @ shiflett.org.
Using all sorts of "HTML purifier" is more than pointless. Sorry but I don't get people who like to use these bloated libraries when a much simpler solution is in hand.

Answer 7

If you're looking make your site "safe" from vulnerabilities, a white list approach is the (only) way to go. I would recommend safely escaping all user generated content, and white listing only markup you know is safe and works on your site. This means not only <B> tags, but also the flash embeddings.

For example, if you want to allow any youtube to be embedded, write a validation RegEx that looks for the embed code they generate. Refuse to accept any others (or simply display it as escaped markup). This is testable. Forget all this parsing nonsense.

If you also want to add vimeo videos, then look at the embed code they provide and accept that as well.

Ugh? I know this seems like a pain, but in reality it's much easier to write than some algorithm that tries to detect "bad" content in some sort of generic fashion.

After getting the simple version of the algorithm working, you could go back and make it nicer. You could "provisionally" accept content with URLs, scripts, etc. that don't pass your white list, and have an admin process to add approved regexes to your output escaping routine. This way legitimate users aren't left out in the cold, but you don't open your self up to attacks of this nature.