views:

516

answers:

13

I'm using an '&' symbol with HTML5 and UTF-8 in my site's <title>. Google shows the ampersand fine on its SERPs, as do all the browsers in their titles.

http://validator.w3.org is giving me this:

& did not start a character reference. (& probably should have been escaped as &amp;.)

Do I really need to do &amp;?

I'm not fussed about my pages validating for the sake of validating, but I'm curious to hear people's opinions on this and if it's important and why.

+3  A: 

Well, if it comes from user input then absolutely yes, for obvious reasons. Think if this very website didn't do it: the title of this question would show up as do i really need to encode ‘&’ as ‘&’?

If it's just something like echo '<title>Dolce & Gabbana</title>'; then strictly speaking you don't have to. It would be better, but if you don't no user will notice the difference.

Andreas Bonini
+17  A: 

Yes. Just as the error said, in HTML, attributes are #PCDATA meaning they're parsed. This means you can use character entities in the attributes. Using & by itself is wrong and if not for lenient browsers and the fact that this is HTML not XHTML, would break the parsing. Just escape it as &amp; and everything would be fine.

HTML5 allows you to leave it unescaped, but only when the data that follows does not look like a valid character reference. However, it's better just to escape all instances of this symbol than worry about which ones should be and which ones don't need to be.

Keep this point in mind; if you're not escaping & to &amp;, it's bad enough for data that you create (where the code could very well be invalid), you might also not be escaping tag delimiters, which is a huge problem for user-submitted data, which could very well lead to HTML and script injection, cookie stealing and other exploits.

Please just escape your code. It will save you a lot of trouble in the future.

Delan Azabani
Andreas Bonini
Yes. But morally, should we be *relying* on the leniency and "nice" error handling of browsers? Or should we just write correct code?
Delan Azabani
@Delan: while I try to make every page I write validate, I understand from reading his question that he doesn't care about "morally". He just cares if it works or not. They are two different philosophies and both have their pros and cons, and there is not a "correct" one. For example this website doesn't validate, and yet it's a great website.
Andreas Bonini
Also, even if it was XHTML it wouldn't "break the parsing" unless the content type was set to application/xhtml+xml, which no one does because it's dumb that instead of gracefully handling an error the browser must quit. (That's why XHTML is being discontinued in favor of HTML 5)
Andreas Bonini
Jon Hanna
@Jon: I agree that it's in all cases **better** if your pages validate. I'm obviously not contesting that. The gray area is this: is it worth spending X hours of development time to make them validate, or is it better to take the slight risk that in the future, somehow, things may break? I personally think it's worth it, but I don't blame people who think it's not (such as Jeff Atwood) since it's such a gray area. One thing is certain: making pages validate costs money, and it's something important to consider.
Andreas Bonini
In this case, you are wrong. It doesn't take X hours or Y dollars to make it validate *for this particular case*. It's a simple case of `preg_replace('/`
Delan Azabani
Matthew Wilson
Gumbo
Until you've spent the X hours of development time making them validate (X should really be < 1 in most cases) then you don't know why they aren't validating. If you've been paying even reasonable attention to the code in the meantime, then why do you suddenly have nonsense output? You're going to have to investigate to make sure you don't have a serious bug, and then it's 5secs to fix it anyway. One of the big advantages of keeping things valid is that things suddenly being invalid can rapidly flag a subtle bug that would be missed if everything output was gibberish.
Jon Hanna
Making pages validate doesn’t really cost any money at all—at least not if you’re creating new ones. Maintaining invalid ones if things break costs money.
igor
Gosh-darn it. I missed the HTML 5 bit in the question!
Jon Hanna
I'll move it into an answer.
Matthew Wilson
Andreas Bonini
@Delan You say that HTML5 allow it unless *it looks like* a valid character reference. What do you mean by *looks like* exactly? Surely the standard is more precise than this.
Alexandre Jasmin
Delan Azabani
A: 

It depends on the likley-hood of a semicolon ending up near your &, causing it to display something quite different.

For example, in user input (say, if you include the subject of a forum post in your title tags) then you never know where they might be putting random semicolons, and it might randomly display strange entities. So always escape in that situation.

For your own static html, sure, you could skip it, but it's so trivial to include proper escaping, I don't know why you would.

Douglas
+1  A: 

Yes, you should try to serve valid code if possible.

Most browsers will silently correct this error, but there is a problem with relying on the error handling in the browsers. There is no standard for how to handle incorrect code, so it's up to each browser vendor to try to figure out what to do with each error, and the results may vary.

Some examples where browsers are likely to react differently is if you put elements inside a table but outside the table cells, or if you nest links inside each other.

For your specific example it's not likely to cause any problems, but error correction in the browser might for example cause the browser to change from standards compliant mode into quirks mode, which could make your layout break down completely.

So, you should correct errors like this in the code, if not for anything else so to keep the error list in the validator short, so that you can spot more serious problems.

Guffa
A: 

If you're really talking about the static text

<title>Foo & Bar</title>

stored in some file on the hard disk and served directly by a server, then yes: it probably doesn't need to be escaped.

However, since there is very little HTML content nowadays that's completely static, I'll add the following disclaimer that assumes that the HTML content is generated from some other source (database content, user input, web service call result, legacy API result, ...):

If you don't escape a simple &, then chances are you also don't escape a &amp; or a &nbsp; or <b> or <script src="http://attacker.com/evil.js"&gt; or any other invalid text. That would mean that you are at best displaying your content wrongly and more likely are suspectible to XSS attacks.

In other words: when you're already checking and escaping the other more problematic cases, then there's almost no reason to leave the not-totally-broken-but-still-somewhat-fishy standalone-& unescaped.

Joachim Sauer
@Downvoter: care to comment?
Joachim Sauer
Matt
@Matt: I see, and that would be reasonable. I was just assuming that no one writes entirely static HTML pages any more and that pretty much all content is at least somewhat dynamic (usually based on some database content). Maybe that assumption should have been made explicit.
Joachim Sauer
+4  A: 

HTML5 rules are different from HTML4. It's not required in HTML5 - unless the ampersand looks like it starts a parameter name. "&copy=2" is still a problem, for example, since &copy; is the copyright symbol.

However it seems to me that it's harder work to decide to encode or not to encode depending on the following text. So the easiest path is probably to encode all the time.

Matthew Wilson
It’s like quoting attribute values — you don’t have to, but you can’t go wrong if you do it all the time.
Paul D. Waite
+2  A: 

If the user passes it to you, or it will wind up in a URL, you need to escape it.

If it appears in static text on a page? All browsers will get this one right either way, you don't worry much about it, since it will work.

Dean J
A: 

if & is used in html then you should escape it

If & is used in javascript strings e.g. an alert('This & that'); or document.href you don't need to use it.

If you're using document.write then you should use it e.g. document.write(<p>this &amp; that</p>)

Alex
A: 

Could you show us what your title actually is? When I submit

<!DOCTYPE html>
<html>
<title>Dolce & Gabbana</title>
<body>
<p>am i allowed loose & mpersands?</p>
</body>
</html>

to http://validator.w3.org/ - explicitly asking it to use the experimental HTML 5 mode - it has no complaints about the &s...

AakashM
A: 

In HTML a & marks the begin of a reference, either of a character reference or of an entity reference. From that point on the parser expects either a # denoting a character reference, or an entity name denoting an entity reference, both followed by a ;. That’s the normal behavior.

But if the reference name or just the reference opening & is followed by a white space or other delimiters like ", ', <, >, &, the ending ; and even a reference to represent a plain & can be omitted:

<p title="&amp;">foo &amp; bar</p>
<p title="&amp">foo &amp bar</p>
<p title="&">foo & bar</p>

Only in these cases the ending ; or even the reference itself can be omitted (at least in HTML 4). I think HTML 5 requires the ending ;.

But the specification recommends to always use a reference like the character reference &#38; or the entity reference &amp; to avoid confusion:

Authors should use "&amp;" (ASCII decimal 38) instead of "&" to avoid confusion with the beginning of a character reference (entity reference open delimiter). Authors should also use "&amp;" in attribute values since character references are allowed within CDATA attribute values.

Gumbo
That's the HTML 4 spec you link to; from my reading of the (draft) HTML 5 spec, only *ambiguous* ampersands are disallowed. An ampersand followed by a space, for example, isn't ambiguous, and so (again by my reading) should be permitted - see my answer for markup that the HTML 5 validator accepts.
AakashM
@AakashM: Did I say anything wrong?
Gumbo
No; did I say you did? :)
AakashM
@AakashM: I’m not sure, it sounded like that.
Gumbo
+1  A: 

I think this has turned into more of a question of "why follow the spec when browser's don't care." Here is my generalized answer:

Standards are not a "present" thing. They are a "future" thing. If we, as developers, follow web standards, then browser vendors are more likely to correctly implement those standards, and we move closer to a completely interoperable web, where CSS hacks, feature detection, and browser detection are not necessary. Where we don't have to figure out why our layouts break in a particular browser, or how to work around that.

Specifically, if HTML5 does not require using &amp; in your specific situation, and you're using an HTML5 doctype (and also expecting your users to be using HTML5-compliant browsers), then there is no reason to do it.

Ryan Kinal
+8  A: 

Validation aside, the fact remains that encoding certain characters is important to an HTML document so that it can render properly and safely as a web page.

Encoding & as &amp; under all circumstances, for me, is an easier rule to live by, reducing the likelihood of errors and failures.

Compare the following: which is easier? which is easier to bugger up?

Methodology 1

  1. Write some content which includes an ampersands.
  2. Encode them all.

Methodology 2

(with a grain of salt, please ;) )

  1. Write some content which includes an ampersands.
  2. On a case-by-case basis, look at each ampersand. Determine if:
    • It is isolated, and as such unambiguously an ampersand. eg. volt & amp
       > In that case don't bother encoding it.
    • It is not isolated, but you feel it is nonetheless unambiguous, as the resulting entity does not exist and will never exist since the entity list could never evolve. eg amp&volt
       > In that case don't bother encoding it.
    • It is not isolated, and ambiguous. eg. volt&amp
       > Encode it.

??

LeguRi
Gumbo
Agreed! It just contributes to why I think the second methodology is a recipe for making mistakes.
LeguRi
... rephrased second case to "but you feel it is nonetheless unambiguous"
LeguRi
@Richard JP Le Guen: Yes, absolutely.
Gumbo
+1 for arguing from the dev's perspective. :)
pinkgothic
+1 for saying what I was trying to say but being much clearer about it ;-)
Joachim Sauer
A: 

A couple of years ago, we got a report that one of our web apps wasn't displaying correctly in Firefox. It turned out that the page contained a tag that looked like

<div style="..." ... style="...">

When faced with a repeated style attribute, IE combines both of the styles, while Firefox only uses one of them, hence the different behavior. I changed the tag to

<div style="...; ..." ...>

and sure enough, it fixed the problem! The moral of the story is that browsers have more consistent handling of valid HTML than of invalid HTML. So, fix your damn markup already! (Or use HTML Tidy to fix it.)

dan04